Home

AMI plugin

We're interrested if someone start up a new EC2 instance with a previously unseen AMI. Only alerts at the first occourence of an image.

Sample Alert

Rule: ami
Subject: ami-XXXXXXX
Alert: (u'i-<instance>', 1394359207000, [{u'service_name': u'jenkins-precise'}])

Configuration

Some instances are not interresting, because their AMI is generated by some automated process, eg. CI. The following configuration allows to list these machine tags:

"plugin.ami": {
  "allowed_tags": [
    "jenkins",
    "conversionservice_worker",
    "jenkins-precise"
  ]
}

ELB plugin

Alert if a "suspicious" port is forwarded by a load ballancer. For us, everything is suspicious which is not HTTP/HTTPS.

Sample Alert

Rule: elbs,
Subject: test-elb,
Alert: {
    'canonicalHostedZoneName': 'test-hostname',
    'numberOfInstances': 2,
    'listenerDescriptions': [
        {
            'listener': {
                'instancePort': 8443,
                'SSLCertificateId': None,
                'loadBalancerPort': 22,
                'protocol': 'TCP',
                'instanceProtocol': 'TCP'
            },
            'policyNames': []
        }]
}

Configuration

Not suspicious port ranges:

"plugin.elbs": {
    "allowed_ports": [80, 443]
}

IAM Plugin

Alert on

• new users, but filter out users created by automated processes, like CI or CloudFormation.
• if a monitored (= not filtered) user enters a IAM group.

Sample Alert

Rule: iam
Subject: zsellera
Alert: Groups the user has been added to: developers, devops

Configuration

Regexes as filters (what not to monitor).

"plugin.iam": {
    "allowed": [
        "^Deployment[\\w]{5}$"
    ]
}

Missing Instance Tag plugin

It's a best practice to add tags to a every EC2 instance. We'd like to see alerts if someone doesn't follow this best practice.

Sample Alert

Rule: missingtag
Subject: i-XXXXXXXX
Alert: n/a

New Instance Tag Plugin

Alerts when a new machine type is started (with a new instance tag).

Sample Alert

Rule: newtag
Subject: btcminer
Alert: i-XXXXXXXX

Security Group Alerter

Checks security groups, and alerts if a portrange is open to the public.

• "public" means open for 0.0.0.0/0.
• no not alert on every occourence: eg. opening a 443 is considered normal.
• Tipically ICMP isn't suspicious neither

Sample Alert

Rule: secgroups
Subject: sg-XXXXXXXX (default)
Alert: {
    'fromPort': 9080, 
    'ipRanges': [u'0.0.0.0/0'], 
    'toPort': 9080, 
    'ipProtocol': u'tcp', 
    'port_open': True, 
    'machines': [u'i-XXXXXXXX (123.123.123.123): zsati-app,zsellera@kiskiraly']
}

Configuration

"plugin.secgroups": {
    "allowed_protocols": ["icmp"],
    "allowed_ports": [22, 80, 443, 5666]
}

S3 ACL

This is a bit different than the previous plugins, since it use boto to traverse our S3 buckets, and search for publicly accessible files.

Sample Alert

Rule: s3acl
Subject: bucket:filepath
Alert: userid READ

Rule: s3acl
Subject: bucket:filepath
Alert: Everyone READ

Configuration

We store TerraBytes of data, checking every key is obvoiusly not feasable, so we do the stochastic way: pick a bucket randomly, and check some files in it - again, select these files randomly.

What if we have wide and deep hierarchies, like 5 "directory" inside every directory, for 5-level deep? If we choose with p=1/5 a new directory at every level, we'll find a file at the bottom of this hierarchy at every 7th run on average - the remaining 6 run was just a waste of resources. So our approach is a bit different:

• pick N subdirectories at every level randomly,
• decrease N as we get deeper in the hierarchy,
• always keep N >= 1,
• limit N for directories with many files.

The other configuration parameter set is about maintaining exceptions: there are buckets for public access, while others must be writable for specific accounts.

"plugin.s3acl": {
    "user": "<%= @aws_access_key_production %>",
    "key": "<%= @aws_secret_key_production %>",
    "visit_probability": 0.1,
    "visit_max": 10,
    "excluded_buckets": [
        "0103.static.prezi.com"
    ],
    "allowed": [
        {"uid": "b47...", "op": "READ"},
        {"uid": "b47...", "op": "READ_ACP"},
        {"uid": "b47...", "op": "WRITE"},
        {"uid": "b47...", "op": "WRITE_ACP"},
        {"uid": "b47...", "op": "FULL_CONTROL"},
        {"uid": "6aa...", "op": "READ"}
    ],
    "allowed_specific": {
        "logs": [
            {"uid": "65e...", "op": "FULL_CONTROL"}
        ]
    }
}