Nordstrom Sandbox Cluster for Rapid Validation, Initialization

Use to initialize and configure our CoreOS cluster. The control cluster as well as other "sub" clusters.

The cluster runs on AWS EC2 using Cloud Formation. cloud-init includes:

• docker (configure containers for local SkyDNS server)
• etcd
• fleet

For the control cluster, fleet is then used to deploy:

• logspout: log collection from all containers, forwards to syslog-gollector
• syslog-gollector: syslog server on each host, forwards to kafka
• registrator: automated DNS registrations based on container metadata
• skydns: local DNS server backed by etcd
• influxdb: time-series database, mainly used for metrics
• sysinfo_influxdb: collects metrics via CloudFoundry's sigar, sends to influxdb
• cadvisor: collects metrics about all running containers, sends to influxdb
• zookeeper: distributed consistent key/value datastore, used by several others
• kafka: high-volume distributed publish/subscribe message queue, uses zookeeper
• elasticsearch: distributed, lucene-based search, used for log aggregation
• logstash: log processing worker. reads from kafka & indexes into elasticsearch

One-time initialization

This will create IAM role, user, s3 bucket, elb listening on 80/443, a route53 zone, various route53 records as aliases to the elb, route53 record registering the s3 bucket as the zone apex alias.

cd visible
make clean clean-outputs
make launch/visible_stacks
sleep 300
make outputs
make launch/gtin_stack
cd ..

Initializing and configuring the control cluster

make deploy
$(terraform output fleet_env)
sleep 120
./control/launch_units.sh

Tests

After starting up the control cluster via Terraform, and setting your FLEETCTL_TUNNEL variable, run tests:

To install rspec:

bundle install

To run tests:

rspec -f d

Handy hints

You can test some changes to your cloud without needing to destroy and re-create. SCP your file to a host and:

sudo /usr/bin/coreos-cloudinit --from-file /tmp/user-data.yml

Most of our docker images are here: https://hub.docker.com/account/organizations/nordstrom/