[20160624174249] New AMIs and Docker Engine; Process CPU Config; App Load Balancer Security; Build Cleanup

Upgrade to Docker 1.11

With Docker finally removing the -f option (thanks to @asyazwan for bringing it to our attention with #781), we decided the best route is to upgrade to Docker 1.11.
This includes the the alpine imaged used by rack itself but also an AMI optimize for ECS Instances:

The current Amazon ECS-optimized AMI (amzn-ami-2016.03.c-amazon-ecs-optimized) consists of:

The latest minimal version of the Amazon Linux AMI
The latest version of the Amazon ECS container agent (1.10.0)
The recommended version of Docker for the latest Amazon ECS container agent (1.11.1)
The latest version of the ecs-init package to run and monitor the Amazon ECS agent (1.10.0-1)

See PR #797 for more information.

Process CPU Configuration

Thanks to @beedub you can now configure CPU shares for any process type.

The default is still the same – not explicitly changing this setting means that all processes share all of the CPU available on an instance. This is still the sensible default to get the most out of CPUs in the cluster.

But the ability to specify this opens up some more powerful use cases. You’d be able to set the number of CPU units to reserve for for a process.

For example, if you run a single-container web process on a single-core instance type with 512 CPU units specified for that container, and that is the only task running on the container instance, that container could use the full 1,024 CPU unit share at any given time. However, if you launched another copy of the same web process on that container instance, each task would be guaranteed a minimum of 512 CPU units when needed, and each container could float to higher CPU usage if the other container was not using it, but if both tasks were 100% active all of the time, they would be limited to 512 CPU units.

More information is available in the AWS documentation.

See #741 for lots of good discussion about the use cases and implementation.

Custom App Load Balancers Security Groups

Convox handles provisioning and configuring a load balancer based on the docker-compose.yml port mappings:

web:
  image: httpd
  ports:
    - 80:80

By default this load balancer will be exposed to the entire internet. Sometimes we want to tighten this to only allow connections from a single computer or single network.

You can now apply additional restrictions to this load balancer by assigning a custom security group to an app:

# get the VPC ID
$ convox rack
Name     convox
Status   running
Version  20160615203231
Region   us-east-1
Count    3
Type     t2.large

$ convox ps --app convox
ID            NAME      RELEASE                  SIZE  STARTED     COMMAND                           
cde9ac958f36  monitor   20160615203231-20160615  64    4 days ago  /go/bin/init sh -c api/bin/monitor
1845eb6ec788  registry  20160615203231-20160615  128   1 day ago   /docker-entrypoint.sh             
79e05a82d06d  registry  20160615203231-20160615  128   4 days ago  /docker-entrypoint.sh             
541d9b8509cb  web       20160615203231-20160615  128   1 day ago   /go/bin/init sh -c api/bin/web    
e9fd0ed66d41  web       20160615203231-20160615  128   4 days ago  /go/bin/init sh -c api/bin/web 

$ convox exec 541d9b8509cb env --app convox | grep VPC
VPC=vpc-f8006b9c

# create a new security group
$ aws ec2 create-security-group --group-name=office --description="office network only" --vpc-id=vpc-f8006b9c
{
    "GroupId": "sg-3bdaaf40"
}

# restrict ingress to a single subnet
$ aws ec2 authorize-security-group-ingress --group-id=sg-3bdaaf40 --protocol=tcp --port=0-65535 --cidr=24.75.239.0/32

# apply the security group to the app load balancers
$ convox apps params set SecurityGroup=sg-3bdaaf40
Updating parameters... OK

After this, the service will only be accessable to IPs from the specified subnet, blocking access from the internet at large.

See @MiguelMoll’s patch on #775 for more information.

Build Cleanup

AWS ECR has a default limit of 500 Docker Images, which you can eventually hit after many builds. Thanks to @awsmsrc on #773, convox now automatically deletes the very oldest builds after a successful new build. This both avoids hitting the service limit and saves money on storage costs.

As a reminder, Convox still retains the last 50 builds and releases to facilitate a rollback from a bad release:

$ convox builds
ID           STATUS    RELEASE      STARTED      ELAPSED  DESC
BOFEDWPIEEB  complete  RLBTJENIMQI  3 days ago   6s           
BVFZQPTJJNV  complete  RTAYZHUSGCJ  1 week ago   22s          
BKYKUURZKQU  complete  RONCLAIMWTP  2 weeks ago  49s          
BFUOFQOMAFI  complete  RDJQJEUAPTP  2 weeks ago  10s          
BHGAKQOJCCA  complete  RQOHWZJYDGG  2 weeks ago  5s       

$ convox releases promote RTAYZHUSGCJ
Promoting RTAYZHUSGCJ... UPDATING