This Week’s Program: June 27 - July 1

After the unfortunate clunker of last week I received a handful of very kind, encouraging emails. Thank you so much to those of you who reached out. Thanks to all of you for your support and for the time and attention you dedicate to my missives.

This week, I bounced back with aplomb and am pleased with my progress. Over the weekend I had a breakthrough with my struggles over cloud operations. The big mental hurdle I had to overcome was this: “It’s easy to throw away virtual machines.” Just put them in the garbage. Cloud computers are cheap and disposable.

Time was when you got yourself a private server somewhere you just hand-tuned that sucker to do everything you needed. Web server, mail server, ftp server… one machine did all that stuff and restoring a machine to a working state was an effort. The big revelation of cloud computing and IaaS is that your servers are cattle. Farmer wisdom says not to name livestock you plan to slaughter. Morbid but folksy.

It took me a couple of times to get a feel for it, but terminating an EC2 instance is easy and should be done frequently, with gusto. Once I accepted that cloud computers are more akin to a paper boat than a Spanish galleon, I became a lot less precious about my cloud.

651103066095ee08edff31e081cff22b13c62801

On Monday, things got off to a slow start as I get a better understanding of AWS::CloudFormation::Init. I’m feeling out the entire EC2/AWS vocabulary.

33a93fcfc30fe6677edc758d5dee83f2c6c36c07

Things get real. I’m actually running the formation.json template through the Cloud Formation tool and attempting to shell into the EC2 instance. Turns out, I need to make sure the Security Group for the Instance allows network ingress access over TCP port 22. So I need to make sure that Cloud Formation knows how to create the Security Group. In this commit I also expose Cloud Formation Outputs so I know where on the Internet my EC2 instance lives.

I’m able to successfully SSH into my EC2 Instance. I destroy and create the stack over and over again, drunk with power.

14f269f75a0e58df22f22aa67abe9348cc1ff78e

I populate the UserData attribute of the EC2 instance. UserData is used by EC2 to run commands on the instance at launch.

I use this to run through the same provisioning steps that I previously was using for the Packer AMI.

This script is picked up by Ubuntu’s cloud-init, which is a fascinating little tool. A short amount of time after the instance is built, I can SSH into my EC2 Instance and run which lein and get an answer.

In the commit message, I make a note about the next step.

f4d78fd029e089f7f79a5d0597b5d5d85baa8b2a

Here, I pull in the python_setuptools package so that I can easy_install the AWS CloudFormation Helper scripts. These are the scripts that work hand in hand with the AWS::CloudFormation::Init metadata, but that’s not why I’m pulling them in.

18aa74311ad66b3f67f52db382f36f7b95008642

That provisioning step is really important. So important that if the provisioning step fails, I want the entire stack creation to fail. To do this in Cloud Formation, you use the CreationPolicy attribute on a resource. You can then send a signal from the resource (in this case my EC2 Instance) to tell Cloud Formation if the creation of the resource was a success or a failure. In this commit, I’ve stated that I expect that this EC2 resource will receive one signal and the whole stack creation should fail if that signal isn’t observed within 5 minutes.

I use the cfn-signal utility from the previously installed helper scripts to send this signal as the last step of the provisioning process. Because my provisioning script is executed with /bin/bash -e, if there’s an error anywhere in the script the entire provisioning step should bail and this signal won’t be sent.

A few more stack creations and destructions and I’m feeling pretty good about the little machine in the clouds that will run sonic-sketches.

Next week, I’ll be focused on getting the sonic-sketches codebase onto this machine so it can be run. I’ll probably also be taking Monday, July 4th, off. Hope y’all have a great holiday weekend.

Happy Canada Day, too, if that’s your thing.

🇺🇸 Mark