This Week’s Program: July 11 - July 15

I am a veritable Thundurus this week. Because of my prowess with Cloud computing. I don’t know it’s a Pokémon I guess.

49b9b22164c3927d93c6947e7b75f9ebd6cdf592

In my CloudFormation template I add instructions to install the AWS CodeDeploy Agent. This is used to get code from CodeDeploy onto an instance.

41c49c64a8de9716521d339b8c3b1e0e6988760c

I build up a Deployment Group and choose a recent revision of sonic-sketches to deploy onto a machine.

cda4f095269f62dd4cfd6bf8bd7f11e644a9eed1

I bring back the CodeDeploy Appspec file, and begin a series of commits where I tweak the parameters and permissions of this file.

At this point, I’m bringing up CloudFormation stacks and tearing them down, each time tweaking the Appspec file. I can get my code onto the box. Each revision of the Appspec brings me closer to running the code. This is Monday.

81b35621e118735468e5c68f745889692e88684f

It’s Tuesday. I’m adding a hook for the CodeDeploy BeforeInstall event. Later, I would change this to AfterInstall and continue to tweak it.

8aa61334e3a437619349462bcece36f32d1d0c8f

I parameterize the GitHub revision commit to be deployed on initial stack creation.

Eventually, I am able to reliably use CodeDeploy to ensure that the sonic-sketches program is in place on the filesystem after stack creation. I am able to run sonic-sketches successfully.

Now to automate it further.

bb6087d68c924553293608c1981bb457665828fd

I parameterize the GitHub repository that houses sonic-sketches. Of course the default is mine, but if anybody wants to fork the repo and this cloud-formation template, they should be able to see similar success.

I also have have my nohup jackd script run during provisioning. After a typo fix I can just log-in to the box once the CloudFormation stack has been created and run lein trampoline run in my sonic-sketches directory and a song pops in to S3.

92f5af7aacac114e634e635b189fa7abd9875532

I create a new IAM User with access keys for the purposes of deploying through CodeDeploy. At this point, I’m just following along with this tutorial published by Amazon: Automatically Deploy from GitHub Using AWS CodeDeploy. Using the GitHub Auto-Deployment webhook/service works like a charm. When I push to GitHub, it is automatically deployed to my EC2 instance. This is Wednesday.

Then I get a wacky idea in my head.

576f7a8e2a0fe8ff57ddd85fa577511256cf82c8

I change the CodeDeploy tag filter to use the CloudFormation stack ID. This is what CodeDeploy uses to determine what instances to deploy to. By using the Stack ID tag, I can be sure that CodeDeploy only deploys to the instances within its CloudFormation stack.

I spend some time Thursday and today on this next commit.

ab8d28b078e5e3ef5c658abfe0a1c1ba2972a09d

When I pull up the CloudFormation stack for sonic-sketches I want to automatically create the AWS CodeDeploy webhook on the Git repository. This takes some doing.

First, I create a Custom Resource in CloudFormation. This resource represents the GitHub webhook. The Custom Resource is backed by an AWS Lambda function. This function uses the Node runtime in Lambda and uses the cfn-response module to respond to the CloudFormation custom resource: this module tells CloudFormation whether or not that custom resource was able to be created.

Within the template I type out the Lambda function code. After a bunch of trial and error, I put my finished Lambda fn in a gist. This was my first time really using ECMAScript® 2015 in anger and my first time really using NodeJS in several years. Here’s how this function works:

• The Custom Resource in CloudFormation calls this Lambda function and says “Hey, I’ve got a Custom Resource here. Can you make this for me?”
• This Lambda function pulls out a bunch of properties I’ve explicitly passed to the Custom Resource, like a GitHub personal access token with the admin:repo_hook scope, the CodeDeploy Application name and Deployment Group, and the IAM User keys.
• Using these properties, the function makes a POST request to the GitHub API to create a webhook, specifying that this is for the CodeDeploy service. (I find making a straightforward HTTP request in Node cumbersome).
• If the webhook is successfully created, CloudFormation is signaled that this Custom Resource creation was a success. Otherwise, it signals a failure.

I also build another IAM Role so that this Lambda function can write logs to CloudWatch.

After a bunch of trial and error, and watching the CloudWatch logs like a hawk, I can now successfully create a GitHub repository webhook for my CodeDeploy service as a resource within my CloudFormation stack.

Theoretically, this Lambda function should also be triggered when it’s time to delete this resource, but I found in my experiments that this branch of code was never called.

So now that I’ve overengineered the hell out of this I feel quite good about the results. This was my first exposure doing something with Lambda and it is awesome. I have now automated the deploy of my code to my AWS stack whenever I git push.

Next week, I need to make some changes to the application to support some of these new stack ideas. I also intend on setting up a scheduled job to generate a song at regular intervals.

CloudFormation, I choose you!
– Mark