Anthony Harper

Control Plane Deep Dive

Overview

I came up with the idea for this project when I was learning how DevOps managed the AWS resources at one of my previous employers. The DevOps lead explained to me how they were using Terraform to manage infrastructure and my initial impression was "oh, so that means you just commit your configuration files and it deploys automatically?" but that was not the case. They were using source control to track changes to configurations, but they were not using CI / CD to automate the deployment of those changes. And that seemed like something that was very achievable. So I decided to build it.

I chose to go full AWS and to use HashiCorp Terraform. The first goal was to run Terraform once locally, and subsequently have every modification to my infrastructure occur in the cloud. I wanted absolutely no "ClickOps" interactions. I did end up performing one step manually, and that was my DNS registration on Route53.

The basic structure is that I have a CodeCommit source repository which triggers a CodePipeline when changes are pushed to my envs/prod branch, that CodePipeline initiates a CodeBuild, the CodeBuild installs Terraform, runs it to generate a plan (see Terraform docs for more info on plans), saves that plan as a build artifact for debugging purpose, and then deploys that plan. I built this structure with a main.tf that calls modules, the first of which is the pipeline module that handles the logic above. For each subsequent project added, a new module is added, allowing me to deploy multiple projects without any pollution of concern. In practice so far, this means that I have an infra module, a static-site module, and one more module for an upcoming project that's in progress.

What's a control plane?

If you're unfamiliar with the terminology, control plane / data plane are terms that allow us to speak quickly about the software that manages our deployments and resources and the software that actually runs on those resources and deployments. In traditional on-prem terminology, the RDBMS, web server, FTP services, etc. could all be considered control plane, whereas the web application built on top of those services would be the data plane. In my case, the Terraform configuration that deploys my AWS services (and those services themselves) are my control plane, and my website, its content, and all subsequent projects are the data plane.

Shared S3 state management

Terraform uses a concept of a state file to track which resources are deployed and not yet deployed, and from this state, it can build the plan for changes. It is also used for state-locking (to prevent competing operations on the same resources) and dependency tracking. For this project, because I wanted to initially deploy from a local terraform run and then have subsequent runs trigger in CodeBuild, I knew I'd need to share state between the two environments (my machine and CodeBuild), which meant I'd need to store my state file in S3. This is a relatively simple configuration but had the small complication of needing to run terraform with a local state once to get the S3 bucket that would store the state configured, and then clear the state and re-run with the remote backend configured.

IAM Permissions and roles

Terraform needs very broad privileges in order to be able to create and deploy AWS resources. For this purpose, the project specifies for itself a custom role and security policies that define exactly which resources it can create and modify. My initial run was under my own IAM account with broad permissions, but all subsequent runs are a "least-privilege" account. The privileges are still quite expansive, which is necessary, but as they are all defined explicitly, it is not possible that the project can accidentally spin up something I wasn't expecting due to a misconfiguration.

Terraform version parity

To ensure that my local runs and CodeBuild runs were using exactly the same version of Terraform, I pinned to version 1.16.1 and specified that in my versions.tf and buildspec.yml. So if I do a local run, I know it's going to produce the same result as the equivalent run will in CodeBuild.

Bringing it all together

The first local run of TF laid out the S3 state, CodeCommit repo, CodePipeline, and CodeBuild steps. I had to iterate a few times to get that cleanly ready to successfully execute TF in CodeBuild, but once the kinks (mostly pathing issues) were worked out, all further iterations were run as intended with configurations made and validated (using terraform validate), committed and pushed, then merged into envs/prod, which triggered automated deployment. That established the foundational infrastructure, and next came the projects that would run on it.