This note aims to demonstrate how to deploy a couple of CloudFormation templates using GitHub Actions to create Amazon cloud resources. There are two tools we’re discussing. The first one is AWS CloudFormation, an infrastructure as a code tool to provision AWS cloud resources declaratively. The AWS cloud resources and their relationships are declared in an AWS CloudFormation template written in JSON or YAML format. The template is then deployed using the AWS CLI. The second one is GitHub Actions, which enables you to automate workflows and is a suitable tool for running the AWS CLI to deploy the AWS cloud resources. By utilizing both these tools, I will demonstrate how to create an automated process to deploy the IaC and provision AWS cloud resources repetitively.

There are four necessary components to understand while working on AWS CloudFormation. These are ‘templates,’ ‘parameters,’ ‘resources,’ and ‘stacks’. At the heart of AWS CloudFormation are the templates. These templates are in either JSON or YAML format. The parameters are a list of key-value pairs in JSON format to store values referenced in the AWS CloudFormation templates. Resources are the AWS cloud resources such as Amazon VPC and Amazon EC2 and their associations such as SubnetRouteTableAssocation. And finally, a stack is the deployed form of the template accessible on the AWS Console. A stack contains information such as the ID, deployment status, the IAM role used to deploy the stack, the resources deployed, the events while deploying the stack, and such. You can read more about AWS CloudFormation at AWS-Docs: CloudFormation.

Pre-requisites: Configure AWS Credentials for GitHub Actions To authenticate with AWS from your GitHub Actions workflow, you’ll need to configure AWS credentials. This use case requires two AWS IAM roles: one to authenticate to the AWS cloud from the GitHub workflow and the other to deploy the AWS CloudFormation template. Follow the steps in [this detailed guide] to set up the AWS IAM role required to authenticate to the AWS cloud and store them as a GitHub repository secret. The first IAM role (IAM_ROLE in the code repository), apart from the trust relationship policy (with GitHub, explained in the detailed guide), has a permission policy. The permission policy is stored in the GitHub repository as ./iam_policy/github_permission.json. Please replace ${replace-with-arn-of-IAM-role-stored-as-secrets.CFN_ROLE} with the ARN of the second IAM role.

The second IAM role (CFN_ROLE in the code repository) has two policies: a trust policy and a permission policy. The trust policy is stored in the GitHub repository as ./iam_policy/cloudformation_permission.json. Please replace ${replace-with-arn-of-IAM-role-stored-as-secrets.IAM_ROLE} with the ARN of the first IAM role. The permission policy must also have permission to manage the AWS cloud resources, as mentioned in the two template files stored in the templates folder.

I have the code stored in my GitHub repository: add-asg-lb-cloudformation, and you may follow along to learn more. The code repository is divided into four folders. The templates folder contains the AWS CloudFormation templates for deploying AWS cloud resources. The parameters folder contains the key-value pairs of properties used in the AWS CloudFormation templates. The iam_policy folder contains the IAM policies for the two roles used in this project. Finally, the .github/workflows folder contains the logic for deploying the templates with the parameters in a repeatable process.

For this example, I chose two templates. The first one (layer-zero.yaml) is to create an Amazon VPC with four subnets. Two subnets have an internet gateway attached to it while the other two have NAT gateways. The route tables for these four subnets have appropriate associations to enable communication to the internet. The template exports the VPC and Subnet IDs as output to be referenced in the second template. The second template (layer-one.yaml) creates a couple of security groups with associated ingress and egress rules. It also creates an Amazon EC2 auto-scaling group with scale-out and scale-in policies using AWS CloudWatch alarms and a load balancer.

The deployment process is automated via the .github/workflows/deploy-cfn.yml GitHub Actions workflow. I set a few defaults, like the working directory and the pipeline’s permissions. After setting those, the workflow has three primary steps.

Step 1: Checkout and Configure AWS credentials for the workflow The first step is to checkout the code, followed by configuring AWS credentials for the workflow. You’ll need to configure AWS credentials to authenticate with AWS from your GitHub Actions workflow. Please provide the IAM role created in the pre-requisites section.

Step 2: Deploy layer-zero AWS CloudFormation template The AWS CloudFormation deployment process is classified in a layered approach such that the infrastructure layers above depend on the layers below. In the command below, the AWS CLI calls the deploy function with the template, the stack name, parameters, and the IAM role ARN. The IAM role refers to the CFN_ROLE stored in the repository as a secret. You can read more about the deploy command at aws cloudformation deploy. Step 3: Deploy layer-one AWS CloudFormation template Layer zero exports a few properties required in the Layer one stack. Also, since the layer deploys IAM resources, a  --capabilities property is required. The rest of the options are the same as the layer zero stack. After the pipeline ran successfully, I navigated to AWS CloudFormation on the AWS Console. The AWS CloudFormation stack contained information about when it was created, the IAM role associated with it, the events, the resources, and so forth.

As I was working on automating the deployment process, I integrated Checkov to identify security vulnerabilities. You can read more at  -scan-with-checkov-and-github-actions. Although that link is to review Terraform code, the logic remains the same for AWS CloudFormation templates. You can navigate to the workflow file for details: .github/workflows/code-scan.yml.

Best Practices I implemented a few best practices while working on this project, such as: Store Source Code in a GitHub Repository: Leverage version control and collaboration features of Git and GitHub for managing your CloudFormation templates. Use Source Control to Manage Your Templates: Treat your CloudFormation templates as code and follow best practices for source control management, such as branching, committing, and code reviews. Create a Role for Stack Deployment: Follow the principle of least privilege and create a dedicated IAM role with the minimum necessary permissions for deploying CloudFormation stacks. Use Parameter Files with CloudFormation: Separate configuration values from your CloudFormation templates by using parameter files to improve maintainability and manage different configurations for different environments. Leverage Stack Outputs and Imports to Reference Cross-Stack Resources: Export and import output values across stacks to build modular and reusable components and create dependencies between stacks.

Challenges As I was working on this project, I came across a few challenges that I list below: Renaming a Stack in CloudFormation: - If you rename an existing CloudFormation stack, AWS treats it as an entirely new stack. You cannot simply update the existing stack with the new name. - Instead, you must create a new stack with the desired name and migrate all resources from the old stack to the new one. - This process can be time-consuming and error-prone, especially for complex stacks with many resources and dependencies. - To avoid this issue, it’s recommended to carefully plan and choose appropriate stack names from the beginning, as renaming stacks is a complex process.

Rollback and Stack Update Limitations: - If a previous CloudFormation stack deployment process encountered errors and the stack status is set to ROLLBACK_COMPLETE, you cannot directly update that stack with a new template. - In such cases, you need to manually delete the failed stack from the AWS Console or use the AWS CLI before you can deploy the updated CloudFormation template. - Manually deleting a stack can be a lengthy process, especially for stacks with many resources, as CloudFormation needs to delete each resource in the correct order while handling dependencies. - This limitation can delay your deployment process and introduce additional manual steps, which can be error-prone and time-consuming.

Lack of Clear Error Indication in GitHub Actions Workflow: - When a CloudFormation stack deployment fails in a GitHub Actions workflow, the error messages or indications may not be immediately apparent or clear within the workflow logs. - You may need to log into the AWS Console, navigate to the CloudFormation service, and inspect the specific stack’s events to understand the root cause of the failure. - This extra step of navigating to the AWS Console can be inconvenient, especially when troubleshooting issues within a continuous integration and continuous deployment (CI/CD) pipeline, where you expect clear and concise feedback within the workflow logs.

Parameter Handling in CloudFormation Templates: - Simply passing a parameter file to the aws cloudformation deploy command is not sufficient for CloudFormation to recognize and use those parameters. - The parameters must also be explicitly defined and referenced within the CloudFormation template. - Failing to include the parameter definitions in the template can lead to deployment errors or unexpected behavior, as CloudFormation will not be able to map the provided parameter values correctly. - This limitation can be confusing, especially for those new to CloudFormation, as it may not be immediately apparent that parameters must be defined in both the template and the parameter file.

By understanding and addressing these challenges, you can better prepare for and mitigate potential issues when deploying AWS infrastructure with CloudFormation and GitHub Actions. Proper planning, documentation, and adherence to best practices can help minimize the impact of these drawbacks and ensure a smoother deployment process.

Despite these drawbacks, I believe that AWS CloudFormation has its place in the IaC sphere, particularly if a customer wants to use purely AWS systems. That brings us to the end of this note. I hope you learned something new from it. Please do not hesitate to use the comments box below if you have any questions or suggestions.