Generally speaking, when we work with Terraform to provision resources in AWS Cloud, we have a few pre-requisites. These are -a remote backend to store the Terraform state file, a lock table, and IAM user credentials that Terraform will require to provision the resources. I say “generally speaking” because you can get away with the remote backend bit by pursuing the default (local) option. However, that is not recommended from a collaboration, automation, and security standpoint. Since this setting up of the prerequisites is a one-time activity, there is no need to automate that bit beyond a point. Yes, that is true that there will be multiple active Terraform configuration projects in an organization over some time. And in such a situation, having an automated process to configure the pre-requisites is a step in the right direction. Using the AWS CLI to create a backend -Amazon S3 bucket and an Amazon DynamoDB table and an AWS IAM user should be reasonably easy from a repeatability and consistency objective. Step 1: Install AWS CLI on your local and log-in. Download the installer and install it on your local. Once done, open a new windows command prompt and run “aws configure”. If this is a new setup, you will be requested an access key, secret access, etc. There is detailed information on that at AWS CLI Configuration basics docs.
Step 2(a): Create an Amazon S3 bucket The Terraform configuration uses an Amazon S3 bucket to store the remote terraform.tfstate file. There are two steps to this process - (a) create an Amazon S3 bucket and (b) encrypt the bucket. On the AWS authenticated command prompt, key in the following statement, where “skundu-terraform-remote-state-two” is the bucket’s name. On successful completion, the below statement in {} appears. Please note that the bucket name has to be globally unique. More info at aws-cli-create-bucket.
https://gist.github.com/kunduso/f575395837c5ae303291c48cb9edc6af
After creating the bucket, I added server-side encryption using the information available at aws-cli-put-bucket-encryption. https://gist.github.com/kunduso/1681cf3c766afd5611da9d14cddce01f
Note: If you struggled as I did with error messages then check out possible issues at Stack-Overflow.
Step 2(b): Create an Amazon DynamoDB Table A Terraform configuration uses the Amazon DynamoDB table to store the lock to prevent concurrent access to the terraform.tfstate file. I followed the steps listed here to create the command to use. https://gist.github.com/kunduso/cc445a7d9dccc6f0e6dc2f10af927852 Step 3: Create an AWS IAM user with the required permissions Creating an AWS IAM user for Terraform configuration is a multipart process that comprises of the following: Create a policy from a JSON file. Create a user. Create an access key associated with the user. Attach the policy to the user. All these steps are listed below. Here is an example policy file to provide restricted access to Terraform to only communicate with the Amazon S3 and Amazon DynamoDB table. https://gist.github.com/kunduso/bf94f1aa5e683ed66539458a9a44138d I store the same in a backend-role-policy.json file and use it in the step below https://gist.github.com/kunduso/b9e8c47e46216dbf239818b6c45a165e
After executing the above steps, we will have access to a few required variables while working with Terraform configurations. These are the Amazon S3 bucket name and location, the Amazon DynamoDB table name, and the AWS IAM user’s access-key
and secret-access
.
These values will be referred to in the backend.tf file and while executing “terraform init”, “terraform plan”, and “terraform destroy” steps.
Below is a descriptive example of a backend.tf configuration
https://gist.github.com/kunduso/6e1dba046e7889ec08f6b931175757ae
As you can see in the code above, I replaced the values of $(BackendBucketName) with the name of the Amazon S3 bucket, $(BucketRegion) with the region where the bucket’s region, and $(BackendLockTableName) with the DynamoDB table’s name. The value in $(PathToTFStateFile) is relative to the s3 bucket. E.g., if we prefer to store the terraform.tfstate file at the root of the bucket, the value of $(PathToTFStateFile) will be “terraform.tfstate”.
Here is an example of the IAM access key used in “terraform init” https://gist.github.com/kunduso/4816eeecabb7c81553dec939b204119f Note: The above Terraform command belongs to an azure-pipelines.yaml file. I am using Powershell to run them.
And that brings us to the end of creating AWS resources referred to in a Terraform project. I hope you found this post useful. Please let me know if you have any questions or suggestions. Until then, happy terraforming!
Note Two: In the example above, the AWS IAM user has access only to the resources required to manage the backend. The AWS IAM user’s access needs to be set accordingly depending on what resources need to be provisioned in a Terraform configuration. E.g., suppose the Terraform configuration is to create an Amazon VPC, subnet, route table, etc… In that case, the AWS IAM user will require additional permissions to manage those resources.
Note Three: make sure that the resources do not exist already in your AWS account. Also, log out of the AWS console, once the resources are created.