In this tutorial, we are going to connect Terraform Cloud with a GitHub repository and leverage GitHub actions to provision Cloud infrastructure on AWS. We are going to combine IAC and GitOps approaches and manage our cloud resources via code.

Infrastructure as code(IAC)

The modern way to manage Cloud Infrastructure is to treat it like software. The number of resources that we have to handle is increasing daily along with their elasticity. The scale of the infrastructure is getting bigger and bigger every day and too difficult for people to manage with traditional ways.

Even more, the API-driven cloud environments provide many more options for automation than before. Managing our Infra as Code allows us to benefit from versioning, consistency, repeatability, collaboration, traceability while configuring complete solutions in environments that are changing rapidly every day.

GitOps

GitOps is the operational framework that allows us to take the best practices used for application development to infrastructure automation. By combining our Infa as code and GitOps approaches we can now safely operate on our infrastructure by leveraging the same CI/CD tools and automated test pipelines and deployments that we use already.

GitOps provides us the ability and framework to automate our infra provisioning. In practice GitOps is achieved by combining IAC, Git repositories, MR/PR, and CI/CD pipelines. First, we define our infra resources as code.

Then, we commit these in Git repositories that we handle as the source of truth for our cloud environments. When we need to add or perform an update on our environment, we have to modify our code and create a Merge/Pull Request to let our colleagues review our changes.

After validating our changes we merge to our main branch and let our CI/CD tools do their magic, apply our changes to our infrastructure environments.

Terraform Cloud

Terraform Cloud is a cloud-based platform that makes it easier for teams to work with Terraform. Provides an environment for us to run our infrastructure operations and keeps the shared state in the cloud. Allows us to set environment variables and secrets and integrates nicely with most version control and continuous integration systems.

GitHub Actions

GitHub Actions is an API on GitHub that allows us to orchestrate workflows based on different events that happen right in our GitHub repositories. With GitHub Actions, we can automate and customize our pipelines while GitHub is responsible for their execution. The added value is that our CI/CD and other workflows are now part of our repositories.

Terrafom Cloud + GitHub Actions Setup

In this article, we are going to connect Terrafom Cloud & GitHub Actions together to create our infrastructure management workflows. To follow along you’ll need:

Let’s get to it!

First step set up Terraform Cloud. We need to create a workspace for our infrastructure and connect it with our AWS account, via providing our AWS access credentials. In my example, I created a new workspace for an imaginary organization CourageAI named CourageAI-Infrastructure. Select API-driven workflow and create your workspace.

Next, we need to add our AWS access credentials, AWS_ACCESS_KEY_ID AND AWS_SECRET_ACCESS_KEY as environment variables in the configuration of the workspace. If you don’t have these, you can create them by following this guide.

Select the Variables option and set your env vars there, select also the option Sensitive.

Then navigate to User Settings on the top right for your user account. Select Tokens and Create an API token that we are going to use in our GitHub Actions workflow. Let’s name it GitHub Actions.

Moving ahead we have to set up a new repository that will hold our infrastructure status as Terraform files and our GitHub Action workflow that will apply our cloud resources in AWS. You can find the repository I created for this demo here.

Before proceeding, we have to add our Terraform Token to our repository. Click Settings, then Secrets, and create a new Repository Secret. Name it as you like, TF_API_TOKEN in our case, and insert your Terraform Token from the previous step.

Perfect, we are almost ready to try this out.

Let’s have a look at our only & simple main.tf Terraform file that declares a new EC2 instance on AWS and our terraform.yaml GitHub Actions file that declares the workflow to actually apply our infra to AWS.

Our main.tf file is rather simple. For this demo, we are going to create a tiny EC2 instance to validate that we can trigger this action by just committing and pushing our code to Github.

As you see we define a remote backend for Terraform and we add our organization and workspace names. Then we just creating a t2.nano EC2 instance with an ubuntu AMI.

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }

  backend "remote" {
    organization = "CourageAI"

    workspaces {
      name = "CourageAI-Infrastructure"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

resource "aws_instance" "server-1" {
  ami           = "ami-830c94e3"
  instance_type = "t2.nano"
  tags = {
    Name   = "server"
  }
}

The magic happens in our GitHub Actions workflow file. We trigger this on push to master branch events or on pull requests, and the idea is that whenever we have to spawn any new resource in AWS we prepare our Terraform files, we open a new PR targeting the master branch. After peer reviews and when we are ready to launch the resources, we merge our PR and trigger the workflow.

name: "Terraform"

on:
  push:
    branches:
      - master
  pull_request:

jobs:
  terraform:
    name: "Terraform"
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1
        with:
          # terraform_version: 0.13.0:
          cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

      - name: Terraform Format
        id: fmt
        run: terraform fmt -check

      - name: Terraform Init
        id: init
        run: terraform init

      - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        run: terraform plan -no-color
        continue-on-error: true

      - uses: actions/github-script@0.9.0
        if: github.event_name == 'pull_request'
        env:
          PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
            <details><summary>Show Plan</summary>
            \`\`\`${process.env.PLAN}\`\`\`
            </details>
            *Pusher: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;
              
            github.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })
      - name: Terraform Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      - name: Terraform Apply
        if: github.ref == 'refs/heads/master' && github.event_name == 'push'
        run: terraform apply -auto-approve

Basically, we set up terraform-cli using our TF_API_TOKEN, format our files, init terraform, execute the plan and finally apply the changes.

If the event triggered the action is a PR, we output information regarding the terraform plan that will help our engineers validate indeed that the plan is correct and will generate the required resources. If the event is merge to master we apply the Terraform configuration.

Time to see it in action! In our repo we’ll create a new branch and make a small change to main.tf file, for example, change the tag of the instance:

tags = {
    Name   = "server-1"
  }

git checkout -b update-tf
git status
git add main.tf
git commit -m "Modify ec2 instance details"
git push origin update-tf

Alright, back to our GitHub repository we open a new PR with our newly pushed branch.

And shortly, we see that our GitHub Action is triggered!

As discussed earlier we can also view the whole Terraform plan before merging to validate that Terraform will apply the required resources.

Our AWS instance will be created, exactly what we need. Time to merge and finally see this happening.

Merge and then navigate to Actions tab and check the workflow in action.

After a while, we validate that our GitHub Action was completed successfully and the logs for the terraform apply step display that is complete.

Head to AWS console and navigate to EC2 Dashboard on us-west-2 region. You should now see our instance up and running.

We are now ready to trigger any AWS infrastructure-related change via GitHub, how awesome is that?

If you followed along, one last step to actually terminate our demo instance. Head to Terraform Cloud console to our workspace and select Settings and then Destruction and Deletion. This destroys all the resources in the workspace, in our case our only EC2 instance. Click Queue destroy plan and on the next page Confirm & Apply. After a while, you’ll see your instance in terminated status.

That’s all folks, hope you enjoyed this demo. We applied the basic GitOps and Infrastructure as Code approaches with Terraform, Terraform Cloud, GitHub & Github Actions to set up quickly a workflow that handles our AWS cloud resources via a GitHub repository.