Finally, this is the last step in making the Terraform project manageable. I’ll add Terragrunt to the project. Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. It’s an excellent tool for making your Terraform code DRY and reusable. I’m starting with the same project as in the previous post. The current state of code is available here in the branch add_modules.

Prerequisites

Well, you need to install Terragrunt. You can find installation instructions here. That is all.

Adding Terragrunt to the back-office network and vms modules

I’m starting with the back-office module. What I do in this module will be easily copied to other modules.

Notice that common code in back-office/vms and back-office/network is related to the provider and state file configurations. This code is common for all modules. The only thing that differs is the backend.prefix. Its value differs from module to module.

So, the first thing is to make a global configuration on the dev level. To do that, I’m creating a terragrunt.hcl file in the dev folder:

generate "provider" {
    path = "provider.tf"
    if_exists = "overwrite_terragrunt"
    contents = <<EOF
provider "google" {
  project = var.project_id
  region  = var.region
  zone    = var.zone
}

terraform {
  required_version = ">=1.5.7"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "4.77.0"
    }
  }
}
EOF
}

remote_state {
    backend = "gcs"
    generate = {
        path = "backend.tf"
        if_exists = "overwrite_terragrunt"
    }
    config = {
        bucket = "terraform-states-network-playground-382512"
        prefix = "terraform/state/dev/${path_relative_to_include()}"
    }
}

terraform {
    extra_arguments "common" {
        commands = get_terraform_commands_that_need_vars()

        arguments = [
            "-var-file=${path_relative_from_include()}/common.tfvars",
        ]
    }
}

This file has three sections. The first section is generate "provider". This section will generate a provider.tf file in each module. Suppose it already exists and is managed by Terragrunt. In that case, it will be re-generated if there are any changes to the configuration. This file will contain provider configuration.

The second section is remote_state. This section will generate a backend.tf file in each module. This file will contain state file configuration. Notice that prefix is set to terraform/state/dev/${path_relative_to_include()}. This setting will store the state file in the terraform/state/dev folder, and each module will have its subfolder. That is important because each module will have its state file.

The third section is terraform. This section will add extra arguments to Terraform commands. In this case, it will add -var-file=${path_relative_from_include()}/common.tfvars. So, when I run any Terraform command requiring variables, Terragrunt will add this argument. That allows me to have one common.tfvars file in the dev folder, and all modules will use it.

Now, I need to add the common.tfvars file to the dev folder:

project_id = "network-playground-382512"
region = "europe-west1"
zone = "europe-west1-b"

I need to add a terragrunt.hcl file to each module to use this. I’m adding it to the back-office module:

include {
    path = find_in_parent_folders()
}

Now, I can remove the provider.tf and terraform.tfvars files from the back-office module. This time, I’ll be running terragrunt plain instead of terraform plan in the back-office module:

terragrunt plan
Acquiring state lock. This may take a few moments...
google_service_account.back_office_fw_sa: Refreshing state... [id=projects/network-playground-382512/serviceAccounts/back-office@network-playground-382512.iam.gserviceaccount.com]
module.back-office.module.vpc.google_compute_network.network: Refreshing state... [id=projects/network-playground-382512/global/networks/back-office]
module.back-office.module.subnets.google_compute_subnetwork.subnetwork["us-central1/back-office-private"]: Refreshing state... [id=projects/network-playground-382512/regions/us-central1/subnetworks/back-office-private]
module.back-office.module.subnets.google_compute_subnetwork.subnetwork["us-central1/back-office"]: Refreshing state... [id=projects/network-playground-382512/regions/us-central1/subnetworks/back-office]
module.back-office.module.firewall_rules.google_compute_firewall.rules_ingress_egress["back-office-icmp"]: Refreshing state... [id=projects/network-playground-382512/global/firewalls/back-office-icmp]
module.back-office.module.firewall_rules.google_compute_firewall.rules_ingress_egress["back-office-iap"]: Refreshing state... [id=projects/network-playground-382512/global/firewalls/back-office-iap]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

Running the terragrunt plan in the back-office/network module will generate two files: provider.tf and backend.tf, according to the terragrunt.hcl file in the dev folder.

For the back-office/vms module, I need to add the terragrunt.hcl file to the back-office/vms folder:

include "root" {
  path = find_in_parent_folders()
}

dependencies {
  paths = ["../network"]
}

dependency "network" {
  config_path = "../network"
}

inputs = {
  vpc_back_office_id = dependency.network.outputs.vpc_back_office_id
  vpc_back_office_subnetwork = dependency.network.outputs.vpc_back_office_subnetwork
  back_office_fw_sa = dependency.network.outputs.back_office_fw_sa
}

Now, this terragrunt.hcl not only includes files from parent folders but also defines dependency on the ../network module. That means that the back-office/vms module depends on the back-office/network module. A network needs to be created before virtual machines. That is important because the back-office/vms module needs to know some outputs from the back-office/ network module. The section inputs defines inputs for the back-office/vms module. These inputs are outputs from the back-office/network module. The last thing allows me to remove in addition to provider.tf and terraform.tfvars, and inputs.tf files from the back-office/vms module.

I need to add additional variables to the variables.tf file in the back-office/vms:

variable "back_office_fw_sa" {
  description = "back office firewall service account"
  type        = string
}

variable "vpc_back_office_id" {
  description = "back office vpc id"
  type        = string
}

variable "vpc_back_office_subnetwork" {
  description = "back office vpc subnetwork"
  type        = any
}

Running terragrunt plan should give me no changes needed. Also, I can now run terragrunt apply instead of terraform apply in back-office/vms module.

What I did in the back-office module, I need to do in the service and strorage modules. Simple. Remember to properly define dependencies and inputs in other vms modules.

When it comes to the peering module, I need to add the terragrunt.hcl file to the peering folder:

include "root" {
  path = find_in_parent_folders()
}

dependencies {
  paths = ["../back-office/network", "../services/network", "../storage/network"]
}

dependency "back-office" {
  config_path = "../back-office/network"
}

dependency "services" {
  config_path = "../services/network"
}

dependency "storage" {
  config_path = "../storage/network"
}

inputs = {
  vpc_back_office = dependency.back-office.outputs.vpc_back_office
  vpc_services = dependency.services.outputs.vpc_services
  vpc_storage = dependency.storage.outputs.vpc_storage
}

That will ensure that the peering module will be created after the back-office, services, and storage modules. Also, it will provide inputs for the peering module. These inputs are outputs from back-office, services, and storage modules.

Interesting would be generating a graphical representation of the dependencies:

terragrunt graph-dependencies | dot -Tpng > dependencies.png

dependencies

The terragrunt graph-dependencies command will generate a graph of the dependencies between modules. The dot -Tpng > dependencies.png command will convert the graph to a PNG file, that is graphviz. That is a great way to visualize dependencies between modules.

Try from dev folder terragrunt run-all plan. The Terragrunt will split the modules into groups. In each group, it will run plan, but first, it will run it in network modules because they are dependencies for other modules. Then, it will run plan in parallel for other modules in the other group.

Conclusion

Adding Terragrunt to the project is relatively easy. It requires some changes to the project structure and the code. But it is worth it. It makes the project more manageable: code is DRY, and common code is generated in one place. I can run Terraform commands from the root folder and do not need to visit each module, etc…

References