Skip to content

Instantly share code, notes, and snippets.

@zidenis

zidenis/terraform_module_development_environment_setup.md

Last active March 21, 2026 13:21
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save zidenis/93f8f93429e443c72243889b18bccad1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save zidenis/93f8f93429e443c72243889b18bccad1 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Denis revised this gist Dec 20, 2024. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion terraform_module_development_environment_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -8,7 +8,7 @@ Building robust and secure infrastructure with Terraform requires more than just

    ## Ingredients:

    * Linux utililities (curl, jq, tar, unzip, git)
    * Linux utilities (curl, jq, tar, unzip, git)
    * tenv: [version manager for Terraform](https://github.com/tofuutils/tenv)
    * terraform CLI: [HashiCorp's Terraform CLI](https://developer.hashicorp.com/terraform/cli)
    * terraform-docs: [tool for generating module documentation](https://github.com/terraform-docs/terraform-docs)
  2. Denis revised this gist Dec 20, 2024. 1 changed file with 18 additions and 16 deletions.
    34 changes: 18 additions & 16 deletions terraform_module_development_environment_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,14 +1,14 @@
    TLDR: This tutorial provides a step-by-step guide to setting up a complete Terraform development environment, including version management (tenv), documentation generation (terraform-docs), linting (TFLint), security scanning (Trivy, Checkov), pre-commit hooks, and others. We use a cookbook-style approach for easy understanding, but if you just want the script of commands to set up the environment, you can get it [here](https://github.com/zidenis/terraform-module-template-aws/blob/main/environment_setup.sh).
    TLDR: This tutorial provides a step-by-step guide to setting up a complete Terraform development environment, including version management (tenv), documentation generation (terraform-docs), linting (TFLint), security scanning (Trivy, Checkov), pre-commit hooks, and others. We use a cookbook-style approach for easy understanding, but if you just want the command script to set up the environment, you can get it [here](https://github.com/zidenis/terraform-module-template-aws/blob/main/environment_setup.sh).

    # Terraform Cookbook : Development Environment Recipe
    # Terraform Cookbook: Development Environment Recipe

    > A quick and easy recipe to set up a killer development environment for writing Terraform modules. Tested on Linux Ubuntu 24.04.1 LTS for x86_64 architecture (should work on other debian-based distros or others Linuxes with some tweaks).
    > A quick and easy recipe to set up a killer development environment for writing Terraform modules. Tested on Linux Ubuntu 24.04.1 LTS for x86_64 architecture (should work on other debian-based systems. For other Linux distributions, some tweaks are required).
    Building robust and secure infrastructure with Terraform requires more than just the terraform CLI. A well-equipped development environment is crucial for maximizing productivity, ensuring code quality, and preventing costly mistakes. This tutorial serves as your complete recipe for setting up a top-notch Terraform development environment. We'll guide you through installing and configuring essential tools. By the end, you'll have a streamlined workflow, allowing you to focus on crafting amazing infrastructure, not wrestling with configuration. We use a cookbook-style approach for easy understanding.
    Building robust and secure infrastructure with Terraform requires more than just the terraform CLI. A well-equipped development environment is crucial for maximizing productivity, ensuring code quality, and preventing costly mistakes. This tutorial serves as your complete recipe for setting up a top-notch Terraform development environment. I'll guide you through installing and configuring essential tools. By the end of this tutorial, you will have a streamlined workflow, enabling you to focus on building infrastructure rather than managing complex configurations. We use a cookbook-style approach for easy understanding.

    ## Ingredients:

    * Linux utils (curl, jq, tar, unzip, git)
    * Linux utililities (curl, jq, tar, unzip, git)
    * tenv: [version manager for Terraform](https://github.com/tofuutils/tenv)
    * terraform CLI: [HashiCorp's Terraform CLI](https://developer.hashicorp.com/terraform/cli)
    * terraform-docs: [tool for generating module documentation](https://github.com/terraform-docs/terraform-docs)
    @@ -24,10 +24,10 @@ Building robust and secure infrastructure with Terraform requires more than just

    ### 1. Know Your Kitchen, VSCode.

    First things first, every chef needs a well-equipped kitchen. In our case, Visual Studio Code is our kitchen, the place where we'll craft all our infrastructure recipes. But a bare kitchen isn't enough. We need the right tools and appliances to prepare delicious dishes. That's where the [Terraform extension by Hashicorp](https://marketplace.visualstudio.com/items?itemName=HashiCorp.terraform) comes in. The extension provides syntax highlighting, code completion, and more.
    First things first, every chef needs a well-equipped kitchen. In our case, Visual Studio Code is our kitchen, the place where we'll craft all our infrastructure recipes. But a bare kitchen isn't enough. We need the right tools and appliances to prepare delicious dishes. That's where the [Terraform extension by Hashicorp](https://marketplace.visualstudio.com/items?itemName=HashiCorp.terraform) comes in. The extension provides syntax highlighting, code completion, among other features.

    ```bash
    # Follow the [VSCode instalation guide](https://code.visualstudio.com/docs/setup/linux) for other instalation methods.
    # Follow the [VSCode installation guide](https://code.visualstudio.com/docs/setup/linux) for other instalation methods.
    sudo apt-get install curl

    curl -L -o vscode.deb https://go.microsoft.com/fwlink/?LinkID=760868
    @@ -42,7 +42,7 @@ code
    ### 2. Equip Your Kitchen with `tenv`.
    > Manage multiple Terraform versions like a pro. Simplifies the management of multiple Terraform versions, ensuring compatibility with different projects.
    Grab your Terraform version manager, `tenv`. Every good chef needs a well-organized kitchen. When it comes to managing different versions of Terraform, `tenv` will keep all your Terraform versions neatly organized and easily accessible. It also supports OpenTofu, Terragrunt, and Atmos. Let's get it installed:
    Grab your Terraform version manager, `tenv`. Every good chef needs a well-organized kitchen. When it comes to managing different versions of Terraform, `tenv` will keep your environment neatly organized and easily accessible. It also supports OpenTofu, Terragrunt, and Atmos. Let's get it installed:

    ```bash
    TENV_LATEST=$(curl -s https://api.github.com/repos/tofuutils/tenv/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux_x86_64.tar.gz")) | .browser_download_url')
    @@ -60,7 +60,7 @@ tenv completion bash > ~/.tenv/tenv_completion.bash
    echo -e '\n# tenv and terraform tools\nexport PATH="$HOME/.tenv:$PATH"\nsource $HOME/.tenv/tenv_completion.bash' >> ~/.bashrc
    ```

    The lines added to .bashrc ensure that `tenv` is always there when we start cooking terraform modules.
    The lines added to .bashrc ensure that `tenv` is always available when we start cooking terraform modules.

    ### 3. Grab Your Trusty `terraform` CLI Using `tenv`.

    @@ -77,7 +77,7 @@ This is like reaching into our neatly organized tools rack and selecting the per
    ### 4. Compile Your Infrastructure Recipe Notebook With `terraform-docs`.
    > Automatically generate documentation for your terraform modules (inputs, outputs, etc.).
    A good chef keeps a detailed recipe notebook, documenting each dish's ingredients, instructions, and variations. In the world of infrastructure as code, terraform-docs is our tool to mantain recipe documents. It automatically generates documentation for our Terraform modules, neatly organizing inputs, outputs, and other important details. This ensures that anyone (including our future selves) can easily understand and use our infrastructure recipes. Let's add it to our kitchen:
    A good chef keeps a detailed recipe notebook, documenting each dish's ingredients, instructions, and variations. In the world of infrastructure as code, terraform-docs is our tool to maintain recipe documents. It automatically generates documentation for our Terraform modules, neatly organizing inputs, outputs, and other important details. This ensures that anyone (including our future selves) can easily understand and use our infrastructure recipes. Let's add it to our kitchen:

    ```bash
    TDOC_LATEST=$(curl -s https://api.github.com/repos/terraform-docs/terraform-docs/releases/latest | jq -r '.assets[] | select(.name | endswith("linux-amd64.tar.gz")) | .browser_download_url')
    @@ -104,7 +104,7 @@ curl -L -O $TFLINT_LATEST
    unzip -o $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+") -d ~/.tenv
    ```

    The [`.tflint.hcl`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.tflint.hcl) file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to fit your project requirements. The config that I use nowadays for terraform AWS resources is like the bellow example.
    The [`.tflint.hcl`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.tflint.hcl) file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to align with your project requirements. The configuration I currently use for terraform AWS resources is like the bellow example.

    #### Sample [`.tflint.hcl`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.tflint.hcl) Configuration
    ```hcl
    @@ -147,7 +147,7 @@ curl -L -O $TRIVY_LATEST

    tar xzf $(echo $TRIVY_LATEST | grep -o -E "trivy_.+") -C ~/.tenv trivy

    # We are going to install checkov using python Pip and Venv
    # We are going to install checkov using python pip and venv
    sudo apt install python3-pip python3.12-venv

    python3 -m venv ~/.venv
    @@ -256,7 +256,7 @@ cd my-terraform-module
    git remote remove origin # We remove the original remote to start our own module
    # Do not run this script if your intention is to developt the module locally.
    # Do not run this script if your intention is to develop the module locally.
    chmod +x backend_bootstrap.sh; ./backend_bootstrap.sh # optionally, use this bash script to set up the remote backend.
    source ~/.venv/bin/activate
    @@ -297,11 +297,13 @@ git commit -m "Cooking a new module"

    ## Conclusion

    By following this recipe, you've equipped your development environment with essential tools to write, validate, and secure Terraform modules effectively. Remember to regularly update your tools and configurations for optimal performance and compliance.
    By following this recipe, you've equipped your development environment with essential tools to write, validate, and secure Terraform modules effectively. This tutorial was designed to install the latest versions of the tools, but remember to regularly update your tools and configurations for optimal performance and compliance.

    Are there any additional tools you'd like to explore to complement the setup? are you using Generative AI to develop your terraform modules? please, tell me how. Let us know your thoughts and suggestions to improve this guide further. Happy coding!
    Are there any additional tools or configurations you'd like to explore to complement the setup? are you using Generative AI to develop your terraform modules? please, tell me how. While Generative AI has the potential to streamline Terraform development by automating certain tasks like code generation or identifying potential issues. However, at this stage, I think it's likely not a core component for most users setting up a development environment. This is something that I plan to investigate.

    ### Tools versions at the moment of the write up
    Let me know your thoughts and suggestions to improve this guide further. Happy coding!

    ### Tools versions at the time of writing

    ```bash
    tenv version
  3. zidenis revised this gist Dec 20, 2024. No changes.
  4. Denis revised this gist Dec 20, 2024. 1 changed file with 208 additions and 134 deletions.
    342 changes: 208 additions & 134 deletions terraform_module_development_environment_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,25 +1,48 @@
    # Terraform cookbook : development environment setup
    TLDR: This tutorial provides a step-by-step guide to setting up a complete Terraform development environment, including version management (tenv), documentation generation (terraform-docs), linting (TFLint), security scanning (Trivy, Checkov), pre-commit hooks, and others. We use a cookbook-style approach for easy understanding, but if you just want the script of commands to set up the environment, you can get it [here](https://github.com/zidenis/terraform-module-template-aws/blob/main/environment_setup.sh).

    > A recipe to set up a good development environment to write terraform modules. Tested with Linux Ubuntu 24.04.1 LTS on WSL2 (should work with other Linux distros with some minor adaptations).
    # Terraform Cookbook : Development Environment Recipe

    > A quick and easy recipe to set up a killer development environment for writing Terraform modules. Tested on Linux Ubuntu 24.04.1 LTS for x86_64 architecture (should work on other debian-based distros or others Linuxes with some tweaks).
    Building robust and secure infrastructure with Terraform requires more than just the terraform CLI. A well-equipped development environment is crucial for maximizing productivity, ensuring code quality, and preventing costly mistakes. This tutorial serves as your complete recipe for setting up a top-notch Terraform development environment. We'll guide you through installing and configuring essential tools. By the end, you'll have a streamlined workflow, allowing you to focus on crafting amazing infrastructure, not wrestling with configuration. We use a cookbook-style approach for easy understanding.

    ## Ingredients:

    * Linux utils (wget, curl, jq, tar, unzip, git)
    * tenv : https://github.com/tofuutils/tenv
    * terraform CLI : https://developer.hashicorp.com/terraform/cli
    * terraform-docs : https://github.com/terraform-docs/terraform-docs
    * TFLint : https://github.com/terraform-linters/tflint
    * Trivy : https://github.com/aquasecurity/trivy
    * venv for Python3: https://docs.python.org/3/library/venv.html
    * Pre-commit : https://pre-commit.com/
    * Pre-commit-terraform : https://github.com/antonbabenko/pre-commit-terraform
    * Checkov : https://github.com/bridgecrewio/checkov
    * VSCode with terraform extensions : https://marketplace.visualstudio.com/items?itemName=4ops.terraform
    * Linux utils (curl, jq, tar, unzip, git)
    * tenv: [version manager for Terraform](https://github.com/tofuutils/tenv)
    * terraform CLI: [HashiCorp's Terraform CLI](https://developer.hashicorp.com/terraform/cli)
    * terraform-docs: [tool for generating module documentation](https://github.com/terraform-docs/terraform-docs)
    * TFLint: [terraform linter](https://github.com/terraform-linters/tflint)
    * Trivy: [security scanner for IaC and dependencies](https://github.com/aquasecurity/trivy)
    * venv: [virtual environments for Python](https://docs.python.org/3/library/venv.html)
    * Pre-commit: [git hook manager](https://pre-commit.com)
    * Pre-commit-terraform: [terraform git hooks](https://github.com/antonbabenko/pre-commit-terraform)
    * Checkov: [static analysis for IaC](https://github.com/bridgecrewio/checkov)
    * VSCode: [code editor with Terraform extension](https://marketplace.visualstudio.com/items?itemName=4ops.terraform)

    ## Preparation Steps:

    ### 1. Install `tenv`: a version manager for Terraform (also for OpenTofu, Terragrunt and Atmos).
    > Simplifies the management of multiple Terraform versions, ensuring compatibility with different projects.
    ### 1. Know Your Kitchen, VSCode.

    First things first, every chef needs a well-equipped kitchen. In our case, Visual Studio Code is our kitchen, the place where we'll craft all our infrastructure recipes. But a bare kitchen isn't enough. We need the right tools and appliances to prepare delicious dishes. That's where the [Terraform extension by Hashicorp](https://marketplace.visualstudio.com/items?itemName=HashiCorp.terraform) comes in. The extension provides syntax highlighting, code completion, and more.

    ```bash
    # Follow the [VSCode instalation guide](https://code.visualstudio.com/docs/setup/linux) for other instalation methods.
    sudo apt-get install curl

    curl -L -o vscode.deb https://go.microsoft.com/fwlink/?LinkID=760868

    sudo apt install ./vscode.deb

    code
    # After VSCode launch, use Quick Open (Ctrl+P), paste the command "ext install HashiCorp.terraform" and press enter.
    # Or install the extension @id:HashiCorp.terraform using the VSCode GUI.
    ```

    ### 2. Equip Your Kitchen with `tenv`.
    > Manage multiple Terraform versions like a pro. Simplifies the management of multiple Terraform versions, ensuring compatibility with different projects.
    Grab your Terraform version manager, `tenv`. Every good chef needs a well-organized kitchen. When it comes to managing different versions of Terraform, `tenv` will keep all your Terraform versions neatly organized and easily accessible. It also supports OpenTofu, Terragrunt, and Atmos. Let's get it installed:

    ```bash
    TENV_LATEST=$(curl -s https://api.github.com/repos/tofuutils/tenv/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux_x86_64.tar.gz")) | .browser_download_url')
    @@ -37,16 +60,24 @@ tenv completion bash > ~/.tenv/tenv_completion.bash
    echo -e '\n# tenv and terraform tools\nexport PATH="$HOME/.tenv:$PATH"\nsource $HOME/.tenv/tenv_completion.bash' >> ~/.bashrc
    ```

    ### 2. Install `terraform CLI` using `tenv`.
    The lines added to .bashrc ensure that `tenv` is always there when we start cooking terraform modules.

    ### 3. Grab Your Trusty `terraform` CLI Using `tenv`.

    It's time to grab our most essential infrastructure cooking utensil, our pan: the `terraform` CLI. This is the workhorse of our infrastructure kitchen – the pan we'll use to cook up all our infrastructure recipes. We'll use `tenv` to fetch and install the perfect pan for the job. If you wish, use [Opentofu](https://opentofu.org/), which is an "more open" source altenative.

    ```bash
    tenv terraform install

    terraform -install-autocomplete
    ```

    ### 3. Install `terraform-docs`: a tool to generate documentation from Terraform modules.
    > Automates the creation of files documenting inputs, outputs, resources, and dependencies in a Terraform module.
    This is like reaching into our neatly organized tools rack and selecting the perfect terraform CLI pan.

    ### 4. Compile Your Infrastructure Recipe Notebook With `terraform-docs`.
    > Automatically generate documentation for your terraform modules (inputs, outputs, etc.).
    A good chef keeps a detailed recipe notebook, documenting each dish's ingredients, instructions, and variations. In the world of infrastructure as code, terraform-docs is our tool to mantain recipe documents. It automatically generates documentation for our Terraform modules, neatly organizing inputs, outputs, and other important details. This ensures that anyone (including our future selves) can easily understand and use our infrastructure recipes. Let's add it to our kitchen:

    ```bash
    TDOC_LATEST=$(curl -s https://api.github.com/repos/terraform-docs/terraform-docs/releases/latest | jq -r '.assets[] | select(.name | endswith("linux-amd64.tar.gz")) | .browser_download_url')
    @@ -60,8 +91,10 @@ terraform-docs completion bash > ~/.tenv/terraform-docs_completion.bash
    echo -e '\nsource $HOME/.tenv/terraform-docs_completion.bash' >> ~/.bashrc
    ```

    ### 4. Install `TFLint`: a linter for Terraform code.
    > Ensures best practices, identifies syntax errors, and enforces standards for Terraform configurations.
    ### 5. Employ TFLint Quality Control.
    > A linter for Terraform code. Catch syntax errors, enforce standards, and keep your Terraform configs clean.
    Maintaining quality control is crucial. In the Terraform kitchen, `TFLint` scans our Terraform code for syntax errors and potential issues, ensuring our code is sound. Let's add this to our kitchen:

    ```bash
    TFLINT_LATEST=$(curl -s https://api.github.com/repos/terraform-linters/tflint/releases/latest | jq -r '.assets[] | select(.name | endswith("tflint_linux_amd64.zip")) | .browser_download_url')
    @@ -71,67 +104,9 @@ curl -L -O $TFLINT_LATEST
    unzip -o $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+") -d ~/.tenv
    ```

    ### 5. Install `Trivy`: a comprehensive and versatile security scanner.
    > Identifies security risks in container images, IaC files (including Terraform), and dependencies.
    ```bash
    TRIVY_LATEST=$(curl -s https://api.github.com/repos/aquasecurity/trivy/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux-64bit.tar.gz")) | .browser_download_url')

    curl -L -O $TRIVY_LATEST

    tar xzf $(echo $TRIVY_LATEST | grep -o -E "trivy_.+") -C ~/.tenv trivy
    ```


    ### 6. Install `pre-commit`: A framework for managing and maintaining Git hooks.
    > Ensures consistent code quality by running checks (e.g., linting, formatting, security scans) before committing code to a repository.
    ```sh
    sudo apt install python3.12-venv

    python3 -m venv ~/.venv

    source ~/.venv/bin/activate

    pip install pre-commit
    ```

    ### 7. Install `Checkov`: a static code analysis tool for IaC.
    > Detects misconfigurations, security risks, and compliance violations in Terraform and other IaC templates.
    ```bash
    pip install checkov
    ```

    ### 8. Setup VSCode with a terraform extension.

    To develop terraform modules I use VSCode with [Terraform extension by Anton Kulikov](https://marketplace.visualstudio.com/items?itemName=4ops.terraform). It adds syntax support for the Terraform and Terragrunt configuration language.

    ### 9. Setup the terraform root module directory.

    Now that you installed all the devlopment supporting tools, its time to configure the terraform root module directory structure. A Standard Module Structure is presented in https://developer.hashicorp.com/terraform/language/modules/develop/structure . Look at the example bellow.

    ```
    root-module/
    ├── README.md
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── providers.tf
    ├── modules/
    │ ├── child-module-A/
    │ │ ├── README.md
    │ │ ├── main.tf
    │ │ ├── variables.tf
    │ │ ├── outputs.tf
    │ ├── child-module-B/
    │ ├── .../
    ├── .pre-commit-config.yaml
    └── .tflint.hcl
    ```

    The `.tflint.hcl` file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to fit your project requirements. The config that I use for terraform AWS resources is:
    The [`.tflint.hcl`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.tflint.hcl) file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to fit your project requirements. The config that I use nowadays for terraform AWS resources is like the bellow example.

    #### Sample [`.tflint.hcl`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.tflint.hcl) Configuration
    ```hcl
    tflint {
    required_version = "~> 0.54.0"
    @@ -157,9 +132,52 @@ rule "aws_resource_missing_tags" {
    }
    ```

    The `.pre-commit-config.yaml` file is the configuration file for the Pre-commit framework. Its purpose is to define and manage pre-commit hooks, which are scripts that run automatically before a commit is made in a Git repository. These hooks ensure that code quality, consistency, and compliance checks are enforced.
    ### 6. Double Strain Your Infrastructure Code With `Trivy` And `Checkov`.

    > Trivy scan your IaC files, container images, and dependencies for vulnerabilities.
    > Checkov detects misconfigurations, security risks, and compliance violations in Terraform and other IaC templates.
    Ensuring the safety of your code is paramount. You wouldn't want any unwanted bits sneaking into your dishes. Just as a chef uses strainers to remove impurities, we use `Trivy` and `Checkov` to filter out potential vulnerabilities and misconfigurations from our infrastructure code. Think of them as two different types of strainers, each catching different kinds of unwanted elements. Let's add them:

    ```bash
    TRIVY_LATEST=$(curl -s https://api.github.com/repos/aquasecurity/trivy/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux-64bit.tar.gz")) | .browser_download_url')

    curl -L -O $TRIVY_LATEST

    tar xzf $(echo $TRIVY_LATEST | grep -o -E "trivy_.+") -C ~/.tenv trivy

    # We are going to install checkov using python Pip and Venv
    sudo apt install python3-pip python3.12-venv

    python3 -m venv ~/.venv

    source ~/.venv/bin/activate

    pip install checkov
    ```

    Now, with both `Trivy` and `Checkov` in our kitchen, we can confidently cook a safe and secure code. This double-straining approach provides a more comprehensive security check, catching both specific vulnerabilities and broader misconfigurations.

    ### 7. Set Up Your `pre-commit` Kitchen Workflow Checklist

    > Ensures consistent code quality by running checks (e.g., linting, formatting, security scans) before committing code.
    In a busy kitchen, chefs rely on checklists and workflows to ensure every step is followed and no crucial task is missed. `pre-commit` acts as our kitchen workflow checklist, ensuring we systematically employ all our quality control and tools (like `TFLint`, `Trivy`, and `Checkov`) before we "serve" (commit) our infrastructure code.

    `pre-commit` uses git hooks, which are scripts that run automatically at specific points in the Git workflow. In this case, we're going to use `pre-commit` hooks, which run before each git commit. This ensures that our code is always checked before it's added to the project's history. Let's set up our kitchen workflow checklist:

    ```sh
    source ~/.venv/bin/activate

    pip install pre-commit
    ```

    Now that we have pre-commit installed, we'll configure it to use our quality control tools. The [`.pre-commit-config.yaml`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.pre-commit-config.yaml) file is the configuration file for the Pre-commit framework that define the pre-commit hook.

    #### Sample [`.pre-commit-config.yaml`](https://github.com/zidenis/terraform-module-template-aws/blob/main/.pre-commit-config.yaml) Configuration.

    ```yaml
    repos:
    - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.96.2
    @@ -189,83 +207,139 @@ repos:
    name: "Pre-commit remove trailing whitespaces"
    ```
    ### Bonus: A Terraform Module Template for AWS Resources
    Beyond quality and safety, presentation matters too! Just as chefs ensure consistent dish presentation, we also use `terraform fmt` and others `pre-commit` hooks to format our code in a consistent way. This makes our code easier to read, understand, and maintain, both for ourselves and our colleagues.

    Now that our kitchen is equipped, it's time to gather the ingredients for our infrastructure main dish: the Terraform root module.

    ### 8. Our Main Infrastructure Dish: The Terraform Root Module.

    Just like any good recipe, it requires the right ingredients, organized in the right way. HashiCorp suggests a [standard module structure](https://developer.hashicorp.com/terraform/language/modules/develop/structure), which we'll follow to ensure a well-structured and maintainable code. Here is an example:

    ```tree
    root-module/
    ├── README.md
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── providers.tf
    ├── modules/
    │ ├── child-module-A/
    │ │ ├── README.md
    │ │ ├── main.tf
    │ │ ├── variables.tf
    │ │ ├── outputs.tf
    │ ├── child-module-B/
    │ ├── .../
    ├── .pre-commit-config.yaml
    └── .tflint.hcl
    ```

    With our main dish structure defined, let's dive into a sample recipe specifically for AWS dishes.

    #### A Sample Recipe for AWS
    > A template that serves as a foundation for creating Terraform modules.

    > This template serves as a foundation for creating Terraform modules.
    [Here](https://github.com/zidenis/terraform-module-template-aws) is a handy module template to bootstrap your AWS resources. It serves as a foundation starting point for various AWS infrastructure creations, and includes essential ingredients for a robust and secure AWS setup, including:

    I developed a template to bootstrap the development of terraform modules to provision AWS resources. It help me to create an S3 bucket to store terraform state (with server-side encryption, versioning, lifecycle rules for noncurrent object transition and expiration, blocked of public access for enhanced security), and also creates a DynamoDB table for state locking.
    * An S3 Bucket for State Storage. It includes server-side encryption, versioning, and lifecycle rules for noncurrent object transition and expiration, ensuring the safety and management of our stored data. It also blocks public access, preventing unwanted access.

    #### Usage
    * A DynamoDB Table for State Locking. This acts as a coordination system in our kitchen, preventing conflicts when we have multiple chefs.

    #### Template usage example

    ```bash
    # Use AWS CLI to configure the AWS Access Key ID and the AWS Secret Access Key for the user with the propper permission to provision the AWS resources.
    aws configure
    aws configure # You will need the AWS CLI installed. Set up AWS credentials with privileges to provision resources. Remember to use temporary credentials or store permanent credentials outside the git repository.
    git clone https://github.com/zidenis/terraform-module-template-aws.git my-terraform-module
    cd my-terraform-module
    git remote remove origin
    git remote remove origin # We remove the original remote to start our own module
    # Use this bash script to bootstrap the S3 bucket to hold the terraform state remotely
    ./backend_bootstrap.sh
    ```
    # Do not run this script if your intention is to developt the module locally.
    chmod +x backend_bootstrap.sh; ./backend_bootstrap.sh # optionally, use this bash script to set up the remote backend.
    Now the infrastructure is ready to start the development of your module. Use `pre-commit` to run the quality validations.
    source ~/.venv/bin/activate
    ```bash
    $ source ~/.venv/bin/activate

    $ pre-commit run -a
    [INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
    [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
    [INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
    [INFO] Once installed this environment will be reused.
    [INFO] This may take a few minutes...
    Terraform validate.......................................................Passed
    Terraform fmt............................................................Passed
    Terraform validate with tflint...........................................Passed
    Terraform validate with trivy............................................Passed
    Terraform validate with Checkov..........................................Passed
    Terraform docs...........................................................Passed
    Pre-commit detect AWS credentials........................................Passed
    Pre-commit detect private keys...........................................Passed
    Pre-commit fix end of files..............................................Passed
    Pre-commit remove trailing whitespaces...................................Passed
    pre-commit install --install-hooks # set up your git repository to run pre-commit hooks automatically
    pre-commit run -a # optionally, run pre-commit hooks against all the files.
    ```

    ## Cleanup
    You can now start developing your specific module, using the provided structure and backend setup as a solid foundation. Using `pre-commit`, you can run all the quality and security checks we set up earlier, ensuring a perfect and secure final dish.

    #### pre-commit usage example

    ```bash
    rm $(echo $TENV_LATEST | grep -o -E "terraform-docs-.+")
    rm $(echo $TENV_LATEST | grep -o -E "tenv_v.*")
    rm $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+")
    rm $(echo $TRIVY_LATEST | grep -o -E "trivy_.+")
    source ~/.venv/bin/activate
    echo "# cooking a new module" >> main.tf
    git add main.tf
    git commit -m "Cooking a new module"
    # [INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
    # [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
    # [INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
    # [INFO] Once installed this environment will be reused.
    # [INFO] This may take a few minutes...
    # Terraform validate.......................................................Passed
    # Terraform fmt............................................................Passed
    # Terraform validate with tflint...........................................Passed
    # Terraform validate with trivy............................................Passed
    # Terraform validate with Checkov..........................................Passed
    # Terraform docs...........................................................Passed
    # Pre-commit detect AWS credentials........................................Passed
    # Pre-commit detect private keys...........................................Passed
    # Pre-commit fix end of files..............................................Passed
    # Pre-commit remove trailing whitespaces...................................Passed
    ```

    ## Tools versions at the moment of the write up
    ## Conclusion

    By following this recipe, you've equipped your development environment with essential tools to write, validate, and secure Terraform modules effectively. Remember to regularly update your tools and configurations for optimal performance and compliance.

    Are there any additional tools you'd like to explore to complement the setup? are you using Generative AI to develop your terraform modules? please, tell me how. Let us know your thoughts and suggestions to improve this guide further. Happy coding!

    ### Tools versions at the moment of the write up

    ```bash
    $ tenv version
    tenv version v3.2.11
    tenv version
    # tenv version v3.2.11
    terraform version
    # Terraform v1.10.3
    # on linux_amd64
    terraform-docs version
    # terraform-docs version v0.19.0 af31cc6 linux/amd64
    tflint --version
    # TFLint version 0.54.0
    # + ruleset.terraform (0.10.0-bundled)
    trivy --version
    # Version: 0.58.0
    checkov --version
    # 3.2.343
    $ terraform version
    Terraform v1.10.2
    on linux_amd64
    pre-commit --version
    #pre-commit 4.0.1
    ```

    $ terraform-docs version
    terraform-docs version v0.19.0 af31cc6 linux/amd64
    ## Cleanup

    $ tflint --version
    TFLint version 0.54.0
    + ruleset.terraform (0.10.0-bundled)
    Clean up downloaded archives.

    $ trivy --version
    Version: 0.58.0
    ```bash
    rm $(echo $TENV_LATEST | grep -o -E "tenv_v.*")
    $ pre-commit --version
    pre-commit 4.0.1
    rm $(echo $TDOC_LATEST | grep -o -E "terraform-docs-.+")
    $ checkov --version
    3.2.334
    ```
    rm $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+")
    rm $(echo $TRIVY_LATEST | grep -o -E "trivy_.+")
    rm vscode.deb
    ```
  5. zidenis created this gist Dec 13, 2024.
    271 changes: 271 additions & 0 deletions terraform_module_development_environment_setup.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,271 @@
    # Terraform cookbook : development environment setup

    > A recipe to set up a good development environment to write terraform modules. Tested with Linux Ubuntu 24.04.1 LTS on WSL2 (should work with other Linux distros with some minor adaptations).
    ## Ingredients:

    * Linux utils (wget, curl, jq, tar, unzip, git)
    * tenv : https://github.com/tofuutils/tenv
    * terraform CLI : https://developer.hashicorp.com/terraform/cli
    * terraform-docs : https://github.com/terraform-docs/terraform-docs
    * TFLint : https://github.com/terraform-linters/tflint
    * Trivy : https://github.com/aquasecurity/trivy
    * venv for Python3: https://docs.python.org/3/library/venv.html
    * Pre-commit : https://pre-commit.com/
    * Pre-commit-terraform : https://github.com/antonbabenko/pre-commit-terraform
    * Checkov : https://github.com/bridgecrewio/checkov
    * VSCode with terraform extensions : https://marketplace.visualstudio.com/items?itemName=4ops.terraform

    ## Preparation Steps:

    ### 1. Install `tenv`: a version manager for Terraform (also for OpenTofu, Terragrunt and Atmos).
    > Simplifies the management of multiple Terraform versions, ensuring compatibility with different projects.
    ```bash
    TENV_LATEST=$(curl -s https://api.github.com/repos/tofuutils/tenv/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux_x86_64.tar.gz")) | .browser_download_url')

    curl -L -O $TENV_LATEST

    mkdir ~/.tenv

    tar xvzf $(echo $TENV_LATEST | grep -o -E "tenv_v.*") -C ~/.tenv

    export PATH="$HOME/.tenv:$PATH"

    tenv completion bash > ~/.tenv/tenv_completion.bash

    echo -e '\n# tenv and terraform tools\nexport PATH="$HOME/.tenv:$PATH"\nsource $HOME/.tenv/tenv_completion.bash' >> ~/.bashrc
    ```

    ### 2. Install `terraform CLI` using `tenv`.

    ```bash
    tenv terraform install

    terraform -install-autocomplete
    ```

    ### 3. Install `terraform-docs`: a tool to generate documentation from Terraform modules.
    > Automates the creation of files documenting inputs, outputs, resources, and dependencies in a Terraform module.
    ```bash
    TDOC_LATEST=$(curl -s https://api.github.com/repos/terraform-docs/terraform-docs/releases/latest | jq -r '.assets[] | select(.name | endswith("linux-amd64.tar.gz")) | .browser_download_url')

    curl -L -O $TDOC_LATEST

    tar xzf $(echo $TDOC_LATEST | grep -o -E "terraform-docs-.+") -C ~/.tenv terraform-docs

    terraform-docs completion bash > ~/.tenv/terraform-docs_completion.bash

    echo -e '\nsource $HOME/.tenv/terraform-docs_completion.bash' >> ~/.bashrc
    ```

    ### 4. Install `TFLint`: a linter for Terraform code.
    > Ensures best practices, identifies syntax errors, and enforces standards for Terraform configurations.
    ```bash
    TFLINT_LATEST=$(curl -s https://api.github.com/repos/terraform-linters/tflint/releases/latest | jq -r '.assets[] | select(.name | endswith("tflint_linux_amd64.zip")) | .browser_download_url')

    curl -L -O $TFLINT_LATEST

    unzip -o $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+") -d ~/.tenv
    ```

    ### 5. Install `Trivy`: a comprehensive and versatile security scanner.
    > Identifies security risks in container images, IaC files (including Terraform), and dependencies.
    ```bash
    TRIVY_LATEST=$(curl -s https://api.github.com/repos/aquasecurity/trivy/releases/latest | jq -r '.assets[] | select(.name | endswith("Linux-64bit.tar.gz")) | .browser_download_url')

    curl -L -O $TRIVY_LATEST

    tar xzf $(echo $TRIVY_LATEST | grep -o -E "trivy_.+") -C ~/.tenv trivy
    ```


    ### 6. Install `pre-commit`: A framework for managing and maintaining Git hooks.
    > Ensures consistent code quality by running checks (e.g., linting, formatting, security scans) before committing code to a repository.
    ```sh
    sudo apt install python3.12-venv

    python3 -m venv ~/.venv

    source ~/.venv/bin/activate

    pip install pre-commit
    ```

    ### 7. Install `Checkov`: a static code analysis tool for IaC.
    > Detects misconfigurations, security risks, and compliance violations in Terraform and other IaC templates.
    ```bash
    pip install checkov
    ```

    ### 8. Setup VSCode with a terraform extension.

    To develop terraform modules I use VSCode with [Terraform extension by Anton Kulikov](https://marketplace.visualstudio.com/items?itemName=4ops.terraform). It adds syntax support for the Terraform and Terragrunt configuration language.

    ### 9. Setup the terraform root module directory.

    Now that you installed all the devlopment supporting tools, its time to configure the terraform root module directory structure. A Standard Module Structure is presented in https://developer.hashicorp.com/terraform/language/modules/develop/structure . Look at the example bellow.

    ```
    root-module/
    ├── README.md
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── providers.tf
    ├── modules/
    │ ├── child-module-A/
    │ │ ├── README.md
    │ │ ├── main.tf
    │ │ ├── variables.tf
    │ │ ├── outputs.tf
    │ ├── child-module-B/
    │ ├── .../
    ├── .pre-commit-config.yaml
    └── .tflint.hcl
    ```

    The `.tflint.hcl` file is the configuration file for TFLint. Its purpose is to define the rules, plugins, and configurations that TFLint will use when analyzing your Terraform code. This allows you to enforce best practices, identify issues, and customize the linter's behavior to fit your project requirements. The config that I use for terraform AWS resources is:

    ```hcl
    tflint {
    required_version = "~> 0.54.0"
    }
    plugin "terraform" {
    enabled = true
    preset = "all"
    }
    plugin "aws" {
    enabled = true
    version = "0.36.0"
    source = "github.com/terraform-linters/tflint-ruleset-aws"
    deep_check = true
    }
    rule "aws_resource_missing_tags" {
    enabled = true
    tags = [
    "env"
    ]
    }
    ```

    The `.pre-commit-config.yaml` file is the configuration file for the Pre-commit framework. Its purpose is to define and manage pre-commit hooks, which are scripts that run automatically before a commit is made in a Git repository. These hooks ensure that code quality, consistency, and compliance checks are enforced.

    ```
    repos:
    - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.96.2
    hooks:
    - id: terraform_validate
    - id: terraform_fmt
    - id: terraform_tflint
    args:
    - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl
    - id: terraform_trivy
    - id: terraform_checkov
    name: "Terraform validate with Checkov"
    args:
    - --args=--quiet
    - id: terraform_docs
    - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
    - id: detect-aws-credentials
    name: "Pre-commit detect AWS credentials"
    - id: detect-private-key
    name: "Pre-commit detect private keys"
    - id: end-of-file-fixer
    name: "Pre-commit fix end of files"
    - id: trailing-whitespace
    name: "Pre-commit remove trailing whitespaces"
    ```

    ### Bonus: A Terraform Module Template for AWS Resources

    > This template serves as a foundation for creating Terraform modules.
    I developed a template to bootstrap the development of terraform modules to provision AWS resources. It help me to create an S3 bucket to store terraform state (with server-side encryption, versioning, lifecycle rules for noncurrent object transition and expiration, blocked of public access for enhanced security), and also creates a DynamoDB table for state locking.

    #### Usage

    ```bash
    # Use AWS CLI to configure the AWS Access Key ID and the AWS Secret Access Key for the user with the propper permission to provision the AWS resources.
    aws configure

    git clone https://github.com/zidenis/terraform-module-template-aws.git my-terraform-module

    cd my-terraform-module

    git remote remove origin

    # Use this bash script to bootstrap the S3 bucket to hold the terraform state remotely
    ./backend_bootstrap.sh
    ```

    Now the infrastructure is ready to start the development of your module. Use `pre-commit` to run the quality validations.

    ```bash
    $ source ~/.venv/bin/activate

    $ pre-commit run -a
    [INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
    [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
    [INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
    [INFO] Once installed this environment will be reused.
    [INFO] This may take a few minutes...
    Terraform validate.......................................................Passed
    Terraform fmt............................................................Passed
    Terraform validate with tflint...........................................Passed
    Terraform validate with trivy............................................Passed
    Terraform validate with Checkov..........................................Passed
    Terraform docs...........................................................Passed
    Pre-commit detect AWS credentials........................................Passed
    Pre-commit detect private keys...........................................Passed
    Pre-commit fix end of files..............................................Passed
    Pre-commit remove trailing whitespaces...................................Passed
    ```

    ## Cleanup

    ```bash
    rm $(echo $TENV_LATEST | grep -o -E "terraform-docs-.+")
    rm $(echo $TENV_LATEST | grep -o -E "tenv_v.*")
    rm $(echo $TFLINT_LATEST | grep -o -E "tflint_linux.+")
    rm $(echo $TRIVY_LATEST | grep -o -E "trivy_.+")
    ```

    ## Tools versions at the moment of the write up

    ```bash
    $ tenv version
    tenv version v3.2.11

    $ terraform version
    Terraform v1.10.2
    on linux_amd64

    $ terraform-docs version
    terraform-docs version v0.19.0 af31cc6 linux/amd64

    $ tflint --version
    TFLint version 0.54.0
    + ruleset.terraform (0.10.0-bundled)

    $ trivy --version
    Version: 0.58.0

    $ pre-commit --version
    pre-commit 4.0.1

    $ checkov --version
    3.2.334
    ```