20 Terraform Best Practices to Create Clean and Reusable Code
Terraform is a powerful Infrastructure as Code (IaC) tool.
As one of the most popular tools in the IaC space, more and more developers are learning Terraform to deploy infrastructure throughout their organisation.
With this uptake in developers using Terraform (and many easy to avoid mistakes) we’ve put together a list of best practices that should help you use this tool effectively!
In this blog, we’ll help you create clean and reusable terraform code that you or your colleagues can use and improve upon later with relative ease.
The best practices are split into three categories:
But first…
What Is Terraform?
We won’t go into too much detail here, as this article is specifically aimed at people who already have a working knowledge of Terraform, but as a quick refresher:
HashiCorp Terraform is an IaC tool that lets developers define both cloud and on-prem resources that they can then version, reuse, and share.
If you need more of an introduction, check out these resources:
- Introduction to Terraform
- Terraform Basics Tutorials
- Terraform Made Easy by Jordan Taylor
- Contino Engineering blog on why Terraform Code Smells by Hibri Marzook
5 Best Practices for Terraform Deployments
I refer to deployments here as anything that’s not a module—any code where you do terraform plan and apply is a deployment.
Even though you could write all your Terraform code in a deployment and not do any modules at all, it would make your code harder to maintain and not very reusable; these practices aim to prevent that.
1. Deployments Should Use Inputs Over Code Changes
Deployment code should invoke the least amount of modules and resources to accomplish its goals, and most changes to the deployment should be done by either changing the inputs in a tfvar file or the version of the module.
This will help you create an abstraction layer in your deployments, and if you ever need to do multiple similar deployments in different environments; you can use your deployment code as a template and adjust the inputs and module versions accordingly.
But how do you handle multiple instances of a resource or module by only changing the input?
2. Iterate Over Maps and Objects
Rather than instantiating multiple modules or resources in your deployment manually, iterate over them using maps or objects as shown below.
This module will create as many instances of the org_policies_obect as you pass to it or none at all if the object is empty:
Here you define the variables required for the module:
And finally you pass the inputs to the module:
Alternatively your module could be designed to take objects and iterate through them.
Try not to be too repetitive with your objects if you can—if there is a global value; that should be set separately and not in the object itself. There is an experimental feature in Terraform for optional values in objects which will hopefully become part of the stable version soon.
3. Leverage Tfvars Files
For anything that’s not a secret, use tfvars files as much as possible for all your inputs and add them to your source control. That way you can keep track of values and revert to a previous commit if you make a mistake and you can see what’s deployed at a glance. This should also be where most of your deployment changes happen.
4. Consider Splitting Your Deployments Into Layers
If your deployment is large it can be cumbersome to make changes and if you make mistakes then the damage surface is wider. You should split your deployments into multiple layers if they get too large. This has the added benefit that permissions can be narrowed down to each layer and potentially outsourced to multiple teams. Modules can also be tailored for each layer and will be smaller and more manageable to work with. One example that makes use of this practice is the Terraform Example Foundation by Google.
5. Separate Variables and Inputs Based on Their Functionality
In your variables.tf file and terraform.tfvars use comments to separate them based on their function. This makes your code more readable and easy to change when working with it.
11 Best Practices for Terraform Modules
1. Use Opinionated Modules to Do Exactly What You Need
Unless you are creating open source modules or modules that are general purpose to be used by many teams, you should create modules that are opinionated for your particular use case.
These can make use of resources, open source modules or any you created yourself, but be very careful not to create too many module dependencies as you may find it tedious to update your code.
2. Leverage Official Open Source Modules
Consider using open source modules provided freely by Hashicorp and the platform you are using. These modules can be used as primitives by modules you create or they can be used in your deployments as they come if they achieve everything you need, you just need to ensure you call specific versions of them so your deployments are consistent.
I’ve seen many people who advocate forking open source modules and tweaking them. I’d be cautious when following that approach for three reasons:
- Forking an open source repository and changing it means you are now the maintainer of that module, adding more workload to yourself and other members of your team.
- Engineers are likely to be familiar with an open source module, but not your bespoke version of it, hence new staff enrolment would be quicker if you are using standard modules.
- Open source modules are generally too broad, your in-house modules should be opinionated for your use case to make them simpler to use and maintain.
Forking is sometimes mandatory, some companies like banks require you to fork modules and keep them in-house, but if you do that I would consider not changing them at all and just track the official version and update where possible.
That all being said, there may be situations where importing an official module and changing it may suit your needs, in that case I recommend you strip it bare of all the features you don’t need and simplify it as much as possible.
Also in that case consider using open source modules that are designed to be forked if you can find them. For example, Google Cloud has the cloud foundation fabric modules that are designed just for this purpose.
3. Make Extensive Use of Convention Over Configuration
Your modules will be opinionated to do what you need to do, hence you should default as many variables as possible and only require the bare minimum for setup. This will help keep your deployment code clean and easy to understand and change. Ideally you should only require five or six variables at most, default everything else if you can.
This does not mean you cannot…
4. Make Modules Flexible With Multiple Optional Inputs
You should be able to use your modules with minimal inputs, but that doesn’t mean that they shouldn’t be flexible for changes, this minimises the need to have to change code and it gives you options based on different situations. That being said, don’t get too bogged down trying to predict every scenario under the sun and start by making things as simple as possible, based upon your use case.
5. Refer to Modules by Version
Don’t avoid specifying the version of your modules as that could break your deployments whenever you make changes to them. Consider using Semantic Versioning to update your modules.
6. Consider Bundling Modules Together if They Serve a Common Purpose
At the start of a project you may consider keeping your modules and deployments in the same repository and refer to them by path. As your product matures, you may want to move these modules to a separate repository to be able to refer to them by version and maintain them separately.
Having one repository per module is useful if your modules need to be maintained by different teams or if they are common modules used by multiple projects. However, if you have a set of closely related modules consider keeping them all in one repository and use them as submodules.
You can still version them as a whole in this way and it is much easier to manage them. Remember that you can always separate the modules later in their own repositories, but always start with the simplest setup possible.
Here is an example of a repository that uses submodules by Google that you can use as reference:
7. Consider Using Variable and Naming Validation
Terraform has a relatively new feature where you can validate names with REGEX. This is very useful to avoid naming errors with your platform before you hit apply. You could also enforce the way your resources are named by concatenating inputs such as labels and prefixes while validating each of them, keeping your platform naming consistent.
8. Use Locals Correctly
I’ve seen locals in situations where variables would be better suited. I find locals to be very useful especially in the following situations:
- Use of functions on your outputs and/or inputs
- Concatenate variables to form names of resources
- Use conditional expressions
You can keep locals in their own file but I generally recommend keeping them in the same file and close to the code they are used for.
9. Keep the Code in Your Module Logically Separated
I generally advocate keeping the structure of the files standard to avoid confusion. However, if your module requires over 200 lines of code, not including variables, I would consider splitting the main.tf into multiple files according to what they do and keep all related resources and locals within that file. This makes it easier to modify and read than having to search through a long main.tf file—even if that file is separated with comment lines.
10. Separate Required and Optional Variables
To improve readability of your code, keep required Variables at the top and Optional variables at the bottom, separate both with a comment line in your variables.tf file.
11. Always Have an Example/s Folder Within Your Module Folder
The example folder has two advantages.
- It gives users an idea of how to use your module in a deployment
- You can use it to test your module code before creating a new version for it
4 Best Practices for Terraform CI/CD
1. Add Git Hooks
Use pre-commit hooks so you never forget to do things like adding automated documentation or formatting your code. These are good pre-commit hooks to get you started:
- Anton Babenko’s pre-commit hooks: use for Terraform docs, fmt, tflint, tfsec, checkov, etc.
- Pre-commit-hooks: many other hooks available like detect-private-key
There are hundreds of pre-commit hooks you can use but don’t get overwhelmed by all the choices, start with a few and add more as you need them.
2. Aim to do Trunk Based Development
Trunk based development is the most recommended git branching strategy and you can apply it to your terraform code as well. You can achieve this by separating your deployment code into multiple directories and having different triggers or pipelines for every folder. Your main branch and/or release tags can execute a Terraform apply command, whereas every other branch can be set to plan and test your code.
Always keep the main branch ready for deployment and clean of redundant artefacts and files. Also ensure there are READMEs and comments explaining your code. This is particularly important if you work in a team and your colleagues need to look at what you are doing.
Ensure you learn your git well before embarking on your trunk based development journey.
3. Test Your Module Separately Before Integrating in a Deployment
You should test your module separately before integrating in a deployment, this should ideally include a full deployment to a test environment before a new version is pushed.
4. Test as Much as You Can
Consider using tools like terrascan, terratest, tfsec or terraform validator by Google to test your code as much as possible before it gets deployed.
Summary
We’ve covered a lot of best practices in this blog but if you take away anything from it, it should be the following:
- Create Terraform code so that it can be used by people who are not familiar with it by only changing inputs or updating the version of the module.
- Do most, if not all, of your coding on opinionated modules so you only have to change inputs to apply infrastructure changes.
- Create modules that do convention over configuration and default most values.
- Treat modules as easy to understand applications.
- Do this even if only Terraform experts are using your modules and code.
- Don’t reinvent the wheel and favour using officially maintained modules if you can.
If you’d like any more support on using Terraform in your organisation, please reach out!
