Deploying infrastructure manually is an outdated practice. Using Terraform to automate manual deployment is the new normal. In this blog, we have explained in detail, how to create your first module using Terraform.
Before the advent of cloud and DevOps, most companies managed and deployed their infrastructure manually. This used to be risky because not only was it error-prone, it also slowed down the entire infrastructure cycle. The good news is that now, most companies are not deploying infrastructure manually but instead using tools like Terraform. In this blog, we are going to cover Terraform and the use of terraform modules.
The key idea behind Infrastructure as Code(IaC) is to manage almost ‘everything’ as code, where everything involves your servers, network devices, databases, application configuration, automated tests, deployment process, etc. This consists of every stage of your infrastructure lifecycle, starting from defining, deploying, updating, and destroying. The advantage of defining every resource as IaC is you can now version control it, reuse it, validate it and build a self-service model in your organization.
Terraform is an open-source tool written in Go language, created by HashiCorp and is used to provision or manage infrastructure as code. It supports multiple providers like AWS, Google Cloud, Azure, Openstack, etc. To know about the complete list of providers, check this link: https://www.terraform.io/docs/language/providers/index.html
Now that you have a brief idea about Terraform, let's understand how Terraform fits into IaC space and how it's different from other tools(Chef, Puppet, Ansible, CloudFormation) in its space. Some of the key differences are:
Installing Terraform is pretty straightforward as it comes with a single binary and you need to choose the binary depending upon your platform using this link: https://www.terraform.io/downloads.html
Terraform works by making an API call on your behalf to the provider(AWS, GCP, Azure, etc.) you defined. Now to make an API call, it first needs to be authenticated, and that is done with the help of API keys(AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY). To create an IAM user and its corresponding keys, please check this doc: https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html
Now how much permission users have will be defined with the help of an IAM policy. To attach an existing policy to the user, check this doc: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console
Now in order to use these keys, you can export these as environment variables
There are other ways to configure these credentials; check this doc for more info: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication
Now the next question, how does Terraform know which API to call? This is where you need to define the code in a Terraform configuration file(ends typically with .tf). These configuration files are the code in Infrastructure as Code(IaC).
Every time you run Terraform, it records the information about your infrastructure in a terraform state file(terraform.tfstate). This file stores information in json format and contains a mapping of your terraform resource in your configuration file vs. the real world in AWS cloud. Now when you run the terraform command, it fetches the resource's latest status, compares it with the tfstate file, and determines what changes need to be applied. If Terraform sees a drift, it will re-create or modify the resource.
Note: As you can see, this file is critically important. It is always good to store this file in some remote storage, for example, S3. In this way, your team member should have access to the same state file. Also, to avoid race conditions, i.e., two team members running terraform simultaneously and updating the state file, it's a good idea to apply locking, for example, via DynamoDB. For more information on how to do that, please check this doc: https://www.terraform.io/docs/language/settings/backends/s3.html
Terraform module is a set of Terraform configuration files (*.tf) in a directory. The advantage of using modules is reusability. You can use the terraform module available in the terraform registry or share the module you created with your team members.
With all the prerequisites in place(configuring aws credentials access and secret keys), it’s time to write the first terraform code. Before we start writing our first terraform code, let's see how we are going to organize the files:
NOTE: Filename doesn’t have any special meaning for terraform as long as it ends with .tf extension, but this is a standard naming convention followed in terraform community.
Let first start with main.tf
Now that you understand the syntax for creating a resource, it’s time to write our first terraform code.
In the next section, we create a security group using the aws_security_group resource that allows inbound traffic on port 22.
As you can see in the above code, we are hardcoding the value of ami id, instance type, port, and vpc id. Later on, if we need to change these values, we must modify our main configuration file main.tf. It is much better to store these values in a separate file, and that is what we are going to do in the next step by storing all these variables and their definition in a separate file, variables.tf. Syntax of terraform variables look like this:
So if you need to define a variable for ami id, it looks like this:
Our variables.tf after modifying these values will look like this:
To reference these values in main.tf we just need to add var in front of the variable.
Final main.tf file will look like this:
The last file we are going to check is outputs.tf, whose syntax will look like this:
Where NAME is the name of the output variable and VALUE can be any terraform expression that we would like to be the output.
Now the question is, why do we need it? Take a look with the help of a simple example; when we create this ec2 instance, we don't want to go back to the aws console to grab its public IP; in this case, we can provide the IP address to an output variable.
In the example, we refer to aws_instance resource, ec2-instance identifier, and public_ip attribute. To get more information about the exported attributes, please check this link: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#public_ip
Similarly to get the id of the security group:
Now our terraform code is ready, the first command we are going to execute is
Terraform is reading code and translating it to API calls to providers(aws in this case). Go to your AWS console https://us-west-2.console.aws.amazon.com/ec2/ and you will see your instance should be in the creating stage.
NOTE: If you are executing these commands in a test environment and want to save cost, run terraform destroy command to clean up infrastructure.
In the above example, we have created our first terraform code, now convert this code into modules. Syntax of the module will look like this:
Let's understand this with the help of an example. Create a directory ec2-instance and move all the *.tf files(main.tf, variables.tf and outputs.tf) inside it.
Now in the main directory create a file main.tf, so your directory structure will look like this:
Our module code file will look like this:
variables.tf after the change will look like this:
In the previous example, when we created a module, we gave it a location of our local filesystem under the source. But in a real production environment, we can refer to it from a remote location, for example, GitHub, where we can even version control it.
If you again check the previous example, we use instance type as t2.micro, which is good for test or development environments but may not be suitable for production environments. To overcome this problem, what you can do is tag your module. For example, all the odd tags are for the development environment, and all the even tags are for the production environment.
This is how your module code will look like for the Production environment with changes made under source and instance_type.
In the previous step, we have created our own module. If someone in the company needs to build up an ec2 instance, they shouldn’t write the same terraform code from scratch. Software development encourages the practice where we can reuse the code. To reuse the code, most programming languages encourage developers to push the code to centralize the registry. For example, in Python, we have pip, and in node.js, we have npm. In the case of terraform, the centralized registry is called Terraform registry, which acts as a central repository for module sharing and makes it easier to reuse and discover: https://registry.terraform.io/
As you have learned, creating modules in Terraform requires minimal effort. By creating modules, we build reusable components in the form of IaC, but we can also version control it. Each module change before pushing to production will go through a code review and automated process. You can create a separate module for each environment and safely roll it back in case of any issue.
Squadcast is an incident management tool that’s purpose-built for SRE. Your team can get rid of unwanted alerts, receive relevant notifications, work in collaboration using the virtual incident war rooms, and use automated tools like runbooks to eliminate toil.