In this article, I will walk you through what I have learned while developing a system in AWS using Terraform. For context, I created this project as a requirement for my previous internship. I will provide the GitHub repository for the requirements, architecture, and execution steps.

This project implements a highly scalable and available web application adhering to the best practices regarding availability, scalability, and reliability. I have developed this project for nearly four weeks, enhancing my skills in Cloud, Infrastructure as code, and engineering altogether, and here are some of the lessons I have learned and applied during the process:

Big Picture to Small Picture

Working on this project is my first time working on an actual cloud infrastructure, which I have craved for most of my time learning cloud. Having momentum and excitement, I head down to work immediately, approaching the project one resource at a time. I started creating the virtual private cloud and EC2 instances manually. That’s when I made my first mistake; I forgot to understand the whole architecture completely before working on the project. I do need an independent EC2 instance for my bastion host; however, the other EC2 instances should be configured using an Auto-Scaling group, which in Terraform is different from configuring a single EC2 instance.

The lesson I learned here is that when you’re working on a project, individually or with a group, it is important to understand the big picture of your project before proceeding to do the actual work. While developing, it is easy to get lost in many different approaches to complete your system, but if you have your goals set and the big picture is clear, you will not wander far from your objectives. But I am still glad to have failed on my earlier approach, which has led me to another vital lesson: Fail Fast.

Fail Fast

This lesson is quite contradictory to my previous learning, but it is crucial to discover the fine line on where they complement each other. Planning is essential, but choosing which approach to carry to the development phase is often overwhelming. Just like what I did earlier, I took an approach that failed; for some, it is a setback, but I took it as an indicator that my approach would not work and moved on to another approach. The main lesson here is that you shouldn’t be afraid to experiment with your project, but if you’re unsure if it will work for you, you should aim to fail fast to avoid wasting time any further. The same goes for different resources, designs, and system architecture; identify what works for your system and meets your requirements.

Understand Tradeoffs

Next on the list is understanding tradeoffs. AWS, just like any cloud provider, gives its users a set of standards to adhere to its best practices when it comes to security, reliability, and availability. For AWS, it is the AWS Well-Architected Framework. However, following these standards will be costly for a simple project such as this one. I will give you an example. Below is the architecture of the project:

As you can see in the diagram, in Availability Zone 2, there is a NAT Gateway, and none for the other one. A system that follows the AWS Well-Architected Framework should have a NAT Gateway for each Availability Zone to ensure high availability for our system. In case one availability zone crashes, having a NAT Gateway to both AZs ensures that our system retains its access to the internet. However, it is costly. This is where tradeoffs come into play. We traded high availability to save more money; it all comes down to which you, your company, or your client prioritizes. It is best to communicate and be transparent about the tradeoffs of each system; others compromise security, functionality, and redundancy to cut down costs, and that is okay, depending on priorities.

Version Control and Documentation

Working on a project like this makes me realize the importance of version control more than ever. Since I am new to infrastructure as code, there are times when my code would fail, and I would be stuck and unable to progress, or at least go back to the previous version where the code is working. That is because I failed to branch out my source code. In this project, I learned to branch out my code depending on the additions or fixes I made to ensure that whenever there is an error or I get stuck, I can always go back to the previous stable version.

Avoid Redundancy

The last thing I learned from this project is an already existing software engineering principle: DRY or Don’t Repeat Yourself. This project requires us to use modules for Terraform, which is a way to configure our resources and later reuse them in different parts of our code. It is crucial to configure your modules dynamically to be reusable, which is why I used variables. Using modules eliminates redundancy and ensures high reusability for your code.

Overall, this project has helped me drastically in improving my Cloud knowledge and Infrastructure as Code usage experience. Not only does this code challenge me, but it also encourages me to collaborate with my peers and communicate well with others. The real lesson from this project is the practices that I gained from doing this project. I was able to be patient and be aggressive at the same time, I was able to be cautious and have a backup plan, and most importantly, I was able to think outside the box. If you wish to do this project as well, you can check my repository in the references down below.

Reference:

• https://tutorialsdojo.com/no-comment-how-technical-documentation-helps-your-team/
• Project Repository