IaC — State Corruption from Untagged S3 Buckets

Every modern software team has faced the same nightmare: a production server dies at 2am, and the engineer who built it left the company six months ago. Nobody wrote anything down. The replacement server gets rebuilt from memory, Slack messages, and guesswork — and it's never quite right. This isn't a people problem. It's a process problem, and Infrastructure as Code (IaC) exists specifically to eliminate it. When your infrastructure lives in code, it lives in Git, in pull requests, in code reviews, and in your CI/CD pipeline — just like the application it runs.

Before IaC, provisioning infrastructure meant logging into a cloud console, clicking through wizards, and hoping the person next to you was watching and taking notes. Every environment — dev, staging, production — drifted apart over time because human hands configured them differently. This 'configuration drift' is the silent killer of reliable deployments. IaC solves this by making infrastructure declarative and repeatable: you describe the desired state of your system, and a tool like Terraform or Ansible figures out how to get there. The same code that spins up your staging environment spins up production, byte for byte.

By the end of this article you'll understand why IaC exists at a systems level, know the difference between declarative and imperative approaches, and have a real working Terraform + GitHub Actions CI/CD pipeline you can adapt for your own projects. You'll also know the two mistakes that catch almost every intermediate engineer off guard when they go to use IaC in a team setting.

Declarative vs Imperative IaC — Choosing the Right Mental Model

There are two ways to tell someone how to make a cup of coffee. The imperative way: 'Boil water. Measure 18g of beans. Grind them. Pour water at 94°C. Wait 4 minutes.' The declarative way: 'I want a black filter coffee in this cup.' The declarative approach lets the system figure out the steps.

This distinction is the most important conceptual split in IaC. Terraform is declarative — you describe what your infrastructure should look like, and Terraform calculates the diff between current state and desired state, then makes the changes. Ansible is imperative by default — you write a sequence of tasks that run top to bottom. Both are valid. The right choice depends on what you're managing.

Declarative tools shine for cloud resource provisioning: creating VPCs, EC2 instances, databases, and load balancers. You don't want to think about order of operations — you just want the result. Imperative tools shine for configuration management: installing packages, editing config files, restarting services. The order genuinely matters there.

In a mature DevOps pipeline you'll often use both: Terraform provisions the server, Ansible configures it. Understanding why they work differently stops you from fighting the tool when it doesn't behave the way you expect.

IaC in a Real CI/CD Pipeline — Automate the Infrastructure Itself

Knowing how to run Terraform locally is a starting point. But IaC's real power unlocks when it runs automatically inside your CI/CD pipeline. Think about it: your application code goes through automated testing before it deploys. Why should your infrastructure changes be any different? A pull request that adds a new RDS database should go through the same review process as a pull request that adds a new API endpoint.

The pattern that works in production is this: on every pull request, run terraform plan and post the output as a PR comment. This gives reviewers an exact, human-readable diff of what will change in the real cloud — before anyone approves it. On merge to main, run terraform apply automatically. No one runs Terraform from their laptop. Ever.

This approach solves three problems at once. It creates an audit trail (every infrastructure change is a Git commit with an author and a timestamp). It prevents 'works on my machine' infrastructure (the pipeline always runs from a clean state). And it forces infrastructure changes through code review, which catches mistakes before they hit production.

The GitHub Actions workflow below implements this exact pattern. It's the real thing — not a toy example.

Remote State and Modules — The Patterns That Make IaC Scale

A single main.tf file works fine for a hobby project. It falls apart the moment you have two engineers, two environments, or two services. This is where two patterns become non-negotiable: remote state backends and modules.

Remote state is how Terraform remembers what it already built. Without it, every terraform apply is flying blind. With a remote backend — like an S3 bucket with a DynamoDB lock table — the state file lives in the cloud, is accessible to everyone on the team, and is locked during applies so two engineers can't run it simultaneously and corrupt each other's work.

Modules are reusable Terraform components. Think of them as functions for infrastructure. Instead of copy-pasting the same EC2 + security group + IAM role configuration for every service, you write it once as a module and call it with different variables for each service. This is the IaC equivalent of the DRY principle and it's what separates a professional IaC setup from a pile of disconnected config files.

Below is a minimal but real remote backend configuration alongside a module call pattern. This is the structure you'd actually find in a production repository.

IaC Security and Secrets Management — Don't Leak Your Infrastructure's Keys

Infrastructure code often requires secrets: API keys, database passwords, cloud provider credentials. A common rookie mistake is hardcoding these in the IaC files. Terraform state files, in particular, store resource attributes in plaintext, which can include sensitive values like database passwords or IAM secret keys.

The rule: IaC code should never contain secrets. Instead, use environment variables, encrypted variables in your CI/CD platform, or a dedicated secrets manager like HashiCorp Vault, AWS Secrets Manager, or GitHub Actions Secrets. For Terraform, use the sensitive = true attribute on outputs, and avoid outputting secrets in plan output. Use data sources that read from a secrets manager rather than embedding values.

Another critical practice: never commit .tfstate files to version control. Even if you delete them later, secrets are permanently exposed in Git history. Add .tfstate and .tfstate.backup to .gitignore on day one. Use a remote backend with encryption at rest.

Below is an example of using environment variables with Terraform, and a pattern for reading secrets from AWS Secrets Manager.

IaC Testing and Validation — Ensure Your Infrastructure Works Before You Apply

Writing infrastructure code without testing is like deploying a microservice without unit tests. You're one typo away from deleting a production database. IaC testing isn't as mature as application testing, but it's evolving fast. Here are three levels of validation every IaC pipeline needs:

1. Syntax and static analysis: Use terraform validate to catch basic HCL errors. Use tools like tflint for style and potential bugs, and checkov or tfsec for security policy violations. Run these on every PR before the plan step.
2. Plan review: The manual step where a human reviews the terraform plan output. This catches logical mistakes — like changing a security group that breaks connectivity, or accidentally destroying a stateful resource.
3. Integration testing: Tools like Terratest let you write Go tests that deploy real infrastructure, run assertions against it, and then destroy it. This is the gold standard, but it's expensive and slow. Use it sparingly for critical resources.

Below is a minimal GitHub Actions step that runs static analysis before the plan. Integrate this into your workflow to catch issues early.

Frequently Asked Questions