Cloud Cost Optimisation — The $120K Forgotten Database

Cloud bills have a nasty habit of doing one thing: going up. A startup spins up a few EC2 instances to test an idea, forgets to turn them off, adds an RDS database 'just for dev', and six months later the founders are staring at a $40,000 invoice wondering how it happened. This isn't a cautionary tale — it's Tuesday at most engineering companies. Cloud providers design their consoles to make provisioning effortless and deprovisioning forgettable. That asymmetry is expensive.

The problem cloud cost optimisation solves isn't just waste — it's invisible waste. Unused Elastic IPs, idle load balancers, forgotten S3 buckets full of old logs, over-provisioned RDS instances running at 8% CPU — none of these announce themselves. They silently accumulate. Cost optimisation is the discipline of making cloud spend intentional: every resource should be the right size, running only when needed, and tagged so you know exactly which team or product is paying for it.

By the end of this article you'll know how to identify and eliminate the four biggest categories of cloud waste, how to write Infrastructure as Code that enforces cost-aware defaults, how to use reserved capacity and spot instances strategically, and how to build a tagging policy that gives you real visibility. These are the same techniques engineering teams at scale use to routinely cut cloud bills by 30–60% without touching a single line of application code.

What is Cloud Cost Optimisation?

Cloud cost optimisation is the practice of continuously aligning cloud resource provisioning with actual workload requirements. It's not about cutting corners — it's about eliminating the gap between what you pay and what you need. That gap is almost always wider than you think.

Most engineers treat cloud spend as a fixed cost, like rent. It's not. Every resource is individually billable, and providers design their consoles to make adding resources frictionless. The result? A typical AWS account has 30-40% waste: idle instances, oversized databases, unused storage, and orphaned resources that nobody remembers creating.

Each level compounds. Without tactical cleanup, strategic discounts are wasted on empty resources. Without cultural enforcement, tactical wins revert within a quarter.

// TheCodeForge — Terraform example enforcing cost-aware defaults
resource "aws_instance" "web" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = var.env == "production" ? "t3.medium" : "t3.micro"

  // Enforce tags for cost visibility
  tags = {
    Name        = "web-${var.env}"
    Environment = var.env
    Owner       = var.team
    CostCenter  = var.cost_center
    ExpiresOn   = var.env == "staging" ? timeadd(timestamp(), "168h") : "never"
  }

  // Auto-stop for non-production
  user_data = var.env != "production" ? <<-EOF
    #!/bin/bash
    echo "0 20 * * * shutdown -h now" | crontab -
  EOF : null
}

10-Point Cloud Cost Optimization Checklist

Use this checklist as a quick reference to audit your cloud spend. Each point addresses a common source of waste or a best practice for cost control.

1. Stop idle compute resources. Identify instances with less than 5% CPU over the last 14 days. Stop or terminate them. Use AWS Instance Scheduler to automate stop/start.
2. Rightsize over-provisioned instances. Run Compute Optimizer or cross-check CloudWatch metrics. Downsize to the smallest instance type that meets your workload’s peak requirements.
3. Remove orphaned EBS volumes and Elastic IPs. Unattached volumes and unused IPs accrue costs. List and delete them monthly.
4. Set S3 lifecycle policies. Transition logs and backups to cheaper storage classes (Standard-IA → Glacier → Deep Archive) and expire old objects.
5. Abort incomplete multipart uploads. Add a lifecycle rule to remove incomplete uploads older than 7 days. They accumulate hidden storage cost.
6. Enforce tagging on all resources. Use IaC policy (Terraform checkov, CloudFormation stack policy) to require Owner, Environment, and CostCenter tags. Automatically terminate untagged resources.
7. Buy Reserved Instances or Savings Plans only after rightsizing. Commit for steady-state, rightsized workloads. Start with 1-year partial upfront to preserve flexibility.
8. Use Spot instances for fault-tolerant workloads. Batch processing, CI/CD workers, and stateless microservices are ideal. Implement termination handling.
9. Set up cost anomaly alerts. Use AWS Budgets or third-party tools to alert on >30% increases in top services. Assign an owner to review alerts weekly.
10. Conduct monthly cost reviews. Review Cost Explorer, look for new services or unexplained spikes. Maintain a shared spreadsheet of cost optimization actions.

Rightsizing: Match Instance Size to Actual Workload

Rightsizing means picking the instance type and size that matches your workload's actual resource consumption. The default — t3.medium for everything — is almost always wrong. A web server that serves 100 req/s might need 4 vCPUs; one that serves 5 req/s might run fine on a burstable nano.

AWS Compute Optimizer and Azure Advisor give recommendations based on historical CPU, memory, and network utilisation. But they're conservative — they only suggest downsizing if utilisation is below a threshold (e.g., 40% max CPU over 2 weeks). In practice, most engineering teams can downsize 50% of their instances with no performance impact.

Rightsizing is a continuous process. Quarterly reviews catch new waste from autoscaling groups that launched larger instances during a peak and never scaled down.

#!/bin/bash
# TheCodeForge — Fetch EC2 instances with avg CPU <10% over last 14 days
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-1234567890abcdef0 \
  --start-time $(date -d '14 days ago' +%Y-%m-%dT00:00:00Z) \
  --end-time $(date +%Y-%m-%dT23:59:59Z) \
  --period 86400 \
  --statistics Average \
  --query 'Datapoints[*].Average' \
  --output json | jq 'add/length | . < 10'

Reserved Instances and Savings Plans: Commit to Save

Reserved Instances (RIs) and Savings Plans are discount programs in exchange for committed usage. If you know you'll run a database or web server for the next 1-3 years, you can pay upfront or partially upfront and get 30-72% off the on-demand rate.

Senior engineers treat RIs as a second-order optimisation — after rightsizing. You don't want to commit to a m5.large for 3 years only to discover you could have moved to t3.medium and saved more without the commitment.

One real strategy: use a mix of 1-year partial upfront for predictable workloads and 3-year all upfront for baseline production. This balances discount depth with financial flexibility.

# TheCodeForge — Calculate optimal RI coverage using AWS Pricing API
import boto3
import pandas as pd

pricing = boto3.client('pricing', region_name='us-east-1')
ce = boto3.client('ce')

# Get EC2 usage over last 30 days
usage = ce.get_cost_and_usage(
    TimePeriod={'Start': '2026-03-22', 'End': '2026-04-22'},
    Granularity='DAILY',
    Metrics=['UsageQuantity', 'UnblendedCost'],
    GroupBy=[{'Type': 'DIMENSION', 'Key': 'INSTANCE_TYPE'}]
)

# Identify top 5 instance families by cost
costs = {}
for item in usage['ResultsByTime']:
    for group in item['Groups']:
        key = group['Keys'][0]
        costs[key] = costs.get(key, 0) + float(group['Metrics']['UnblendedCost']['Amount'])

top_5 = sorted(costs.items(), key=lambda x: x[1], reverse=True)[:5]
print("Top 5 instance families by cost:")
for family, cost in top_5:
    print(f"{family}: ${cost:.2f}")

# Suggested RI coverage: 50% of top family with 1-year partial upfront

Savings Plans vs Reserved Instances: Which Should You Choose?

Both Reserved Instances (RIs) and Savings Plans offer deep discounts in exchange for committing to a certain dollar amount per hour (Savings Plans) or a specific instance configuration (RIs). The right choice depends on how predictable and stable your workloads are.

Standard RIs give the highest discount (up to 72%) but lock you to a specific instance family in a specific region. They are best for steady-state, long-lived workloads that won't change architecture — e.g., a production MySQL database on db.r5.large.

Convertible RIs offer slightly lower discounts (40-60%) but allow you to change instance family, size, or region. They’re useful if you anticipate moderate changes but still want a commitment discount.

Compute Savings Plans apply to any EC2, ECS, EKS, or Fargate compute usage within a region. Discounts are 30-50%, but you gain flexibility to change instance types, sizes, or even move between compute services. Ideal for containerized workloads.

EC2 Instance Savings Plans are similar to Standard RIs — they apply to a specific instance family in a region — but are slightly more flexible because they cover any instance size within that family.

If your architecture is cloud-native and you use containers or Lambda, Compute Savings Plans are the safest bet. If you have legacy monoliths on known instance types, Standard RIs maximize savings. Never buy a 3-year RI until you have at least 6 months of stable usage data — financial flexibility is worth the lower discount.

Spot and Preemptible Instances: Cheap Compute for the Brave

Spot instances (AWS) and preemptible VMs (GCP) let you use spare cloud capacity at a steep discount — typically 60-90% off on-demand price. The trade-off: the cloud provider can reclaim the instance with just a few minutes' notice (2 minutes in AWS, 30 seconds in GCP).

The challenge is handling termination gracefully. Your application must save state or be able to restart. AWS sends a Spot Instance Termination Notice event 2 minutes before reclaiming the instance. You can catch this via the instance metadata endpoint.

Avoid spot for stateful databases, single-instance workloads, or anything that can't tolerate an abrupt stop. The cost savings are real, but the operational cost of re-architecting for spot can be significant.

# TheCodeForge — Listen for spot termination and checkpoint work
import requests
import json
import time

METADATA_URL = "http://169.254.169.254/latest/meta-data/spot/termination-time"

def check_termination():
    try:
        response = requests.get(METADATA_URL, timeout=5)
        if response.status_code == 200:
            print("⚠️  Spot termination imminent. Checkpointing...")
            # Save state to S3 or distribute work
            return True
    except requests.exceptions.RequestException:
        pass
    return False

while True:
    if check_termination():
        # Graceful shutdown logic
        break
    # Normal processing
    time.sleep(5)

Tagging and Cost Allocation: Visibility Is the Prerequisite

You can't optimise what you can't see. Tagging is the foundation of cost visibility — attaching metadata (key-value pairs) to every cloud resource so you can attribute costs to teams, products, environments, or cost centers.

But tags are only useful if they're: 1. Mandatory: IaC policies should reject deployments without required tags. 2. Standardised: A fixed set of tags (e.g., Owner, Environment, CostCenter, Project) used everywhere. 3. Enforced: Resources created without required tags are automatically terminated or reported.

Without enforcement, tags become optional and quickly rot. After three months, half your resources are untagged and you're back to guessing who's spending what.

AWS provides Cost Allocation Tags that appear in the Cost Explorer and detailed billing reports. You can also create user-defined tags. Once tags are in place, you can run reports per team, set budget alerts for specific CostCenter tags, and even block untagged resources from launching via SCP (Service Control Policies) or Custom Lambda functions.

# TheCodeForge — AWS Lambda function that terminates untagged resources
import boto3
from botocore.exceptions import ClientError

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')
    required_tags = ['Owner', 'Environment', 'CostCenter']
    
    instances = ec2.describe_instances(
        Filters=[{'Name': 'instance-state-name', 'Values': ['running', 'stopped']}]
    )
    
    for reservation in instances['Reservations']:
        for instance in reservation['Instances']:
            tags = {t['Key']: t['Value'] for t in instance.get('Tags', [])}
            missing = [tag for tag in required_tags if tag not in tags]
            if missing:
                print(f"Terminating instance {instance['InstanceId']} — missing tags: {missing}")
                # Option 1: Send alert via SNS
                # Option 2: Terminate directly (use with caution!)
                # ec2.terminate_instances(InstanceIds=[instance['InstanceId']])

The Cost of Untagged Resources: A Visual Impact

Tags are the single most important tool for cost attribution, but they only work when enforced. Untagged resources are invisible in cost reports — they fall into a generic "Untagged" bucket that tells you nothing about ownership, project, or environment. This invisibility is a direct cause of cloud waste.

The diagram below shows the chain reaction from an untagged resource to a growing bill. When a developer provisions an EC2 instance without tags:

1. The instance shows up in Cost Explorer under "No Tag: Environment".
2. The operations team doesn’t know who owns it or why it exists.
3. No one is responsible for stopping it, so it runs indefinitely.
4. The cost is buried in a generic line item — nobody notices.
5. After months, the bill has crept up by thousands of dollars.

With tagging, the opposite happens: resources are automatically attributed to a team, cost allocation reports surface anomalies, and the team lead gets a budget alert when spend exceeds threshold.

The impact is not just financial. Untagged resources slow down incident response (who to page?), hinder compliance audits, and make capacity planning guesswork. A single untagged production database can delay a root cause analysis by hours because no one knows who stacked it.

The visual below summarises the flow from untagged resource to cost leak. Use it to justify enforcing tag policies across your organization.

Storage Lifecycle Management: Stop Paying for Old Logs

Storage costs are the sneakiest creepers in your cloud bill. Data accumulates, and once written, it almost never gets deleted. S3 charges per GB per month, and that cost grows linearly with data volume. A project that stores 500GB of logs and deletes nothing will be paying for 5TB in a year.

The solution is lifecycle policies — automatically transition objects to cheaper storage classes and expire them after a set period. Typical data lifecycle: - 0-30 days: S3 Standard (hot, frequent access) - 30-90 days: S3 Infrequent Access (lower storage cost, higher retrieval cost) - 90-365 days: S3 Glacier (long-term archival, retrieval takes minutes-hours) - 365+ days: S3 Glacier Deep Archive (cheapest, retrieval takes 12-24 hours) - Expiration: Delete objects after, say, 3 years.

Set these policies on every bucket from day one. Retroactively adding them to existing buckets with millions of objects can be done via S3 Batch Operations.

Also watch for incomplete multipart uploads. S3 charges you for the chunks even if the upload never finished. They accumulate silently. Add a rule to expire incomplete uploads after 7 days.

{
  "Rules": [
    {
      "Id": "Standard-to-IA-after-30d",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        },
        {
          "Days": 365,
          "StorageClass": "DEEP_ARCHIVE"
        }
      ],
      "Expiration": {
        "Days": 1095
      }
    },
    {
      "Id": "AbortIncompleteMultipartUpload",
      "Status": "Enabled",
      "Filter": {"Prefix": ""},
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    }
  ]
}