Microsoft Azure — Cost Optimization & Governance

Azure is Microsoft's cloud computing platform offering over 200 services. This article covers cost optimization & governance with production-ready configurations, best practices, and hands-on examples.

Why Azure Cost Spiral Happens — And How to Stop It

Azure cost overruns are rarely due to a single mistake. They accumulate from orphaned resources, oversized VMs, forgotten storage accounts, and lack of governance. In production, we've seen teams burn $50k/month on idle AKS clusters. The root cause? No tagging, no budgets, no automated shutdown. This section sets the stage: you need a cost governance framework, not just a dashboard. We'll build one step by step, starting with visibility, then enforcement, then automation. Expect to cut costs by 30-60% without refactoring code.

#!/bin/bash
# Query Azure Cost Management for last 30 days, grouped by resource type
az consumption usage list \
  --billing-period-name $(date +%Y%m -d '1 month ago') \
  --query "[].{Resource: resourceType, Cost: pretaxCost}" \
  --output table

Tagging Strategy: The Foundation of Cost Allocation

Without tags, you can't attribute costs to teams, environments, or projects. Azure Policy can enforce mandatory tags at resource creation. Define a tag schema: Environment (dev/staging/prod), CostCenter, Owner, and ShutdownTime. Use Azure Policy to deny creation of untagged resources. In production, we apply tags via Terraform modules — every resource inherits tags from the module call. This ensures 100% coverage. Example: a VM without 'ShutdownTime' tag gets auto-stopped at 7 PM.

resource "azurerm_resource_group" "example" {
  name     = "rg-prod-app-001"
  location = "East US"
  tags = {
    Environment = "Production"
    CostCenter  = "CC-1234"
    Owner       = "team-alpha"
    ShutdownTime = "19:00"
  }
}

# Policy to enforce tags
resource "azurerm_policy_definition" "require_tags" {
  name         = "require-tags"
  policy_type  = "Custom"
  mode         = "Indexed"
  display_name = "Require specific tags"
  policy_rule  = <<POLICY_RULE
{
  "if": {
    "not": {
      "field": "tags",
      "exists": "true"
    }
  },
  "then": {
    "effect": "deny"
  }
}
POLICY_RULE
}

Rightsizing VMs: Stop Paying for Idle Capacity

Most production VMs are overprovisioned. Use Azure Advisor recommendations, but verify with custom metrics. Collect CPU, memory, and disk IOPS over 30 days. Rightsize down: if average CPU < 20%, drop to next tier. For predictable workloads, use Reserved Instances (RI) or Savings Plans. For dev/test, use Azure Spot VMs. Automate rightsizing with Azure Automation runbooks that resize VMs based on metrics. Example: a runbook that checks CPU and resizes if underutilized for 7 days.

# Azure Automation runbook to resize underutilized VMs
param([string]$ResourceGroupName, [string]$VMName)

$vm = Get-AzVM -ResourceGroupName $ResourceGroupName -Name $VMName
$metrics = Get-AzMetric -ResourceId $vm.Id -MetricName "Percentage CPU" -TimeGrain 00:05:00 -StartTime (Get-Date).AddDays(-7) -EndTime (Get-Date)
$avgCpu = ($metrics.Data | Measure-Object -Property Average -Average).Average

if ($avgCpu -lt 20) {
    $currentSize = $vm.HardwareProfile.VmSize
    $newSize = Get-AzVMSize -Location $vm.Location | Where-Object {$_.NumberOfCores -lt $vm.HardwareProfile.VmSize.NumberOfCores} | Select-Object -First 1
    if ($newSize) {
        $vm.HardwareProfile.VmSize = $newSize.Name
        Update-AzVM -VM $vm -ResourceGroupName $ResourceGroupName
        Write-Output "Resized $VMName from $currentSize to $($newSize.Name)"
    }
}

Storage Tiering and Lifecycle Management

Storage costs sneak up because data accumulates. Use Azure Blob Storage access tiers: Hot, Cool, Archive. Set lifecycle management policies to move blobs automatically. For example: move blobs not accessed for 30 days to Cool, 90 days to Archive. Delete snapshots older than 7 days. Use Azure Policy to enforce that all storage accounts have lifecycle rules. In production, we reduced storage costs by 70% by archiving logs after 30 days.

{
  "rules": [
    {
      "name": "move-to-cool",
      "enabled": true,
      "type": "Lifecycle",
      "definition": {
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["logs/"]
        },
        "actions": {
          "baseBlob": {
            "tierToCool": {
              "daysAfterModificationGreaterThan": 30
            },
            "tierToArchive": {
              "daysAfterModificationGreaterThan": 90
            },
            "delete": {
              "daysAfterModificationGreaterThan": 365
            }
          }
        }
      }
    }
  ]
}

Automating Shutdown of Non-Production Resources

Dev/test environments often run 24/7 but are used only 8 hours a day. Use Azure Automation to start/stop VMs on a schedule. Tag VMs with 'ShutdownTime' and 'StartupTime'. A runbook reads tags and executes shutdown. For AKS clusters, scale node pools to 0 during off-hours. For databases, use Azure SQL Serverless or pause Azure SQL Data Warehouse. In production, we saved $15k/month by shutting down 50 dev VMs overnight.

# Runbook to stop VMs based on ShutdownTime tag
$vms = Get-AzVM | Where-Object {$_.Tags.Keys -contains "ShutdownTime"}
$currentTime = Get-Date -Format "HH:mm"

foreach ($vm in $vms) {
    $shutdownTime = $vm.Tags["ShutdownTime"]
    if ($currentTime -eq $shutdownTime) {
        Stop-AzVM -ResourceGroupName $vm.ResourceGroupName -Name $vm.Name -Force
        Write-Output "Stopped $($vm.Name)"
    }
}

Azure Policy and Budgets: Enforcing Cost Governance

Cost governance requires enforcement, not just visibility. Use Azure Policy to restrict VM sizes (e.g., only allow D-series), deny public IPs for dev, and require tags. Set budgets with alerts at 50%, 90%, and 100% of spend. When budget is exceeded, trigger an automation runbook to lock resources or send a Slack notification. In production, we use Azure Policy to block creation of expensive GPU VMs in non-prod subscriptions.

{
  "if": {
    "allOf": [
      {
        "field": "type",
        "equals": "Microsoft.Compute/virtualMachines"
      },
      {
        "field": "Microsoft.Compute/virtualMachines/sku.name",
        "notIn": ["Standard_D2s_v3", "Standard_D4s_v3", "Standard_D8s_v3"]
      }
    ]
  },
  "then": {
    "effect": "deny"
  }
}

Cost Optimization for AKS (Azure Kubernetes Service)

AKS clusters can be expensive due to overprovisioned node pools and idle pods. Use cluster autoscaler to scale nodes based on demand. Use node pools with Spot VMs for batch jobs. Right-size pod requests/limits — many teams set requests too high. Use Azure Policy for AKS to enforce resource limits. For dev clusters, scale to 0 nodes during off-hours. In production, we reduced AKS costs by 40% by switching to Spot VMs for non-critical workloads.

apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-autoscaler-status
  namespace: kube-system
data:
  # Enable cluster autoscaler with min 1, max 10 nodes
  scale-down-delay-after-add: "10m"
  scale-down-unneeded-time: "10m"
  max-node-provision-time: "15m"

Monitoring and Alerting on Cost Anomalies

Cost spikes happen — a misconfigured resource, a DDoS attack, or a runaway pipeline. Use Azure Cost Management alerts for anomaly detection. Set up a custom dashboard in Azure Monitor showing daily cost by resource type. Use Log Analytics to query cost data and trigger alerts when daily spend exceeds 2x the average. In production, we have a runbook that automatically suspends a subscription if cost spikes 500% above baseline.

// Query cost anomalies: days where cost > 2x average of last 30 days
let avgCost = materialize(
    UsageDetails
    | where Date between (ago(30d) .. ago(1d))
    | summarize avgDaily = avg(CostInBillingCurrency)
);
UsageDetails
| where Date >= ago(1d)
| summarize dailyCost = sum(CostInBillingCurrency)
| where dailyCost > avgCost * 2
| project Date, dailyCost, avgCost

Reserved Instances and Savings Plans: Strategic Commitment

For predictable workloads, commit to 1 or 3 years with Reserved Instances (RIs) or Azure Savings Plans. RIs are VM-specific; Savings Plans apply to compute across regions. Use Azure Cost Management's 'Reservation recommendations' to identify candidates. But don't commit to RIs for workloads that may change. In production, we buy RIs for baseline capacity and use pay-as-you-go for burst. This hybrid approach saves 40% on base load.

#!/bin/bash
# Purchase a 1-year Reserved Instance for Standard_D4s_v3 in East US
az reservations reservation-order calculate \
  --sku 'Standard_D4s_v3' \
  --location 'eastus' \
  --quantity 5 \
  --term 'P1Y' \
  --billing-scope 'Subscription' \
  --display-name 'Prod-D4s-v3-RI'

Governance at Scale: Management Groups and Subscriptions

For enterprises, cost governance starts with Azure Management Groups. Structure them by environment (Prod, NonProd, Sandbox) and apply policies at the group level. Use subscription quotas to limit spend per team. Implement Azure Blueprints to deploy consistent environments with cost controls. In production, we have a 'Sandbox' management group with a $500/month budget and auto-deletion of resources after 30 days.

{
  "properties": {
    "displayName": "NonProd",
    "policyDefinitions": [
      {
        "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/require-tags"
      },
      {
        "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/restrict-vm-sizes"
      }
    ]
  }
}

Continuous Optimization: Culture and Automation

Cost optimization is not a one-time project. Build a culture of cost awareness: include cost in code reviews, run monthly cost reviews, and use dashboards. Automate remediation: if a resource is idle for 7 days, send an email; if no response in 3 days, delete it. Use Azure Logic Apps to orchestrate workflows. In production, we have a 'cost champion' in each team who reviews weekly reports.

{
  "definition": {
    "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
    "actions": {
      "Find_idle_VMs": {
        "type": "ApiConnection",
        "inputs": {
          "host": {
            "connection": {
              "name": "@parameters('$connections')['azureautomation']['connectionId']"
            }
          },
          "method": "post",
          "path": "/runbooks/Find-IdleVMs/start"
        }
      },
      "Send_email": {
        "type": "ApiConnection",
        "inputs": {
          "host": {
            "connection": {
              "name": "@parameters('$connections')['office365']['connectionId']"
            }
          },
          "method": "post",
          "path": "/v2/Mail/SendMail",
          "body": {
            "To": "team@example.com",
            "Subject": "Idle VMs found",
            "Body": "Please review and take action."
          }
        }
      }
    },
    "triggers": {
      "recurrence": {
        "frequency": "Week",
        "interval": 1
      }
    }
  }
}

Putting It All Together: A Cost Governance Framework

Combine all previous steps into a repeatable framework: 1) Tagging and policy enforcement, 2) Rightsizing and tiering, 3) Scheduled shutdown, 4) Budgets and alerts, 5) Reserved Instances, 6) Continuous monitoring. Implement this as a Terraform module or Azure Blueprint. In production, we deploy this framework to every new subscription. It takes 2 hours to set up and saves 40% on average. Start with one subscription, measure savings, then roll out.

module "cost_governance" {
  source = "github.com/yourorg/terraform-azure-cost-governance"
  
  subscription_id = "00000000-0000-0000-0000-000000000000"
  environment     = "production"
  budget_amount   = 10000
  vm_whitelist    = ["Standard_D2s_v3", "Standard_D4s_v3"]
  shutdown_time   = "19:00"
  enable_ri       = true
}

Network Cost Optimization: Reducing Egress and Data Transfer

Network costs — especially egress (data leaving Azure) — are one of the most overlooked areas of cloud spend. Egress is charged per GB and can exceed compute costs for data-intensive workloads. Optimization strategies: use Azure CDN or Front Door to cache content at edge locations, reducing origin server egress. Use Virtual Network peering instead of VPN/ExpressRoute for intra-region traffic (free within same region). Consolidate services in the same region to minimize cross-region data transfer costs. For large data transfers, use Azure Data Box instead of network transfer. Enable compression on application responses and optimize API payload sizes. In production, we reduced egress costs by 60% by adding a CDN in front of our static assets and moving cross-region services to the same availability zone. Monitor egress costs using the 'Data Transfer' meter in Cost Management and set alerts for spikes.

#!/bin/bash
# Check data transfer costs by service type
az consumption usage list \
  --billing-period-name $(date +%Y%m) \
  --query "[?contains(meterCategory, 'Data Transfer')].{Service: serviceName, Cost: pretaxCost}" \
  --output table

# Output example:
# ServiceName                    Cost
# ExpressRoute                   $2,345.00
# Virtual Network Data Transfer  $890.00
# CDN Egress                     $12,450.00

Database Cost Optimization: Serverless, Autoscale, and Reserved Capacity

Databases are often the second-largest cost driver after compute. Azure SQL Database offers serverless compute tier that auto-pauses during inactivity — ideal for dev/test and intermittent workloads, saving up to 60% compared to provisioned. For Cosmos DB, use autoscale mode to pay only for the throughput you consume, and enable TTL (time-to-live) to automatically expire old data. For Azure SQL Managed Instance, use reserved capacity for predictable workloads (up to 33% savings on 1-year, 55% on 3-year). Implement elastic pools to share resources across multiple databases with varying usage patterns. Use Azure Database for PostgreSQL/Citrus flexible server with burstable compute for low-utilization workloads. In production, we saved $15k/month by moving 20 development databases to SQL Serverless and switching Cosmos DB containers from manual 10K RU/s to autoscale 1K-10K RU/s. Always right-size your DTU/vCore selection based on Query Store insights.

#!/bin/bash
# Get underutilized SQL databases
az sql db list --resource-group prod-rg --server prod-server \
  --query "[?status == 'Online'].{Name: name, Sku: currentServiceObjectiveName, SizeGB: maxSizeBytes}" \
  --output table

# Check DTU consumption via metrics
az monitor metrics list \
  --resource /subscriptions/.../sqlDatabases/prod-db \
  --metric dtu_consumption_percent \
  --interval PT1H \
  --aggregation Average \
  --query "[?average < 20].timestamp"

FinOps Framework: Aligning Cost Optimization with Business Value

FinOps (Financial Operations) is the industry-standard framework for managing cloud costs as a cross-functional discipline involving engineering, finance, and business teams. The FinOps lifecycle has three phases: Inform (visibility and allocation), Optimize (rightsizing and commitment discounts), and Operate (continuous improvement and governance). In Azure, Cost Management supports all three phases. Mature FinOps practices assign cost ownership to engineering teams (showback/chargeback), track unit economics (cost per transaction, cost per user), and conduct regular 'FinOps sprint reviews.' In production, we implemented a showback model where each team sees their costs in a Power BI dashboard. The team that reduced cost per transaction by 30% won the quarterly FinOps award. FinOps is not about minimizing spend — it's about maximizing the business value per cloud dollar. A cost optimization that reduces performance or reliability is a false economy.

// Calculate cost per transaction (unit economics)
let totalCost = materialize(
    UsageDetails
    | where Date between (startofday(now()) .. endofday(now()))
    | where TagTeam == "Checkout"
    | summarize TotalCost = sum(CostInBillingCurrency)
);
let totalTransactions = materialize(
    requests
    | where timestamp > startofday(now())
    | where name == "POST /api/checkout"
    | count
);
totalCost
| extend CostPerTransaction = TotalCost / totalTransactions
| project TotalCost, totalTransactions, CostPerTransaction