GCP Billing Setup: Budgets, Alerts, and Cost Controls

A startup burned $72,000 in one weekend because a single misconfigured Dataflow job spun up 200 workers and never shut down. Their GCP billing alerts were set to email only, and nobody checked email on Saturday. That's the kind of pain that makes you religious about budgets, alerts, and cost controls. This isn't about saving pennies—it's about preventing existential financial events. GCP gives you the tools: budgets that trigger alerts at thresholds, Pub/Sub notifications that can auto-shutdown resources, and quota policies that cap usage. But if you set them up wrong, they're just feel-good checkboxes. Here's how to configure them so they actually protect you.

Why Budgets Alone Won't Save You

Budgets in GCP are not hard limits—they are monitoring thresholds. When you create a budget, you define an amount and a set of alert thresholds (e.g., 50%, 90%, 100% of budget). When actual or forecasted spend crosses a threshold, GCP sends a notification. But that's it. It does not stop spending. A budget is a speedometer, not a brake. Many teams assume budgets prevent overruns, then get surprised when the bill arrives. The key insight: budgets are the first layer of defense, but they must be paired with automated actions (via Cloud Functions or Pub/Sub) to actually enforce cost limits. Without that, you're just getting a warning after the damage is done.

#!/bin/bash
# Create a budget with 50% and 90% alerts
PROJECT_ID="my-project"
BUDGET_NAME="monthly-budget"
AMOUNT=1000.00

gcloud billing budgets create \
  --billing-account=$(gcloud billing accounts list --format='value(name)' --limit=1) \
  --display-name=$BUDGET_NAME \
  --budget-amount=$AMOUNT \
  --threshold-rules=percent=0.5,percent=0.9 \
  --filter-projects=$PROJECT_ID

echo "Budget $BUDGET_NAME created with $AMOUNT USD"

Setting Up Budget Alerts with Pub/Sub

To make budgets actionable, route alerts to Pub/Sub instead of just email. This allows you to trigger Cloud Functions, Cloud Run, or even external systems via webhooks. Start by creating a Pub/Sub topic and subscription. Then, when creating or updating a budget, specify the topic as the notification channel. GCP will publish a message to that topic each time a threshold is crossed. The message contains the budget name, current spend, forecasted spend, and alert type. You can then write a Cloud Function that reads this message and takes action—like disabling a billing account, stopping expensive VMs, or sending a Slack message. This turns a passive alert into an active control.

import base64
import json
import os

def handle_budget_alert(event, context):
    """Triggered from a Pub/Sub message."""
    pubsub_message = base64.b64decode(event['data']).decode('utf-8')
    alert = json.loads(pubsub_message)
    
    budget_name = alert['budgetDisplayName']
    current_spend = alert['costAmount']
    budget_amount = alert['budgetAmount']
    threshold = alert['alertThresholdExceeded']
    
    print(f"Budget {budget_name} exceeded {threshold}%")
    print(f"Current spend: {current_spend} / {budget_amount}")
    
    # Example: disable billing if over 100%
    if threshold >= 1.0:
        billing_account = os.environ['BILLING_ACCOUNT']
        # Disable billing (requires billing account admin)
        # gcloud billing accounts disable --account=$BILLING_ACCOUNT
        print(f"ALERT: Billing account {billing_account} should be disabled.")

Automated Cost Enforcement with Cloud Functions

Once you have Pub/Sub delivering budget alerts, the next step is to write a Cloud Function that enforces cost limits. Common actions: disable the billing account (nuclear option), stop all compute instances, scale down Kubernetes clusters, or restrict IAM permissions. The function should be idempotent and include safety checks—like verifying the alert is from a known budget and that the threshold is high enough to warrant action. Also, consider a grace period: don't disable billing on the first 100% alert; wait for a second alert after 5 minutes to avoid flapping. Deploy the function with a service account that has minimal permissions (e.g., only compute.instances.stop for the project). This is your safety net.

import base64
import json
import os
from google.cloud import compute_v1

def enforce_cost_limit(event, context):
    pubsub_message = base64.b64decode(event['data']).decode('utf-8')
    alert = json.loads(pubsub_message)
    
    threshold = alert['alertThresholdExceeded']
    if threshold < 1.0:
        print("Threshold below 100%, no action.")
        return
    
    project = os.environ['PROJECT_ID']
    zone = os.environ['ZONE']
    instance_client = compute_v1.InstancesClient()
    
    # List all instances in the project
    request = compute_v1.AggregatedListInstancesRequest()
    request.project = project
    agg_list = instance_client.aggregated_list(request=request)
    
    for zone, instances_in_zone in agg_list:
        for instance in instances_in_zone.instances:
            if instance.status == 'RUNNING':
                stop_request = compute_v1.StopInstanceRequest()
                stop_request.project = project
                stop_request.zone = zone
                stop_request.instance = instance.name
                instance_client.stop(request=stop_request)
                print(f"Stopped instance {instance.name} in {zone}")
    print("All running instances stopped.")

Using Quotas and Limits as Hard Caps

While budgets are soft, quotas are hard. GCP quotas limit the amount of a resource you can use (e.g., max 100 CPUs per region). You can request quota increases, but by default they act as a cap. Use quotas to prevent runaway resource creation. For example, set a low quota for GPU instances if you don't need them, or limit the number of VMs per project. Quotas are set per project and per region. You can also use IAM conditions to restrict resource creation based on size (e.g., only allow n1-standard-1 instances). This is a proactive control that prevents cost spikes before they happen.

#!/bin/bash
# Check current quota for compute instances
PROJECT_ID="my-project"
REGION="us-central1"

gcloud compute regions describe $REGION \
  --project=$PROJECT_ID \
  --format='json(quotas)' | jq '.quotas[] | select(.metric=="CPUS")'

echo "---"
echo "To request quota increase:"
echo "gcloud compute quotas update --project=$PROJECT_ID --region=$REGION --metric=CPUS --value=200"

Cost Controls via IAM and Organization Policies

IAM and Organization Policies are your first line of defense against unauthorized resource creation. Use IAM roles that restrict who can create expensive resources (e.g., only allow project owners to create Compute Engine instances with GPUs). Organization Policies can enforce constraints like 'Disable the creation of preemptible VMs' or 'Require labels on all resources'. Labels are critical for cost allocation—they let you track spend by team, environment, or application. Without labels, you can't attribute costs, making it impossible to optimize. Enforce labeling with a policy that denies creation of resources without required labels.

# Organization Policy to require labels on all Compute Engine instances
name: projects/PROJECT_ID/policies/compute.requireOsLogin
spec:
  rules:
  - enforce: true
---
# Custom constraint to require labels
name: organizations/ORG_ID/policies/constraints/compute.requireLabels
spec:
  rules:
  - enforce: true
  - condition:
      expression: "resource.labels.size() > 0"

Real-Time Cost Monitoring with Billing Reports and Dashboards

Budgets and alerts are reactive. For proactive cost management, set up real-time dashboards using GCP's Billing Reports or export billing data to BigQuery. Billing Reports give you a high-level view, but BigQuery export allows custom queries and alerts. Export billing data to a BigQuery dataset, then use Looker Studio or custom queries to build dashboards. You can set up scheduled queries that check for anomalies (e.g., spend > 2x daily average) and send alerts via Pub/Sub. This gives you near-real-time visibility into cost trends, so you can catch spikes before they trigger budget alerts.

-- Query to detect daily spend anomalies
WITH daily_spend AS (
  SELECT
    DATE(usage_start_time) as usage_date,
    SUM(cost) as total_cost
  FROM `project.dataset.billing_export`
  WHERE DATE(usage_start_time) >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
  GROUP BY usage_date
),
stats AS (
  SELECT
    AVG(total_cost) as avg_cost,
    STDDEV(total_cost) as stddev_cost
  FROM daily_spend
  WHERE usage_date < CURRENT_DATE()
)
SELECT
  usage_date,
  total_cost,
  avg_cost,
  stddev_cost,
  (total_cost - avg_cost) / stddev_cost as z_score
FROM daily_spend, stats
WHERE usage_date = CURRENT_DATE()
  AND (total_cost - avg_cost) / stddev_cost > 3

Automating Cost Reports with Scheduled Queries

Manual cost reviews are unreliable. Automate them with scheduled queries in BigQuery that run daily and send results to a Slack channel or email. For example, a query that lists the top 10 most expensive resources by project, or one that compares today's spend to yesterday's. Use BigQuery's scheduled queries feature (or Cloud Scheduler + Cloud Functions) to run these queries and publish results. This keeps the team informed without manual effort. Include a summary of any anomalies or threshold breaches.

-- Top 10 most expensive resources today
SELECT
  project.id as project_id,
  service.description as service,
  sku.description as sku,
  SUM(cost) as total_cost
FROM `project.dataset.billing_export`
WHERE DATE(usage_start_time) = CURRENT_DATE()
GROUP BY project_id, service, sku
ORDER BY total_cost DESC
LIMIT 10

Handling Multi-Project Billing with Budget Scopes

In large organizations, costs are spread across many projects. GCP budgets can be scoped to a single project, a set of projects, or all projects under a billing account. Use budget scopes to allocate budgets per team or environment. For example, create a budget for 'production' projects and another for 'development'. This allows granular alerts and enforcement. When a team exceeds their budget, you can take action only on their projects, not the entire org. Combine with labels to further refine scopes.

#!/bin/bash
# Create a budget for multiple projects
BILLING_ACCOUNT=$(gcloud billing accounts list --format='value(name)' --limit=1)
PROJECTS="prod-web,prod-api,prod-db"

gcloud billing budgets create \
  --billing-account=$BILLING_ACCOUNT \
  --display-name="production-budget" \
  --budget-amount=5000.00 \
  --threshold-rules=percent=0.8,percent=1.0 \
  --filter-projects=$PROJECTS

echo "Production budget created for projects: $PROJECTS"

Testing Your Cost Controls: Chaos Engineering for Billing

You wouldn't deploy a new feature without testing it. The same applies to cost controls. Set up a test project, create a budget with a low amount, and deliberately trigger it (e.g., by spinning up expensive resources). Verify that alerts fire, Pub/Sub messages are delivered, and your Cloud Function executes the intended action. Test edge cases: what happens if the function fails? What if the Pub/Sub message is malformed? Have a rollback plan. This chaos engineering approach ensures your controls work when you need them.

#!/bin/bash
# Test budget alert by creating expensive resources
PROJECT_ID="test-project"
ZONE="us-central1-a"

gcloud compute instances create test-cost-instance \
  --project=$PROJECT_ID \
  --zone=$ZONE \
  --machine-type=n1-standard-64 \
  --image-family=debian-11 \
  --image-project=debian-cloud

echo "Created expensive instance. Check budget alerts."

# Wait for alert, then clean up
sleep 60
gcloud compute instances delete test-cost-instance \
  --project=$PROJECT_ID \
  --zone=$ZONE --quiet

echo "Test complete. Instance deleted."

Putting It All Together: A Production-Ready Cost Control Pipeline

A robust cost control system combines all the pieces: budgets with Pub/Sub alerts, Cloud Functions for enforcement, quotas for hard limits, IAM policies for prevention, and dashboards for visibility. Here's a typical pipeline: 1) Budget crosses 80% threshold → Pub/Sub message sent. 2) Cloud Function receives message, logs it, sends Slack alert. 3) Budget crosses 100% → second Pub/Sub message. 4) Cloud Function stops all non-critical instances (based on labels). 5) Daily BigQuery query checks for anomalies and sends report. 6) Weekly review of quota usage and budget adjustments. This pipeline has saved companies from six-figure surprise bills.

# Conceptual pipeline
resources:
  - type: budget
    name: monthly-budget
    amount: 10000
    thresholds: [0.5, 0.8, 1.0]
    notifications:
      - pubsub_topic: budget-alerts
  - type: cloud_function
    name: budget-enforcer
    trigger: pubsub_topic: budget-alerts
    action: stop_instances_with_label: 'cost-center=non-critical'
  - type: scheduled_query
    name: daily-cost-report
    query: "SELECT ... FROM billing_export WHERE date = CURRENT_DATE()"
    schedule: "0 6 * * *"
    destination: slack_webhook

Cost Anomaly Detection: Catching Spikes Before Budgets Fire

Budgets are threshold-based and reactive — they alert you when spend has already crossed a line. Cost Anomaly Detection fills the gap by using machine learning to detect unusual spending patterns in near-real-time. It analyzes historical spend data per project, service, and SKU, and flags deviations that exceed a configurable cost impact threshold. When an anomaly is detected, it can send Pub/Sub notifications just like budgets, including detailed root cause analysis showing which service, region, and SKU drove the spike. This is especially valuable for catching misconfigurations (e.g., a Dataflow job that didn't shut down) within hours instead of days. To set it up, navigate to Billing > Anomalies in the GCP console, set a cost impact threshold (e.g., $100), and connect a Pub/Sub topic. The anomaly data includes expected spend, actual spend, deviation percentage, and a nested root_causes structure that pinpoints the offending resource. In production, anomaly detection caught a runaway BigQuery query within 2 hours — before it crossed the monthly budget threshold.

import base64
import json

def handle_anomaly(event, context):
    pubsub_data = base64.b64decode(event['data']).decode('utf-8')
    anomaly = json.loads(pubsub_data)
    
    resource = anomaly['resourceDisplayName']
    deviation = anomaly['deviationPercentage']
    expected = anomaly['expectedSpendAmount']
    actual = anomaly['actualSpendAmount']
    
    print(f"Anomaly detected in {resource}: {deviation:.1f%} deviation")
    print(f"Expected: ${expected}, Actual: ${actual}")
    
    # Root cause analysis
    for cause in anomaly.get('rootCauses', []):
        print(f"  Cause: {cause['displayName']} ({cause['causeType']})")
        print(f"  Deviation: {cause['deviation']['deviationPercentage']:.1f}%")