Google Cloud — Cloud Load Balancing

Cloud Load Balancing distributes incoming traffic across multiple backends to ensure high availability and scalability. Unlike traditional load balancers that require pre-warming, GCP's global load balancer scales instantly and serves traffic from a single anycast IP. Understanding the types — HTTP(S), TCP/UDP, and Internal — is critical for architecting resilient systems.

Why Cloud Load Balancing Matters in Production

In production, traffic spikes are inevitable. A single server can't handle millions of concurrent users, and even if it could, a single point of failure is unacceptable. Cloud Load Balancing distributes incoming traffic across multiple backend instances, ensuring high availability, fault tolerance, and scalability. Google Cloud offers several load balancers: External HTTP(S), Internal TCP/UDP, and SSL Proxy. Each serves a specific use case. For most web applications, the External HTTP(S) Load Balancer is the go-to choice because it supports global anycast IP, autoscaling, and advanced features like URL rewriting and traffic splitting. Without a load balancer, you risk downtime during traffic surges and manual failover nightmares. This section sets the foundation: understand the problem before choosing the tool.

gcloud compute forwarding-rules create http-content-rule \
  --global \
  --target-http-proxy http-lb-proxy \
  --ports 80 \
  --ip-version IPV4

Architecture: Frontend, Backend, and Routing Rules

A Google Cloud Load Balancer consists of three components: forwarding rules (frontend), target proxies (routing), and backend services (instance groups). The forwarding rule binds an IP and port to a target proxy. The proxy interprets the protocol (HTTP, HTTPS, TCP) and applies URL maps to route requests to the correct backend service. Backend services define the instance group, health checks, and balancing mode (rate or utilization). This layered design allows you to decouple traffic routing from backend management. For example, you can route /api/ to one backend and / to another without changing the frontend IP. Understanding this architecture is critical for debugging routing issues and scaling efficiently.

defaultService: https://www.googleapis.com/compute/v1/projects/my-project/global/backendServices/web-backend
hostRules:
- hosts:
  - example.com
  pathMatcher: path-matcher-1
pathMatchers:
- name: path-matcher-1
  defaultService: https://www.googleapis.com/compute/v1/projects/my-project/global/backendServices/web-backend
  pathRules:
  - paths:
    - /api/*
    service: https://www.googleapis.com/compute/v1/projects/my-project/global/backendServices/api-backend

Health Checks: The Difference Between Graceful Degradation and Outage

Health checks are the load balancer's eyes into backend health. Without them, the load balancer blindly sends traffic to unhealthy instances, causing errors. Google Cloud supports HTTP, HTTPS, TCP, and SSL health checks. For HTTP(S) health checks, you define a request path (e.g., /healthz) and expected response codes. The load balancer probes each instance at a configurable interval (default 5s) and marks it unhealthy after consecutive failures. In production, set the health check path to a lightweight endpoint that validates critical dependencies (database, cache). Avoid heavy endpoints that could fail under load. Also, configure the 'unhealthy threshold' to 2-3 to avoid flapping. A common mistake is using the same path as the main application, which can cause cascading failures.

from flask import Flask, jsonify
import redis
import psycopg2

app = Flask(__name__)

@app.route('/healthz')
def healthz():
    health = True
    try:
        r = redis.Redis(host='redis-service', port=6379, socket_connect_timeout=2)
        r.ping()
    except:
        health = False
    try:
        conn = psycopg2.connect(host='db-service', dbname='app', user='user', password='pass', connect_timeout=2)
        conn.close()
    except:
        health = False
    if health:
        return jsonify({'status': 'healthy'}), 200
    else:
        return jsonify({'status': 'unhealthy'}), 503

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Session Affinity: When You Need Sticky Sessions (and When You Don't)

Session affinity (sticky sessions) ensures that a client's requests are always sent to the same backend instance. This is useful for stateful applications like shopping carts or WebSocket connections. Google Cloud supports three modes: NONE (default), CLIENT_IP, and GENERATED_COOKIE. CLIENT_IP uses the client's IP, but fails if clients share IPs (e.g., behind NAT). GENERATED_COOKIE inserts a cookie (GCLB) to track the backend. However, sticky sessions reduce load balancing effectiveness and can cause uneven load distribution. In production, prefer stateless backends with external session stores (Redis, Memcached). If you must use affinity, set a reasonable cookie TTL (e.g., 1 hour) and monitor backend load imbalance.

gcloud compute backend-services update web-backend \
  --global \
  --session-affinity GENERATED_COOKIE \
  --affinity-cookie-ttl 3600

Traffic Splitting: Canary Deployments and A/B Testing

Traffic splitting lets you route a percentage of requests to different backend services. This is essential for canary deployments (gradually rolling out a new version) and A/B testing. Google Cloud's External HTTP(S) Load Balancer supports this via the 'weight' field in backend services. You can assign weights to multiple backends in the same URL map path rule. For example, 90% traffic to v1, 10% to v2. Monitor error rates and latency before shifting more traffic. In production, start with 1% traffic to the canary, then increase by 5-10% every 10 minutes if metrics are healthy. Use request-level cookies or headers for consistent routing if needed.

gcloud compute backend-services add-backend web-backend \
  --global \
  --instance-group canary-instance-group \
  --balancing-mode UTILIZATION \
  --max-utilization 0.8 \
  --capacity-scaler 0.1

gcloud compute backend-services update web-backend \
  --global \
  --backend-group web-backend \
  --capacity-scaler 0.9

SSL/TLS Termination: Offload Crypto to the Load Balancer

SSL termination at the load balancer reduces CPU load on backend instances by handling encryption/decryption at the edge. Google Cloud supports SSL certificates managed by Certificate Manager or self-managed. You can also enable TLS 1.3 and configure cipher suites. In production, use managed certificates for automatic renewal and rotation. For internal traffic, consider using Internal HTTPS Load Balancer with private certificates. Important: after termination, traffic between the load balancer and backend is unencrypted by default. To encrypt it, enable 'backend service protocol' HTTPS and configure certificates on the backend. This is called end-to-end encryption and is recommended for compliance.

gcloud compute ssl-certificates create my-cert \
  --domains example.com \
  --global

gcloud compute target-https-proxies create https-lb-proxy \
  --ssl-certificates my-cert \
  --url-map http-lb-url-map

Logging and Monitoring: See Every Request

Without visibility, you're flying blind. Google Cloud Load Balancer integrates with Cloud Logging and Cloud Monitoring. Enable access logs to capture request metadata: client IP, latency, status code, backend name. Logs are sampled by default (1 in 10) to reduce cost; in production, set sampling to 1.0 during incidents. Use Cloud Monitoring to create dashboards for request count, latency (p50, p95, p99), error rate, and backend utilization. Set up alerts for 5xx error rate > 1% and latency p99 > 500ms. Also monitor health check status to detect backend failures early. Logs can be exported to BigQuery for advanced analytics.

gcloud compute backend-services update web-backend \
  --global \
  --enable-logging \
  --logging-sample-rate 1.0

Scaling: Autoscaling Backends with Load Balancer Awareness

The load balancer and autoscaler work together to handle traffic spikes. Autoscalers use metrics like CPU utilization, requests per second, or custom metrics to add/remove instances. For optimal performance, set the autoscaler's target utilization to 60-70% to leave headroom for spikes. The load balancer's balancing mode (RATE or UTILIZATION) affects how traffic is distributed. Use RATE for request-based scaling (e.g., 1000 requests per second per instance) and UTILIZATION for CPU-based. In production, combine both: set a max RATE per instance to prevent overload, and use CPU utilization for scaling. Also, configure cool-down periods to avoid thrashing.

gcloud compute instance-groups managed set-autoscaling web-instance-group \
  --region us-central1 \
  --max-num-replicas 20 \
  --min-num-replicas 2 \
  --target-cpu-utilization 0.6 \
  --cool-down-period 120

Internal Load Balancing: Secure Microservices Communication

For microservices running in VPC, use Internal Load Balancers (ILB) to distribute traffic between services without exposing them to the internet. ILB supports TCP/UDP and HTTP(S) protocols. It uses RFC 1918 IP addresses and is only accessible within the VPC. In production, use ILB for service-to-service communication (e.g., frontend to backend). Combine with Private Service Connect for hybrid connectivity. ILB also supports health checks and session affinity. A common pattern is to front each microservice with an ILB, allowing independent scaling and blue/green deployments.

gcloud compute forwarding-rules create internal-lb-rule \
  --region us-central1 \
  --load-balancing-scheme INTERNAL \
  --network default \
  --subnet default \
  --address 10.128.0.10 \
  --ports 80 \
  --target-http-proxy internal-http-proxy

Cost Optimization: Pay Only for What You Use

Cloud Load Balancing pricing is based on forwarding rules and data processed. Each forwarding rule costs a fixed hourly rate (e.g., $0.025/hr for external HTTP LB). Data processing is charged per GB (e.g., $0.008/GB). To optimize, consolidate multiple services under one load balancer using URL maps instead of separate LBs. Use internal LBs for backend traffic to avoid egress charges. Also, consider using preemptible VMs for autoscaling groups to reduce compute costs. Monitor your LB usage with billing reports and set budget alerts. A common waste is leaving forwarding rules for deleted backends; clean up regularly.

gcloud compute forwarding-rules list --global
# Check for unused rules
gcloud compute forwarding-rules delete unused-rule --global

Disaster Recovery: Multi-Region Failover with Global LB

Global load balancers can route traffic to backends in multiple regions. If one region fails, the LB automatically redirects traffic to healthy regions. This is achieved by adding backend services from different regions to the same LB. Configure health checks per region and set failover priority. For active-passive, set one region as primary with high priority, and secondary with lower priority. For active-active, distribute traffic across regions using weights. In production, test failover regularly by simulating a region outage. Also, ensure your database can handle multi-region writes or use a global database like Spanner.

gcloud compute backend-services create web-backend \
  --global \
  --health-checks http-health-check \
  --load-balancing-scheme EXTERNAL

gcloud compute backend-services add-backend web-backend \
  --global \
  --instance-group us-central1-instances \
  --balancing-mode UTILIZATION \
  --max-utilization 0.8 \
  --failover-ratio 0.0

gcloud compute backend-services add-backend web-backend \
  --global \
  --instance-group europe-west1-instances \
  --balancing-mode UTILIZATION \
  --max-utilization 0.8 \
  --failover-ratio 1.0

Common Pitfalls and How to Avoid Them

Even experienced teams make mistakes. Here are the top pitfalls: (1) Health check path too heavy — use a lightweight endpoint. (2) Not setting connection draining — when an instance is removed, existing connections are dropped. Enable connection draining (timeout up to 1 hour) to allow in-flight requests to complete. (3) Ignoring backend capacity — if all backends are at max capacity, new requests fail. Set max RATE per instance and autoscale. (4) Misconfigured firewall rules — load balancer health checks come from specific IP ranges (35.191.0.0/16, 130.211.0.0/22). Ensure your firewall allows these. (5) Not testing failover — simulate a region outage in staging. (6) Overlooking SSL certificate expiry — use managed certificates to auto-renew.

gcloud compute backend-services update web-backend \
  --global \
  --connection-draining-timeout 300