Kubernetes Ingress Controllers — NGINX, Traefik, AWS ALB

Welcome to Kubernetes Ingress Controllers — NGINX, Traefik, AWS ALB. We'll break this down from first principles.

The Ingress Resource vs. Ingress Controller

An Ingress resource is a Kubernetes API object that defines routing rules, but it's inert without a controller. The controller is the actual reverse proxy that watches for Ingress resources and configures itself accordingly. This separation of concerns allows you to define routing declaratively while the controller handles the implementation details. In production, you must choose a controller that fits your infrastructure—NGINX for general-purpose, Traefik for dynamic environments, or AWS ALB for native cloud integration. Without a controller, Ingress resources are just YAML files doing nothing.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

NGINX Ingress Controller: The Battle-Tested Workhorse

The NGINX Ingress Controller is the most popular choice, backed by the Kubernetes community. It uses NGINX as a reverse proxy and load balancer, with a Lua-based configuration model. It supports advanced features like canary deployments, custom snippets, and rate limiting. In production, it's known for stability and performance, handling thousands of requests per second with low latency. However, its configuration reloads can be expensive—every Ingress change triggers a full NGINX reload, which can drop connections if not tuned. Use it when you need maximum control and are willing to manage its complexity.

controller:
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
  config:
    use-proxy-protocol: "true"
    proxy-body-size: "10m"
  replicaCount: 3
  resources:
    requests:
      cpu: 500m
      memory: 512Mi

Traefik: Dynamic and Cloud-Native

Traefik is a modern reverse proxy designed for dynamic environments. It automatically discovers services and updates routing without reloads, using a hot-reload mechanism. It supports multiple providers (Kubernetes, Docker, Consul) and has built-in observability with metrics and tracing. In production, Traefik shines in microservices architectures where services come and go frequently. However, its performance is slightly lower than NGINX under extreme load, and its configuration via annotations can become messy. Use it when you prioritize automation and ease of use over raw performance.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: example-route
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`example.com`) && PathPrefix(`/api`)
    kind: Rule
    services:
    - name: api-service
      port: 80
  tls:
    certResolver: letsencrypt

AWS ALB Ingress Controller: Native Cloud Integration

The AWS ALB Ingress Controller provisions an Application Load Balancer per Ingress resource, integrating deeply with AWS services like WAF, Cognito, and ACM. It supports path-based routing, SSL termination at the load balancer, and sticky sessions via cookies. In production, it offloads traffic management to AWS, reducing cluster resource usage. However, it introduces latency due to the extra network hop and has limited features compared to NGINX (e.g., no canary deployments out of the box). Use it when you're fully on AWS and want to leverage managed services.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: alb-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Performance Comparison: Latency and Throughput

In production, performance varies significantly. NGINX typically has the lowest latency (sub-millisecond) and highest throughput (100k+ req/s per replica). Traefik adds ~1-2ms latency due to its dynamic routing, but handles 50k+ req/s. AWS ALB adds 5-10ms latency due to the network hop, but scales automatically to millions of requests. Choose based on your latency budget: NGINX for latency-sensitive apps, Traefik for moderate loads, ALB for ease of scaling. Always benchmark with realistic traffic patterns—don't rely on synthetic tests.

kubectl run -it --rm loadtest --image=williamyeh/hey --restart=Never -- hey -n 10000 -c 100 http://nginx-ingress.example.com/api

SSL/TLS Termination: Where and How

SSL termination can happen at the ingress controller or at the backend. Terminating at the controller reduces backend CPU load but means traffic inside the cluster is unencrypted. For compliance, you may need end-to-end encryption. NGINX supports cert-manager for automatic certificate provisioning. Traefik has built-in ACME support. ALB integrates with ACM for managed certificates. In production, always use cert-manager for NGINX/Traefik to automate renewal. Never hardcode certificates in Ingress annotations—they expire.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-tls
spec:
  secretName: example-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
  - example.com

Advanced Routing: Canary Deployments and A/B Testing

Canary deployments allow you to route a percentage of traffic to a new version. NGINX supports this via custom annotations (nginx.ingress.kubernetes.io/canary). Traefik has built-in weighted round-robin via IngressRoute. ALB does not support canary natively—you'd need a service mesh or external tool. In production, canary deployments reduce risk but require careful monitoring. Always start with 1% traffic and increase gradually. Use metrics (error rate, latency) to decide whether to proceed or rollback.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-v2
            port:
              number: 80

Observability: Metrics, Logs, and Tracing

All three controllers export Prometheus metrics. NGINX exposes NGINX-specific metrics like active connections and request latency. Traefik has built-in metrics and supports OpenTelemetry. ALB publishes metrics to CloudWatch. In production, you need a unified dashboard. Use Grafana with prebuilt dashboards for each controller. For logs, NGINX and Traefik output structured JSON; ALB logs to S3. Enable access logs with request IDs for debugging. Tracing is critical for microservices—Traefik supports Zipkin and Jaeger natively.

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress-metrics
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"
spec:
  selector:
    app.kubernetes.io/name: ingress-nginx
  ports:
  - port: 10254
    targetPort: 10254
    name: metrics

Security: WAF, Rate Limiting, and IP Whitelisting

Security features vary. NGINX supports ModSecurity via a plugin, rate limiting via annotations, and IP whitelisting. Traefik has middleware for rate limiting, IP whitelisting, and basic auth. ALB integrates with AWS WAF for web application firewall. In production, always enable rate limiting to protect against DDoS. Use IP whitelisting for admin endpoints. For WAF, start with AWS managed rules and customize. Never rely solely on the ingress controller for security—defense in depth is key.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rate-limited
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "100"
    nginx.ingress.kubernetes.io/limit-whitelist: "10.0.0.0/8"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

High Availability and Scaling

Ingress controllers must be highly available. Deploy multiple replicas with pod anti-affinity to spread across nodes. Use a LoadBalancer service (NLB for AWS) to distribute traffic. For NGINX and Traefik, enable leader election to avoid duplicate processing. ALB is managed, so scaling is automatic. In production, set resource requests/limits to prevent OOM kills. Use HPA based on CPU or custom metrics. Test failover by killing a pod—expect <1s downtime if configured correctly.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-ingress-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-ingress-controller
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Migration Strategies Between Controllers

Migrating between ingress controllers is risky. Plan carefully: 1) Install the new controller alongside the old one with a different ingress class. 2) Update Ingress resources to point to the new class gradually. 3) Use DNS weighting to shift traffic. 4) Monitor metrics and rollback if issues arise. In production, never do a big bang migration. We've seen teams spend weeks debugging routing issues after a sudden switch. Test with a subset of hosts first.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
spec:
  ingressClassName: traefik
  rules:
  - host: new.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app
            port:
              number: 80

Cost Analysis: Self-Managed vs. Managed

Self-managed controllers (NGINX, Traefik) cost only the compute resources (EC2 instances). Managed controllers (ALB) incur per-hour and per-GB costs. For a cluster with 10 Ingress resources, ALB costs ~$200/month vs. ~$50/month for NGINX on 3 small instances. However, ALB reduces operational overhead. In production, factor in engineering time: managing NGINX requires expertise in reloads, configs, and debugging. ALB is simpler but can surprise you with costs if traffic spikes. Always use cost allocation tags.

# NGINX: 3 x t3.medium instances = ~$90/month
# ALB: $22.40 + $0.008/GB data = ~$100/month for 1TB
# Traefik: similar to NGINX
# Choose based on traffic and operational budget.