Senior 5 min · March 06, 2026

Firewall Rule Order — Misordered DENY Led to 45-Min Outage

A misplaced ALLOW rule overrode a DENY for 45 minutes, causing a production outage—discover the firewall rule ordering mistake and how to prevent it.

N
Naren · Founder
Plain-English first. Then code. Then the interview question.
About
 ● Production Incident 🔎 Debug Guide
Quick Answer
  • Firewalls filter traffic by rules; proxies relay traffic on behalf of clients or servers
  • Three firewall types: packet-filter, stateful, application-layer (NGFW)
  • Two proxy types: forward (hides clients) and reverse (hides servers)
  • Performance rule: packet-filter firewalls inspect in microseconds; NGFWs add ~50µs per packet
  • Production insight: misordered firewall rules silently bypass security — always place DENY above ALLOW
  • Biggest mistake: treating firewall and proxy as alternatives — they're stacked layers, not competitors
Plain-English First

Imagine your school has a security guard at the front gate who checks everyone's ID before letting them in — that's a firewall. Now imagine the school also has a secretary who makes phone calls on your behalf so the other person never gets your direct number — that's a proxy. The firewall decides WHO gets through. The proxy decides HOW the conversation happens, and who the other side thinks they're talking to. Together they're the two-person security team of every serious network.

Every time you open a browser at work, stream a video on a corporate Wi-Fi, or deploy an API to the cloud, there are invisible gatekeepers deciding whether your traffic is allowed, where it should go, and what the destination is allowed to know about you. Firewalls and proxies are those gatekeepers — and understanding them is the difference between a developer who ships code and one who ships secure, production-ready systems. Misunderstanding them causes real outages, security holes, and hours of debugging 'why can't my app connect?'

What a Firewall Actually Does — Beyond Just Blocking Ports

A firewall is a network security system that inspects incoming and outgoing traffic and decides whether to allow or deny it based on a predefined set of rules. Think of rules like a guest list — if your IP, port, or protocol isn't on the list, you don't get in.

There are three main generations of firewalls you'll encounter in the real world. Packet-filtering firewalls (the oldest) inspect each packet in isolation — they check source IP, destination IP, protocol, and port number. They're fast but naive: they can't tell if a packet is part of a legitimate TCP session or an attack masquerading as one.

Stateful firewalls level up by tracking connection state. They know whether a packet is the start of a new connection, part of an existing one, or completely unexpected. If you never sent a SYN to a server, that server's SYN-ACK coming back looks suspicious — a stateful firewall will drop it.

Application-layer firewalls (often called Next-Generation Firewalls or NGFWs) go even deeper — they can inspect HTTP headers, DNS queries, and even TLS metadata to make decisions based on content, not just packets. This is how corporate firewalls block TikTok even when it runs on standard HTTPS port 443.

SimplePacketFilter.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
# Simulating a basic stateless packet-filtering firewall in Python.
# In production, this logic lives in kernel space (iptables, nftables, Windows Firewall).
# Here we model it in userspace so you can SEE the decision process.

from dataclasses import dataclass
from typing import List

@dataclass
class NetworkPacket:
    source_ip: str
    destination_ip: str
    destination_port: int
    protocol: str  # 'TCP', 'UDP', 'ICMP'

@dataclass
class FirewallRule:
    """
    A single firewall rule. Rules are evaluated top-to-bottom.
    The first matching rule wins — just like real iptables chains.
    """
    source_ip: str        # '*' means any
    destination_port: int # -1 means any
    protocol: str         # '*' means any
    action: str           # 'ALLOW' or 'DENY'

class PacketFilterFirewall:
    def __init__(self, default_policy: str = 'DENY'):
        """
        Default policy is 'DENY' — this is the secure default.
        'Allow all, deny specific' is the WRONG approach (allowlist vs denylist).
        """
        self.rules: List[FirewallRule] = []
        self.default_policy = default_policy

    def add_rule(self, rule: FirewallRule):
        # Rules are ordered — first match wins, so order matters enormously
        self.rules.append(rule)

    def inspect(self, packet: NetworkPacket) -> str:
        for rule in self.rules:
            # Check each condition — '*' is a wildcard that matches anything
            ip_match = (rule.source_ip == '*' or rule.source_ip == packet.source_ip)
            port_match = (rule.destination_port == -1 or rule.destination_port == packet.destination_port)
            proto_match = (rule.protocol == '*' or rule.protocol == packet.protocol)

            if ip_match and port_match and proto_match:
                # Found the first matching rule — enforce it immediately
                return rule.action

        # No rule matched — fall back to the default policy
        return self.default_policy

# --- Setting up the firewall ---
firewall = PacketFilterFirewall(default_policy='DENY')

# Rule 1: Allow all HTTP traffic from anywhere
firewall.add_rule(FirewallRule(source_ip='*', destination_port=80, protocol='TCP', action='ALLOW'))

# Rule 2: Allow HTTPS from anywhere
firewall.add_rule(FirewallRule(source_ip='*', destination_port=443, protocol='TCP', action='ALLOW'))

# Rule 3: Allow SSH only from the trusted admin IP
firewall.add_rule(FirewallRule(source_ip='10.0.0.5', destination_port=22, protocol='TCP', action='ALLOW'))

# Rule 4: Explicitly block a known malicious IP on any port
firewall.add_rule(FirewallRule(source_ip='185.220.101.9', destination_port=-1, protocol='*', action='DENY'))

# --- Testing packets against the firewall ---
test_packets = [
    NetworkPacket('203.0.113.42', '192.168.1.1', 80,  'TCP'),  # Regular web request
    NetworkPacket('203.0.113.42', '192.168.1.1', 443, 'TCP'),  # HTTPS request
    NetworkPacket('198.51.100.7', '192.168.1.1', 22,  'TCP'),  # SSH from unknown IP
    NetworkPacket('10.0.0.5',     '192.168.1.1', 22,  'TCP'),  # SSH from trusted admin
    NetworkPacket('185.220.101.9','192.168.1.1', 443, 'TCP'),  # Known bad actor on HTTPS
    NetworkPacket('203.0.113.42', '192.168.1.1', 8080,'TCP'),  # Non-standard port — no rule
]

print(f"{'Source IP':<20} {'Port':<6} {'Result'}")
print('-' * 40)
for pkt in test_packets:
    result = firewall.inspect(pkt)
    print(f"{pkt.source_ip:<20} {pkt.destination_port:<6} {result}")
Output
Source IP Port Result
----------------------------------------
203.0.113.42 80 ALLOW
203.0.113.42 443 ALLOW
198.51.100.7 22 DENY
10.0.0.5 22 ALLOW
185.220.101.9 443 DENY
203.0.113.42 8080 DENY
Watch Out: Rule Order Is Everything
If you put the 'ALLOW * port 443' rule BEFORE your 'DENY 185.220.101.9' rule, the malicious IP slips through — the first matching rule wins and evaluation stops. Always put your most specific DENY rules above broad ALLOW rules, just like the code above does.
Production Insight
A single misordered rule can nullify your entire security policy.
Production firewalls have hundreds of rules — auditing order is a dedicated task.
Always use a tool like iptables-save and diff against a known-good baseline on every deploy.
Key Takeaway
Firewalls filter by IP:port:protocol, not content.
State tracking adds session awareness but can't inspect HTTP payloads.
To stop application attacks, you need a WAF on top of your firewall.

Forward Proxies vs Reverse Proxies — Two Tools With Opposite Jobs

The word 'proxy' trips people up because it means two completely different things depending on which side of the connection it sits on. Getting this wrong in an interview is an instant red flag.

A forward proxy sits between your users and the internet. Your client makes a request to the proxy, and the proxy makes the real request on the client's behalf. The destination server sees the proxy's IP, not yours. This is how corporate networks enforce browsing policies (blocking social media), how VPNs mask your origin, and how Tor anonymizes traffic. The key insight: the CLIENT knows about the forward proxy.

A reverse proxy sits in front of your servers, facing the internet. Clients think they're talking directly to your backend, but they're actually talking to the proxy. The proxy decides which backend server handles the request. The key insight: the CLIENT does not know about the reverse proxy — they think they're hitting your server directly. Nginx, Cloudflare, and AWS ALB are all reverse proxies in disguise.

The mental model: a forward proxy protects and controls the CLIENT. A reverse proxy protects and controls the SERVER. Both hide one side from the other — they just hide different sides.

ProxyBehaviorDemo.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
# This demo simulates the REQUEST FLOW through both a forward and reverse proxy.
# We use Python's http.server and threading to run real local HTTP servers.
# Run this script and watch the printed logs to see who talks to whom.

import threading
import time
from http.server import HTTPServer, BaseHTTPRequestHandler
from urllib.request import urlopen, Request
from urllib.error import URLError

# ─────────────────────────────────────────────────
# 1. THE ORIGIN SERVER — represents your backend API
# ─────────────────────────────────────────────────
class OriginServerHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        # Log who is connecting — in a reverse proxy setup, this will be
        # the proxy's IP, NOT the original client's IP
        print(f"[ORIGIN SERVER] Received request from: {self.client_address[0]}")
        print(f"[ORIGIN SERVER] Path requested: {self.path}")

        response_body = b"Hello from the Origin Server! Path: " + self.path.encode()
        self.send_response(200)
        self.send_header('Content-Type', 'text/plain')
        self.send_header('Content-Length', len(response_body))
        self.end_headers()
        self.wfile.write(response_body)

    # Suppress default request logging to keep output clean
    def log_message(self, format, *args): pass

# ─────────────────────────────────────────────────
# 2. THE REVERSE PROXY — sits in front of origin
# ─────────────────────────────────────────────────
class ReverseProxyHandler(BaseHTTPRequestHandler):
    ORIGIN_SERVER_URL = 'http://127.0.0.1:9001'
    BLOCKED_PATHS = ['/admin', '/internal']

    def do_GET(self):
        print(f"\n[REVERSE PROXY] Client {self.client_address[0]} wants: {self.path}")

        # Block sensitive internal paths — client never knows these exist
        if self.path in self.BLOCKED_PATHS:
            print(f"[REVERSE PROXY] BLOCKING sensitive path: {self.path}")
            self.send_response(403)
            self.end_headers()
            self.wfile.write(b"403 Forbidden")
            return

        # Forward the request to the origin server on the client's behalf
        # The origin server will see 127.0.0.1, not the real client IP
        target_url = self.ORIGIN_SERVER_URL + self.path
        print(f"[REVERSE PROXY] Forwarding to origin: {target_url}")

        try:
            with urlopen(Request(target_url), timeout=5) as origin_response:
                body = origin_response.read()

            # Pass the origin's response back to the original client
            self.send_response(200)
            self.send_header('Content-Type', 'text/plain')
            self.send_header('X-Served-By', 'TheCodeForge-ReverseProxy')  # Custom header
            self.end_headers()
            self.wfile.write(body)
            print(f"[REVERSE PROXY] Successfully relayed {len(body)} bytes back to client")

        except URLError as network_error:
            print(f"[REVERSE PROXY] Origin server unreachable: {network_error}")
            self.send_response(502)  # 502 Bad Gateway — classic reverse proxy error
            self.end_headers()
            self.wfile.write(b"502 Bad Gateway")

    def log_message(self, format, *args): pass

# ─────────────────────────────────────────────────
# 3. SPIN UP BOTH SERVERS IN BACKGROUND THREADS
# ─────────────────────────────────────────────────
def start_server(handler_class, port):
    server = HTTPServer(('127.0.0.1', port), handler_class)
    server.serve_forever()

origin_thread = threading.Thread(target=start_server, args=(OriginServerHandler, 9001), daemon=True)
proxy_thread  = threading.Thread(target=start_server, args=(ReverseProxyHandler,  9000), daemon=True)

origin_thread.start()
proxy_thread.start()
time.sleep(0.5)  # Give servers a moment to bind their ports

# ─────────────────────────────────────────────────
# 4. SIMULATE CLIENT REQUESTS (client talks to PROXY only)
# ─────────────────────────────────────────────────
print("=" * 55)
print("CLIENT: Requesting /api/users through the reverse proxy")
print("=" * 55)
with urlopen('http://127.0.0.1:9000/api/users', timeout=5) as resp:
    print(f"CLIENT received: {resp.read().decode()}")

time.sleep(0.2)

print("\n" + "=" * 55)
print("CLIENT: Attempting to access /admin (should be blocked)")
print("=" * 55)
try:
    urlopen('http://127.0.0.1:9000/admin', timeout=5)
except Exception as e:
    print(f"CLIENT received error (expected): HTTP 403")

time.sleep(0.2)
print("\nDemo complete.")
Output
=======================================================
CLIENT: Requesting /api/users through the reverse proxy
=======================================================
[REVERSE PROXY] Client 127.0.0.1 wants: /api/users
[REVERSE PROXY] Forwarding to origin: http://127.0.0.1:9001/api/users
[ORIGIN SERVER] Received request from: 127.0.0.1
[ORIGIN SERVER] Path requested: /api/users
[REVERSE PROXY] Successfully relayed 43 bytes back to client
CLIENT received: Hello from the Origin Server! Path: /api/users
=======================================================
CLIENT: Attempting to access /admin (should be blocked)
=======================================================
[REVERSE PROXY] Client 127.0.0.1 wants: /admin
[REVERSE PROXY] BLOCKING sensitive path: /admin
CLIENT received error (expected): HTTP 403
Demo complete.
Pro Tip: The X-Forwarded-For Header
When a reverse proxy forwards a request, the origin server loses the real client IP. The industry standard fix is the X-Forwarded-For header — the proxy injects it with the original client's IP so your backend logs remain accurate. Always check this header in your app if you're behind Nginx, Cloudflare, or a load balancer, or your rate-limiting and geo-blocking will target the proxy's IP instead of the real user.
Production Insight
X-Forwarded-For is trivially spoofable by attackers.
Only trust it when your proxy sets it and you validate the source.
Use set_real_ip_from in Nginx to restrict which proxies can set it.
Key Takeaway
Forward proxy shields the client; reverse proxy shields the server.
Both are proxies — the name tells you who they protect.
In interviews, always clarify which direction the proxy faces.

How Firewalls and Proxies Work Together in Real Architectures

In production, firewalls and proxies don't compete — they layer. Each handles a different concern, and combining them is what gives you defense in depth. Here's the pattern you'll see in virtually every serious web company.

At the network perimeter, a stateful firewall (hardware or cloud security group like AWS's Security Groups) allows only ports 80 and 443 inbound from the internet. Everything else is dropped at the packet level — attackers can't even probe your database port because the firewall silently discards the packets.

Behind that, a reverse proxy (Nginx, HAProxy, or a cloud load balancer) terminates TLS, inspects HTTP, and routes traffic to the right backend service. It also rate-limits, caches responses, and handles DDoS mitigation. Your actual backend servers aren't even directly reachable from the internet — they live in a private subnet.

Optionally, a Web Application Firewall (WAF) sits inline with the reverse proxy and inspects HTTP payloads specifically for application-layer attacks: SQL injection strings in query parameters, XSS payloads in headers, path traversal attempts. A WAF is essentially an application-layer firewall bolted onto a reverse proxy.

For outbound corporate traffic, a forward proxy (Squid, Zscaler) ensures employees' internet requests are logged, filtered, and controlled — and that your internal server IPs are never exposed to the outside world.

reverse_proxy_with_access_control.confNGINX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# Production-grade Nginx config that acts as both a reverse proxy AND
# an application-layer access control layer.
# This sits in front of a Node.js app running on port 3000 internally.

# --- Rate limiting zone: track clients by IP, 10MB memory, max 30 req/min ---
limit_req_zone $binary_remote_addr zone=api_rate_limit:10m rate=30r/m;

server {
    listen 443 ssl http2;
    server_name api.yourcorp.com;

    # TLS termination happens HERE — backend never handles raw TLS
    ssl_certificate     /etc/ssl/certs/yourcorp.crt;
    ssl_certificate_key /etc/ssl/private/yourcorp.key;
    ssl_protocols       TLSv1.2 TLSv1.3;  # Reject weak TLS 1.0 and 1.1

    # ── Block internal/admin paths from public internet ──────────────────────
    location ~ ^/(internal|metrics|health/debug) {
        # Only allow requests from the internal VPC CIDR range
        allow 10.0.0.0/8;
        deny  all;  # Everyone else gets 403 — firewall at the HTTP layer
    }

    # ── Public API — apply rate limiting ─────────────────────────────────────
    location /api/ {
        # Apply rate limit — burst of 10 requests allowed before throttling
        limit_req zone=api_rate_limit burst=10 nodelay;

        # THE CORE PROXY ACTION: forward to backend, hide backend's identity
        proxy_pass http://127.0.0.1:3000;

        # Tell the backend the REAL client IP (not Nginx's loopback address)
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Strip headers the backend might leak to clients
        proxy_hide_header X-Powered-By;  # Don't expose 'Express' or 'Node'
        proxy_hide_header Server;        # Don't expose backend server version

        # Timeout settings — don't let slow backends hold connections forever
        proxy_connect_timeout 5s;
        proxy_read_timeout    30s;
    }

    # ── Static assets — serve directly, never hit the backend ────────────────
    location /static/ {
        root /var/www/yourcorp;
        expires 30d;       # Cache for 30 days — no need to hit backend
        add_header Cache-Control "public, immutable";
    }

    # ── Redirect all HTTP to HTTPS at the server level ───────────────────────
    # (The firewall allows port 80 inbound only for this redirect)
}

server {
    listen 80;
    server_name api.yourcorp.com;
    return 301 https://$host$request_uri;  # Permanent redirect to HTTPS
}
Output
# No runtime output — this is a config file.
# When a client hits https://api.yourcorp.com/api/users:
# 1. Nginx terminates TLS (backend sees plain HTTP on port 3000)
# 2. X-Forwarded-For is set to the real client IP
# 3. Rate limiter checks: if >30 req/min, returns HTTP 429
# 4. Request proxied to 127.0.0.1:3000
# 5. X-Powered-By and Server headers stripped from response
#
# When a client hits https://api.yourcorp.com/metrics:
# → 403 Forbidden (unless from 10.0.0.0/8 range)
#
# When a client hits http://api.yourcorp.com/anything:
# → 301 Redirect to https://api.yourcorp.com/anything
Interview Gold: Defense in Depth
When an interviewer asks 'how would you secure a web API?', don't just say 'add authentication'. Walk through the layers: network firewall blocks unwanted ports → reverse proxy terminates TLS and rate-limits → WAF inspects HTTP payloads → app-level auth validates tokens. Each layer stops a different class of attack. This layered thinking is what separates a junior from a mid-senior engineer.
Production Insight
If your backend is reachable directly on port 3000 from the internet, your reverse proxy is pointless.
Always put backend servers in a private subnet with a security group that only allows traffic from the proxy.
Never trust the proxy header alone — use network-level isolation as your first layer.
Key Takeaway
Firewall + reverse proxy + WAF = defense in depth.
Each layer solves a different problem; none is a silver bullet.
The most common production gap is exposing backend ports despite having a proxy.

Firewall in the Cloud: Security Groups, NACLs and Their Gotchas

Cloud firewalls are not the same as on-prem ones. AWS, Azure, and GCP provide two separate firewall layers: Security Groups (instance-level stateful firewalls) and Network ACLs (subnet-level stateless firewalls). Confusing the two causes outages.

A Security Group acts as a virtual firewall for an EC2 instance or RDS database. It's stateful — if you allow inbound traffic on port 443, the outbound reply is automatically allowed regardless of outbound rules. It's also implicit deny by default: you don't need an explicit deny rule.

A Network ACL is a stateless firewall applied at the subnet level. Since it's stateless, you must explicitly allow both inbound and outbound traffic. If you allow inbound on port 443 but forget the outbound ephemeral port range (1024-65535), the response packets are silently dropped — clients see a timeout rather than a connection refused.

Common cloud mistake: engineers add a Security Group rule allowing SSH from 0.0.0.0/0 'temporarily' and forget to revert. That instance becomes reachable by attackers scanning for open port 22. Always restrict management access to your corporate IP or use a bastion host.

aws_security_groups_and_nacl.tfHCL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
# Terraform example showing Security Group vs NACL rules for a web tier.
# The Security Group allows HTTP/S inbound; the NACL additionally controls ephemeral ports.

resource "aws_security_group" "web_sg" {
  name_prefix = "web-tier-sg-"
  description = "Allow HTTP and HTTPS from internet"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
    description = "HTTPS from anywhere"
  }

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
    description = "HTTP redirect from anywhere"
  }

  # SSH only from corporate CIDR (bastion or VPN)
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["203.0.113.0/24"]  # Replace with your corp IP range
    description = "SSH from corporate network"
  }

  # Security Group is stateful — no need for explicit egress rule for replies
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_network_acl" "public_subnet_nacl" {
  vpc_id = aws_vpc.main.id
  subnet_ids = [aws_subnet.public.id]

  # NACL is STATELESS — must allow both inbound AND outbound ephemeral responses
  ingress {
    rule_no    = 100
    from_port  = 80
    to_port    = 80
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  ingress {
    rule_no    = 110
    from_port  = 443
    to_port    = 443
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  # CRITICAL: allow ephemeral inbound responses from internet (stateless requires this)
  ingress {
    rule_no    = 120
    from_port  = 1024
    to_port    = 65535
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  # Outbound: allow HTTP/S to internet, and ephemeral for responses to clients
  egress {
    rule_no    = 100
    from_port  = 80
    to_port    = 80
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  egress {
    rule_no    = 110
    from_port  = 443
    to_port    = 443
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  # Outbound ephemeral responses to clients
  egress {
    rule_no    = 120
    from_port  = 1024
    to_port    = 65535
    protocol   = "tcp"
    cidr_block = "0.0.0.0/0"
    action     = "allow"
  }

  # Deny all other traffic (implicit but best practice to have explicit deny)
  ingress {
    rule_no    = 200
    from_port  = 0
    to_port    = 0
    protocol   = "-1"
    cidr_block = "0.0.0.0/0"
    action     = "deny"
  }

  egress {
    rule_no    = 200
    from_port  = 0
    to_port    = 0
    protocol   = "-1"
    cidr_block = "0.0.0.0/0"
    action     = "deny"
  }
}
Output
# No runtime output — Terraform configuration.
# Key differences:
# - Security Group: stateful, so egress for replies is automatic when ingress allowed
# - Network ACL: stateless, requires explicit egress rules for ephemeral response ports
# - If you forget ephemeral rules in NACL, users see timeouts, not connection refused
# - Always use explicit deny as final rule in NACL for auditability
Watch Out: Stateless vs Stateful Confusion
When debugging a 'connection timeout' in AWS, check if the issue is in a Security Group (stateful) or Network ACL (stateless). If you see the TCP handshake SYN sent but no SYN-ACK received, the packet is being dropped at a stateless layer — likely a missing egress rule for ephemeral ports in your NACL.
Production Insight
The single most common cloud firewall mistake is leaving SSH open to 0.0.0.0/0.
Attackers scan the entire IPv4 space daily — your open port 22 will be found within hours.
Use a bastion host or AWS Systems Manager Session Manager instead of direct SSH access.
Key Takeaway
Security Groups are stateful, NACLs are stateless — they enforce at different layers.
Always allow ephemeral port ranges in NACLs for return traffic.
Never use 0.0.0.0/0 for SSH; restrict to your corporate CIDR or a bastion.

Proxy Authentication, Caching and Logging — What Actually Happens in Production

A proxy isn't just a relay — it's a traffic cop with memory. In production, proxies perform three crucial tasks beyond basic forwarding: authentication, caching, and logging.

Authentication: Forward proxies often require authentication (basic, digest, NTLM, or Kerberos) before allowing outbound access. Reverse proxies can validate JWT tokens or session cookies before the request reaches your backend. This offloads auth from your application and provides a single enforcement point. But misconfigured proxy auth can block legitimate traffic — especially if the proxy expects a header that your client doesn't send.

Caching: Reverse proxies like Nginx and Varnish cache static responses, reducing backend load by 50–80% for high-traffic endpoints. The key is cache invalidation: if you cache a user-specific response without varying on the session cookie, User A sees User B's data. Use the 'proxy_cache_key' directive to include headers like $http_cookie or $http_authorization for private content.

Logging: Proxies produce logs that are invaluable for debugging. Every request is logged with source IP, timestamp, URL, status code, and bytes transferred. But logging at high throughput (10k+ req/s) can overload the proxy's disk. Use buffered logging (syslog-ng, rsyslog) or ship logs to a centralized aggregator instead of writing directly to disk.

proxy_auth_cache_log.confNGINX
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# Nginx reverse proxy configuration with authentication, caching, and logging.

# Cache zone: 1GB memory, lasts 60 minutes, not accessed for 10 minutes = purge
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:1g max_size=10g inactive=10m use_temp_path=off;

server {
    listen 443 ssl http2;

    # ── Authentication: validate JWT from header before proxying ──────────
    # (Simplified — in production use auth_request subrequest or modules)
    location /api/ {
        # Custom auth check: if header missing, return 401
        if ($http_authorization = "") {
            return 401;
        }

        # Cache configuration — vary on Authorization header for private responses
        proxy_cache my_cache;
        proxy_cache_key "$scheme$request_method$host$request_uri$http_authorization";
        proxy_cache_valid 200 1h;       # Cache 200 OK for 1 hour
        proxy_cache_valid 404 1m;       # Cache 404 for 1 minute (avoids stampede)
        proxy_cache_use_stale error timeout updating;  # Serve stale on backend failure

        proxy_pass http://backend:3000;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
    }

    # ── Caching static assets aggressively ─────────────────────────────────
    location /static/ {
        proxy_cache my_cache;
        proxy_cache_valid 200 30d;      # Cache images for 30 days
        add_header Cache-Control "public, immutable";
        expires 30d;
        root /var/www/static;
    }

    # ── Logging with buffering to avoid disk I/O bottleneck ────────────────
    access_log /var/log/nginx/api_access.log buffer=32k flush=5s;
    error_log  /var/log/nginx/api_error.log warn;

    # JSON formatted access log for easier parsing
    log_format json escape=json
        '{'
            '"time":"$time_iso8601",'
            '"remote_addr":"$remote_addr",'
            '"request":"$request",'
            '"status":$status,'
            '"body_bytes_sent":$body_bytes_sent,'
            '"request_time":$request_time,'
            '"http_x_forwarded_for":"$http_x_forwarded_for"'
        '}';
    access_log /var/log/nginx/api_json.log json buffer=32k flush=5s;
}
Output
# Nginx config — no runtime output.
# When a request arrives:
# - If no Authorization header: 401 immediately
# - If cached response exists (keyed by method, URI, and auth header): serve from cache
# - Else forward to backend, cache the response, and log the transaction in JSON format
# - Buffered logging prevents disk I/O from blocking request processing
The Proxy as a Decoupling Layer
  • Authentication: proxy validates tokens, backend trusts that validation
  • Caching: proxy stores frequent responses, backend serves less
  • Logging: proxy records every request, backend logs only business events
  • Rate limiting: proxy drops excess traffic before it reaches your code
  • TLS termination: proxy handles encryption, backend runs plain HTTP
Production Insight
If your proxy cache keys don't include authentication data, you'll serve private data to the wrong users.
Always test cache invalidation before going live — a cached 401 response can lock all users out for hours.
Use proxy_cache_use_stale to serve stale content during backend failures — it's better than 502.
Key Takeaway
A proxy authenticates, caches, and logs — it's not just a relay.
Cache key variation must include auth headers for private content.
Buffered logging prevents proxy disk I/O from becoming a bottleneck at high throughput.
● Production incidentPOST-MORTEMseverity: high

Misordered Firewall Rule Caused 45-Minute Outage in Production

Symptom
Monitoring showed high outbound traffic from a backend server to an unknown IP address. The firewall was configured to block that IP, yet traffic was flowing. The incident started 45 minutes before the alarm fired.
Assumption
The team assumed that adding a DENY rule for the malicious IP at the end of the rule list would block it. They believed the default-deny policy would also catch any unmatched traffic.
Root cause
The DENY rule for the malicious IP was placed AFTER an ALLOW rule that permitted all HTTPS traffic (port 443) from any source. Because the firewall evaluates rules top-down and first-match-wins, the ALLOW rule matched first for the malicious IP's HTTPS traffic, and the subsequent DENY rule was never evaluated. The default-deny policy was only for packets that matched no rule, but the ALLOW rule matched, so the packet was allowed.
Fix
Moved the specific DENY rule for the malicious IP above the broad ALLOW rule. Added a rule audit step in the deployment pipeline that checks for any DENY rules that are positioned after a more general ALLOW rule. Implemented automated firewall rule order validation using iptables-save and a custom Python script that flags ordering anomalies.
Key lesson
  • Rule order is not cosmetic — first match wins, and a misplaced ALLOW can silently override DENY.
  • Always place specific DENY rules above broad ALLOW rules.
  • Automate rule order validation in your CI/CD pipeline — manual review misses subtle ordering issues.
  • Default-deny alone does not protect against ordering errors; it only applies when no rule matches.
Production debug guideSymptom-based guide to diagnosing connectivity problems through network security layers5 entries
Symptom · 01
Client gets 'connection timeout' (no SYN-ACK received)
Fix
Check network path: 1) Verify firewall allows inbound traffic on the target port. 2) For stateful firewalls, ensure the connection table isn't full. 3) For stateless (NACL), verify ephemeral port outbound rules. 4) Use traceroute to see where packets stop.
Symptom · 02
Client gets 'connection refused' (RST received)
Fix
The packet reached the target but the backend process is not listening on that port. Check: 1) Service is running. 2) Reverse proxy is properly forwarding to the correct backend port. 3) No intermediate firewall is actively rejecting (RST) instead of dropping (timeout).
Symptom · 03
Requests work locally but fail through reverse proxy
Fix
1) Check proxy logs for 502/503 errors. 2) Verify proxy_pass URL and backend health. 3) Check if the proxy modifies request headers (Host, X-Forwarded-For) and if backend expects them. 4) Verify TLS certificates on proxy if terminating HTTPS.
Symptom · 04
Cloud firewall shows 'allow' but traffic still fails
Fix
1) Check both Security Group (stateful) AND Network ACL (stateless) — both must allow the traffic. 2) Verify route tables point to correct internet gateway. 3) Check if the instance has a public IP or is behind a NAT gateway. 4) Use VPC Flow Logs to see if traffic is being accepted or rejected.
Symptom · 05
Application receives requests from unexpected IP (e.g., 127.0.0.1 or proxy IP)
Fix
1) Configure proxy to set X-Forwarded-For header. 2) Configure backend application to read X-Forwarded-For instead of remote_addr. 3) In Nginx, ensure proxy_set_header X-Forwarded-For $remote_addr; is set. 4) Validate that only your proxy can set this header (use set_real_ip_from).
★ Quick Firewall & Proxy Debugging Cheat SheetCommands and checks for diagnosing the most common firewall and proxy issues in production.
Can't reach service on port 443 from outside
Immediate action
Check if port is open on the firewall: nc -zv 203.0.113.10 443
Commands
iptables -L -n -v | grep 443
sudo netstat -tulpn | grep :443
Fix now
Add firewall rule: iptables -A INPUT -p tcp --dport 443 -j ACCEPT (or AWS Security Group rule)
Reverse proxy returning 502 Bad Gateway+
Immediate action
Check if backend service is running: curl -I http://127.0.0.1:3000/health
Commands
tail -100 /var/log/nginx/error.log | grep 'connect() failed'
systemctl status backend.service
Fix now
Restart backend service or fix proxy_pass address in nginx.conf
Application logs show all IPs as 127.0.0.1+
Immediate action
Check proxy config for X-Forwarded-For header
Commands
grep proxy_set_header /etc/nginx/sites-enabled/*
curl -v -H 'X-Forwarded-For: 1.2.3.4' http://app/
Fix now
Add 'proxy_set_header X-Forwarded-For $remote_addr;' to nginx location block
AWS Security Group rule allows SSH but connection times out+
Immediate action
Check if NACL is blocking ephemeral ports
Commands
aws ec2 describe-network-acls --filters Name=association.subnet-id,Values=subnet-xxxx
Check VPC Flow Logs: aws logs filter-log-events --log-group-name /vpc/flow-logs
Fix now
Add inbound NACL rule for ephemeral ports (1024-65535) from 0.0.0.0/0
Firewall vs Reverse Proxy — Comparison
Feature / AspectFirewallProxy (Reverse)
Primary jobAllow or deny traffic based on rulesRoute and relay traffic between clients and servers
Operates at OSI layerLayer 3-4 (packet/transport) or Layer 7 (NGFW)Layer 7 (application — HTTP, HTTPS, WebSocket)
Sees packet content?Only with NGFW / DPI enabledYes — always reads HTTP headers and request path
Hides server identity?No — it blocks, but IP may still be probedYes — clients talk to proxy IP, not backend IP
TLS termination?No (unless specifically a TLS inspection proxy)Yes — standard feature in Nginx, HAProxy, ALB
Rate limiting?Only at IP level (connection rate)Yes — per-route, per-header, per-user-agent
Caching responses?NoYes — Nginx, Varnish, Cloudflare all cache
Typical real toolsiptables, AWS Security Groups, pfSense, Palo AltoNginx, HAProxy, Cloudflare, AWS ALB, Traefik
Set up byNetwork / DevOps / Security engineerDevOps / Backend engineer
First line of defense?Yes — blocks at the network perimeterSecond layer — after network firewall

Key takeaways

1
A firewall controls WHETHER traffic is allowed; a proxy controls HOW traffic is relayed
they solve different problems and should be layered together, not treated as alternatives.
2
Rule order in a firewall is not cosmetic
the first matching rule wins and evaluation stops. A misplaced ALLOW rule above a DENY rule can silently negate your entire security policy.
3
A reverse proxy is your server's bodyguard
it terminates TLS, hides your backend topology, rate-limits abusive clients, strips leaky headers, and can cache responses — all before a single byte reaches your application code.
4
X-Forwarded-For is useful for preserving client IP through a proxy chain, but it's trivially spoofable if you don't restrict which IPs are allowed to set it. Always validate it against known proxy addresses before trusting it for security logic.
5
Cloud firewalls have two layers
Security Groups (stateful) and NACLs (stateless). Forgetting ephemeral port rules in NACLs causes frustrating connection timeouts.

Common mistakes to avoid

3 patterns
×

Confusing forward and reverse proxies in interviews

Symptom
Saying 'a proxy hides the client' is only half true and will get you corrected. Interviewees fail to distinguish which side is hidden.
Fix
Always state the direction: 'A forward proxy hides the client from the server; a reverse proxy hides the server from the client.' Example: 'This is a reverse proxy — it sits in front of our servers. Clients have no idea our backends are Node.js instances on port 3000.'
×

Trusting X-Forwarded-For blindly for security decisions

Symptom
Attackers can spoof the header by sending 'X-Forwarded-For: 127.0.0.1' in their request, bypassing rate limits or geo-blocking that depend on it.
Fix
Only trust this header if it was set by YOUR proxy. In Nginx, use '$realip_module' with 'set_real_ip_from' pointing to your known proxy IPs — then use '$remote_addr' instead of the raw header value for security logic.
×

Using a default-ALLOW firewall policy

Symptom
Blocking specific bad IPs but allowing everything else (denylist) is fundamentally insecure because you can't enumerate all bad actors.
Fix
Default to DENY ALL, then explicitly allow only what your application needs (allowlist). This means an attacker probing an unknown port gets silence, not an open door.
INTERVIEW PREP · PRACTICE MODE

Interview Questions on This Topic

Q01SENIOR
What's the difference between a forward proxy and a reverse proxy? Can y...
Q02SENIOR
If a client sends a request through a reverse proxy to your backend, and...
Q03SENIOR
A stateless packet-filtering firewall is blocking all traffic on port 44...
Q01 of 03SENIOR

What's the difference between a forward proxy and a reverse proxy? Can you give a real-world example of each, and explain what each side of the connection knows about the other?

ANSWER
A forward proxy sits between clients and the internet. Clients explicitly configure their browser to use it. The destination server sees the proxy's IP, not the client's. Example: a corporate web filter like Squid. A reverse proxy sits between the internet and your servers. Clients think they're talking directly to your backend. The proxy terminates TLS, routes requests, and hides backend identity. Example: Nginx in front of an API. In a forward proxy, the CLIENT knows about the proxy; the SERVER does not know the client. In a reverse proxy, the CLIENT does not know about the proxy; the SERVER receives requests from the proxy.
FAQ · 5 QUESTIONS

Frequently Asked Questions

01
What is the difference between a firewall and a proxy server?
02
Does a VPN work like a proxy?
03
Can a firewall stop application-layer attacks like SQL injection?
04
What is the difference between a Security Group and a Network ACL in AWS?
05
How do I choose between a forward proxy and a reverse proxy?
🔥

That's Computer Networks. Mark it forged?

5 min read · try the examples if you haven't

Previous
Network Security Basics
13 / 22 · Computer Networks
Next
VPN Explained