iptables: Linux Firewall Configuration and Management — Rules That Don't Burn You at 3 AM

I've seen a single misplaced iptables rule take down a production payment gateway at 2 AM on a Saturday. The rule was meant to block a malicious IP, but a missing -s flag made it block all traffic. The on-call engineer spent 45 minutes SSH'd into the DR site, sweating, because the primary's iptables had locked out even the monitoring system. That's the power of iptables: one line can save your bacon or deep-fry it. This isn't a tutorial for hobbyists — it's a survival guide for engineers who manage firewalls in production, where a typo means a P0 incident.

Netfilter Architecture: The Kernel's Packet Processing Pipeline

Before you touch iptables, you need to understand netfilter — the kernel framework that iptables talks to. Netfilter hooks into five key points in the network stack: PREROUTING, INPUT, FORWARD, OUTPUT, and POSTROUTING. Each hook corresponds to a chain in iptables. When a packet arrives, it traverses these hooks in order. The filter table's INPUT chain runs at the INPUT hook, deciding whether to deliver the packet locally. The nat table's PREROUTING chain runs at the PREROUTING hook, before routing decisions. Understanding this flow is critical: a rule in the wrong table or chain won't work as expected. For example, DNAT (destination NAT) must be in the PREROUTING chain of the nat table, not in the filter table. I've seen engineers waste hours trying to DNAT in the FORWARD chain — it doesn't work because FORWARD runs after routing.

// io.thecodeforge — DevOps tutorial

# Show the packet flow through netfilter hooks
# This is a conceptual diagram, not runnable code

# Packet flow for incoming packet destined to local process:
# NIC -> PREROUTING (raw, mangle, nat) -> Routing Decision -> INPUT (mangle, filter) -> Local Process

# Packet flow for forwarded packet:
# NIC -> PREROUTING (raw, mangle, nat) -> Routing Decision -> FORWARD (mangle, filter) -> POSTROUTING (mangle, nat) -> NIC

# Packet flow for locally generated outgoing packet:
# Local Process -> Routing Decision -> OUTPUT (raw, mangle, nat, filter) -> POSTROUTING (mangle, nat) -> NIC

# Verify current hook traversal with trace: iptables -t raw -A PREROUTING -j TRACE
# Then check kernel log: dmesg -w | grep TRACE

Tables and Chains: Where Rules Live and Why It Matters

iptables has five tables: filter (default), nat, mangle, raw, and security. Each table has built-in chains. The filter table has INPUT, FORWARD, and OUTPUT chains. The nat table has PREROUTING, OUTPUT, and POSTROUTING. Mangle has all five. Raw has PREROUTING and OUTPUT. Security is for SELinux. Most people only use filter, but ignoring the others leads to brittle setups. For example, rate-limiting SYN packets should be done in the mangle table's PREROUTING chain, not filter's INPUT, because mangle runs before the first routing decision and can drop packets before they consume connection tracking resources. I once saw a DDoS knock over a server because the rate limit was in filter INPUT — the connection tracking table filled up before the rule was evaluated. Move it to mangle PREROUTING and the server survived.

// io.thecodeforge — DevOps tutorial

# List all rules in all tables
for table in filter nat mangle raw security; do
  echo "=== Table: $table ==="
  iptables -t $table -L -n -v --line-numbers 2>/dev/null || echo "Table $table not available"
done

# Example output for filter table (abbreviated):
# === Table: filter ===
# Chain INPUT (policy DROP 0 packets, 0 bytes)
# num   pkts bytes target     prot opt in     out     source               destination
# 1        0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
# 2      100  5000 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state ESTABLISHED,RELATED
# 3       10   600 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
# Chain FORWARD (policy DROP 0 packets, 0 bytes)
# Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)

Rule Syntax: The Devil in the Flags

An iptables rule is a chain of matches and one target. The basic structure: iptables -A <chain> <matches> -j <target>. Matches include -s (source), -d (destination), -p (protocol), --dport (destination port), -i (input interface), -o (output interface), -m state (connection state), -m limit (rate limit), and many more. The target can be ACCEPT, DROP, REJECT, LOG, RETURN, or a user-defined chain. The order of matches matters: put the most specific matches first to short-circuit evaluation. For example, -m state --state ESTABLISHED,RELATED -j ACCEPT should be near the top to allow return traffic. A common mistake is putting -j DROP before allowing established connections — that kills all return traffic. Also, remember that -s and -d can be IP addresses or CIDR ranges. Never use hostnames; iptables resolves them at rule insertion time, and if DNS fails, the rule won't apply.

// io.thecodeforge — DevOps tutorial

# Allow all loopback traffic (essential for local services)
iptables -A INPUT -i lo -j ACCEPT

# Allow established and related connections (so return traffic works)
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH from specific management subnet
iptables -A INPUT -p tcp -s 10.0.0.0/8 --dport 22 -j ACCEPT

# Allow HTTP/HTTPS from anywhere
iptables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT

# Log and drop everything else (log first, then drop)
iptables -A INPUT -j LOG --log-prefix "iptables-dropped: " --log-level 4
iptables -A INPUT -j DROP

# Set default policies (should be done after rules)
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Verify: iptables -L -n -v

Connection Tracking: The Hidden State Machine

Connection tracking (conntrack) is what makes stateful firewalling possible. The -m state module uses conntrack to classify packets as NEW, ESTABLISHED, RELATED, or INVALID. Without it, you'd have to open ephemeral ports for return traffic. Conntrack tracks all connections in a table in kernel memory. The table size is configurable via net.netfilter.nf_conntrack_max. Default is often 65536, which is too low for busy servers. When the table fills, new connections are dropped silently. I've seen this happen on a web server handling 10k concurrent connections — the symptom was intermittent timeouts. The fix: increase nf_conntrack_max and reduce nf_conntrack_tcp_timeout_established from the default 5 days to something sane like 1 hour. Also, use the raw table to bypass conntrack for high-traffic UDP services (like DNS) with -j NOTRACK.

// io.thecodeforge — DevOps tutorial

# Check current conntrack usage
cat /proc/net/nf_conntrack | wc -l
sysctl net.netfilter.nf_conntrack_max
sysctl net.netfilter.nf_conntrack_count

# Increase conntrack table size (persist in /etc/sysctl.conf)
sysctl -w net.netfilter.nf_conntrack_max=262144

# Reduce established timeout from default 432000 seconds (5 days) to 3600 (1 hour)
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_established=3600

# Bypass conntrack for DNS (UDP 53) using raw table
iptables -t raw -A PREROUTING -p udp --dport 53 -j NOTRACK
iptables -t raw -A OUTPUT -p udp --sport 53 -j NOTRACK

# Monitor conntrack in real time
conntrack -E -o timestamp

NAT: Masquerading, DNAT, and the Routing Trap

Network Address Translation in iptables is done in the nat table. SNAT (source NAT) is used for masquerading — hiding internal IPs behind a single public IP. DNAT (destination NAT) is used for port forwarding. The classic setup: iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE for SNAT. For DNAT: iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.100:8080. A common mistake is forgetting to enable IP forwarding: sysctl net.ipv4.ip_forward=1. Without it, DNAT won't work. Also, DNAT only changes the destination — the return traffic must be un-DNATed automatically by conntrack. If you're doing DNAT to a different port, the return traffic's source port is the original port, not the translated one. This usually works, but can break with some protocols. For hairpin NAT (accessing a DNATed service from inside the same network), you need additional rules.

// io.thecodeforge — DevOps tutorial

# Enable IP forwarding (must be persistent in /etc/sysctl.conf)
sysctl -w net.ipv4.ip_forward=1

# SNAT: Masquerade all outgoing traffic on eth0 (typical for home router)
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

# DNAT: Forward incoming HTTP to internal server 192.168.1.100:8080
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.100:8080

# Allow forwarded traffic through filter table (FORWARD chain)
iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 8080 -d 192.168.1.100 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT

# Hairpin NAT: Allow internal clients to access the DNATed service using the public IP
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.1.100 -p tcp --dport 8080 -j SNAT --to-source 192.168.1.1

# Verify NAT rules
iptables -t nat -L -n -v

Rate Limiting and DDoS Mitigation with iptables

iptables can do basic rate limiting using the limit and connlimit modules. The limit module matches at a certain rate (e.g., 10/second) and can be used to throttle ICMP or SYN packets. The connlimit module limits the number of concurrent connections per IP. For SYN flood mitigation, use -m limit --limit 10/s --limit-burst 20 in the mangle table's PREROUTING chain. This drops excess SYN packets before they reach conntrack. For per-IP connection limits: -m connlimit --connlimit-above 100 --connlimit-mask 32 -j DROP. This is useful for preventing a single client from exhausting server resources. However, iptables is not a full DDoS mitigation solution — it's a first line of defense. For large attacks, you need a dedicated appliance or cloud-based protection. Also, be careful with connlimit on NATed clients; they all share the same public IP, so you might block legitimate users.

// io.thecodeforge — DevOps tutorial

# Rate limit incoming SSH connections to prevent brute force
# Allow 3 new connections per minute, burst of 5
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m limit --limit 3/minute --limit-burst 5 -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j DROP

# Limit SYN packets to 100 per second (place in mangle PREROUTING)
iptables -t mangle -A PREROUTING -p tcp --syn -m limit --limit 100/s --limit-burst 200 -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp --syn -j DROP

# Limit concurrent connections per IP to 50 for HTTP
iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 50 --connlimit-mask 32 -j REJECT --reject-with tcp-reset

# Limit total connections to 1000
iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 1000 -j DROP

# Log dropped SYN packets
iptables -A INPUT -p tcp --syn -j LOG --log-prefix "SYN-DROP: " --log-level 4

Logging and Debugging: Seeing What the Firewall Sees

The LOG target sends packet info to the kernel log (viewable via dmesg or /var/log/kern.log). Use --log-prefix to tag entries for easy grepping. The ULOG target sends logs to userspace via netlink, which is more efficient. For debugging, use the TRACE target in the raw table to trace packet traversal through all chains. This is invaluable when a rule isn't matching as expected. Also, iptables -L -n -v shows packet and byte counters per rule. If a rule's counter isn't incrementing, the packet isn't reaching that rule — check previous rules or chain order. Another trick: insert a temporary LOG rule at the top of a chain to see if packets arrive. Remember to remove it afterwards.

// io.thecodeforge — DevOps tutorial

# Add a LOG rule at the top of INPUT to see all incoming packets
iptables -I INPUT 1 -j LOG --log-prefix "INPUT-PACKET: " --log-level 4

# Watch kernel log in real time
dmesg -w | grep INPUT-PACKET

# Use TRACE to follow packet through chains
iptables -t raw -A PREROUTING -p tcp --dport 80 -j TRACE
# Then trigger a packet and check dmesg
curl http://localhost:80
dmesg | grep TRACE

# Remove TRACE rule after debugging
iptables -t raw -D PREROUTING -p tcp --dport 80 -j TRACE

# Check rule counters
iptables -L -n -v --line-numbers

# Reset counters for a specific chain
iptables -Z INPUT

Persistence: Making Rules Survive Reboot

iptables rules are ephemeral — they vanish on reboot. To make them persistent, save them to a file and restore on boot. On Debian/Ubuntu, use iptables-save > /etc/iptables/rules.v4 and install iptables-persistent. On RHEL/CentOS, use service iptables save which writes to /etc/sysconfig/iptables. The modern way is to use iptables-restore with a systemd service. Never rely on startup scripts that re-run iptables commands — they're fragile. Instead, maintain a single rules file versioned in your config management. Also, always test a new ruleset by saving the current one first: iptables-save > /root/iptables.backup.$(date +%s). If something breaks, restore with iptables-restore < /root/iptables.backup....

// io.thecodeforge — DevOps tutorial

# Save current rules to file
iptables-save > /etc/iptables/rules.v4

# Restore rules from file
iptables-restore < /etc/iptables/rules.v4

# Create a systemd service to restore rules on boot (if not using iptables-persistent)
cat > /etc/systemd/system/iptables-restore.service <<EOF
[Unit]
Description=Restore iptables rules
Before=network-pre.target
Wants=network-pre.target

[Service]
Type=oneshot
ExecStart=/sbin/iptables-restore /etc/iptables/rules.v4
ExecReload=/sbin/iptables-restore /etc/iptables/rules.v4
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF

systemctl enable iptables-restore
systemctl start iptables-restore

# Backup current rules before any change
iptables-save > /root/iptables.backup.$(date +%s)

# Atomic reload: flush and restore in one command
iptables-restore <<< "$(iptables-save | sed 's/^-A/-I/') "  # This is dangerous; better to use a file

Performance: How Rules Affect Throughput

iptables rules are evaluated in linear order. The more rules, the slower the packet processing. For high-throughput environments (10Gbps+), minimize the number of rules and use the most specific matches first. Use iptables -L -n -v to identify rules with high packet counts — those are your hot paths. Consider moving frequently matched rules to the top. Also, use the -m set module with ipset for large IP blocklists — it uses hash tables and is O(1) instead of O(n). For connection tracking, ensure nf_conntrack_max is adequate and timeouts are tuned. On multi-core systems, iptables uses a single lock, which can become a bottleneck. Newer nftables (the successor to iptables) has better concurrency. If you're pushing serious traffic, consider migrating to nftables or using XDP/BPF for packet filtering.

// io.thecodeforge — DevOps tutorial

# Check rule hit counts to identify hot paths
iptables -L -n -v --line-numbers | head -20

# Example output showing a rule with high packet count:
# num   pkts bytes target     prot opt in     out     source               destination
# 1   500000  50M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state ESTABLISHED,RELATED
# 2   100000  10M ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80

# Use ipset for large blocklists (e.g., 10,000 IPs)
ipset create blocklist hash:ip maxelem 100000
iptables -A INPUT -m set --match-set blocklist src -j DROP

# Benchmark packet processing (using perf or simply measure throughput)
# Example: send 1 million packets and measure CPU usage
# Not a command, but a methodology: use `sar -n DEV 1` to see pps (packets per second)

When Not to Use iptables: Modern Alternatives

iptables is legacy. The modern replacement is nftables, which offers better performance, atomic rule replacement, and a more consistent syntax. nftables also supports sets, maps, and concatenations natively. For container environments, consider using CNI plugins like Calico or Cilium that use eBPF for high-performance networking. For host-based firewalls, firewalld (which uses nftables under the hood) provides a dynamic zone-based interface. If you're starting a new project, use nftables. If you're maintaining legacy systems, iptables is still fine, but plan to migrate. The migration is straightforward: iptables-translate can convert rules. However, for simple setups, iptables is still widely used and understood. Don't rewrite a working system just because it's old — but don't start new projects with it.

// io.thecodeforge — DevOps tutorial

# Convert iptables rules to nftables format
iptables-save > /tmp/rules.v4
iptables-restore-translate -f /tmp/rules.v4 > /tmp/rules.nft

# Apply nftables rules
nft -f /tmp/rules.nft

# Example nftables equivalent of a simple iptables rule:
# iptables -A INPUT -p tcp --dport 22 -j ACCEPT
# nft add rule inet filter input tcp dport 22 accept

# List nftables rules
nft list ruleset

# Note: nftables uses a single ruleset with tables and chains, not multiple tables like iptables.

Production Checklist: Deploying iptables Safely

Before you deploy any iptables change in production, follow this checklist. First, ensure you have out-of-band access (IPMI, iDRAC, serial console) in case you lock yourself out. Second, save the current rules: iptables-save > /root/iptables.backup. Third, write your new rules in a file and test with iptables-restore --test < newrules.v4 to check for syntax errors. Fourth, apply with a timeout using iptables-apply -t 60 newrules.v4. Fifth, verify connectivity from another host: nc -zv <server> 22. Sixth, monitor logs for dropped packets: dmesg -w | grep iptables. Seventh, if everything works, make the rules persistent. Eighth, document the change in your runbook. Never make changes directly on a production server without a rollback plan. I've seen too many engineers panic when SSH drops — having a backup plan is non-negotiable.

// io.thecodeforge — DevOps tutorial

# Step 1: Backup current rules
iptables-save > /root/iptables.backup.$(date +%s)

# Step 2: Test new rules syntax
iptables-restore --test < /path/to/newrules.v4 || echo "Syntax error!"

# Step 3: Apply with timeout (60 seconds)
iptables-apply -t 60 /path/to/newrules.v4
# If you lose SSH, rules revert after 60 seconds

# Step 4: Verify connectivity from another host
ssh admin@monitoring-host "nc -zv $TARGET_SERVER 22"

# Step 5: Check logs for drops
dmesg | grep -i "iptables.*DROP" | tail -10

# Step 6: Make persistent
iptables-save > /etc/iptables/rules.v4

# Step 7: Document in runbook
echo "Deployed new iptables rules on $(date) by $USER" >> /var/log/iptables_changelog