Linux Networking Commands — Why Ping Works But TCP Fails

Every production outage has a moment — usually at 2am — where someone types a networking command into a terminal and either finds the problem in 30 seconds or spends three hours guessing. Linux networking tools are what separates a DevOps engineer who can diagnose a flaky microservice from one who just restarts containers and hopes for the best. These commands are your stethoscope for the network layer.

Why Ping Works But TCP Fails

Linux networking commands are the tools that expose the OSI stack's raw behavior — they let you probe, diagnose, and manipulate network state from user space. Ping uses ICMP echo requests, which operate at the network layer (L3) and require no port or connection state. TCP, on the other hand, requires a three-way handshake, kernel socket buffers, and firewall rules that inspect L4 headers. This fundamental difference means ping can succeed while TCP connections hang or reset.

In practice, ICMP packets bypass iptables rules that filter TCP ports, and they don't consume ephemeral ports or socket file descriptors. A server can respond to ping even when its TCP listen backlog is full, or when the application process has crashed but the kernel network stack still handles ICMP. This is why ping is a poor proxy for application health — it tests L3 reachability, not service availability.

Use ping first to rule out L1-L3 issues (cable, ARP, routing). When ping succeeds but TCP fails, focus on firewall rules (iptables, nftables), socket limits (net.core.somaxconn, net.ipv4.tcp_max_syn_backlog), and application process state. In production, always pair ping with a TCP-specific probe like nc -zv or curl to get the full picture.

ip vs ifconfig — Why the Old Tool Is Dead and What Replaced It

For years, ifconfig was the go-to command for inspecting network interfaces. It still works on many systems, but it's been deprecated and is no longer installed by default on modern Linux distributions like Ubuntu 20.04+ and RHEL 8+. The replacement is the ip command, which is part of the iproute2 package and talks directly to the kernel's netlink socket instead of parsing /proc files.

The key difference isn't just syntax — it's capability. ip can manage routing tables, network namespaces, tunnels, and ARP/NDP caches all through one unified tool. ifconfig could only touch interfaces and basic IP configuration.

When you're debugging a container networking issue in Kubernetes, you'll often drop into a pod's network namespace and run ip addr to see what the container thinks its IP is. That's impossible with ifconfig, which has no namespace awareness. Know ip deeply and you'll be comfortable anywhere from a bare-metal server to a Docker container.

#!/bin/bash
# Shows practical ip command usage for interface inspection
# Run these on any modern Linux host

# List all network interfaces with their IP addresses
# The 'addr' subcommand shows Layer 3 (IP) info bound to each interface
ip addr show

# Filter to just one interface — useful when you have many (eth0, lo, docker0, etc.)
ip addr show dev eth0

# Show the routing table — which gateway handles which destination networks
# This is the first thing to check when packets aren't leaving the host
ip route show

# Show the default gateway specifically — the 'exit door' for all unknown traffic
ip route show default

# Add a temporary static route for a specific subnet via a specific gateway
# This survives until reboot — use /etc/netplan or /etc/network/interfaces for persistence
ip route add 192.168.50.0/24 via 10.0.0.1 dev eth0

# Bring an interface DOWN and back UP without rebooting
# Useful after changing IP config or when an interface is in a bad state
ip link set eth0 down
ip link set eth0 up

# Check ARP cache — maps IP addresses to MAC addresses on your local network
# If a host's MAC shows as INCOMPLETE, ARP resolution is failing — suspect a firewall or VLAN issue
ip neigh show

ss and netstat — Seeing Every Open Door on Your Server

Think of your server as a building with thousands of numbered doors (ports). ss and netstat let you see exactly which doors are open, who's standing at each one, and which processes are responsible. This is critical when deploying a new service — you need to know whether port 8080 is already taken before your app fails to bind to it.

netstat is the old tool, ss (Socket Statistics) is the modern replacement. ss talks directly to the kernel via netlink, which makes it dramatically faster on systems with thousands of connections. On a busy web server, netstat can take 10+ seconds while ss returns instantly.

The real power of ss is in its filtering. You can filter by state (ESTABLISHED, LISTEN, TIME_WAIT), by port, by process, or by remote address. In a microservices environment, you might want to see all connections from this service to the database on port 5432 — ss makes that a one-liner.

Understanding TCP connection states matters here. TIME_WAIT is normal and means your server is waiting for late packets before closing a connection. A flood of TIME_WAIT entries is usually fine. CLOSE_WAIT means the remote side closed but your application hasn't — that often points to a bug in connection handling code.

#!/bin/bash
# Practical ss usage patterns for diagnosing connection issues

# Show all LISTENING TCP ports with the process that owns them
# -t = TCP only, -l = listening sockets, -n = numeric (don't resolve hostnames, much faster),
# -p = show the process (requires root or sudo for processes owned by other users)
ss -tlnp

# Show all ESTABLISHED connections — who is currently connected to this server
ss -tn state established

# Find what's using a specific port (e.g., 8080)
# The 'sport' filter means 'source port' — the local port your service is bound to
ss -tlnp sport = :8080

# Count connections per remote IP — useful for spotting a single client hammering your API
# This pipes ss output through awk to extract remote IPs and count occurrences
ss -tn state established | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head -20

# Show all UDP sockets (DNS, DHCP, NTP all use UDP — don't forget to check these)
ss -ulnp

# Show connection summary statistics by state
# Huge TIME_WAIT count is normal; huge CLOSE_WAIT count may indicate a connection leak
ss -s

# The netstat equivalent for those on older systems without ss
# -a = all sockets, -n = numeric, -t = TCP, -u = UDP, -p = process
netstat -antp

curl, wget and dig — Testing Connectivity Layer by Layer

When a service is unreachable, you need to narrow down the layer where things break. Is it DNS? TCP connectivity? HTTP routing? TLS? curl is exceptional here because it can test each layer independently and gives you precise timing data. dig is your dedicated DNS debugging tool, and together they let you methodically eliminate suspects.

curl's --verbose flag is one of the most useful things in networking debugging. It shows you the DNS resolution, the TCP handshake, the TLS negotiation, and the HTTP headers — all in sequence. When your HTTPS endpoint is slow, curl -w timing reveals whether the slowness is in DNS, in TCP, in TLS, or in the actual server response time.

dig is purpose-built for DNS and goes far beyond what you can learn from ping or curl. You can query specific record types, target specific nameservers, and trace the full delegation chain from root to authoritative server. This matters when you're investigating DNS propagation issues after a domain change, or debugging split-horizon DNS in a VPN setup.

wget is simpler and better for quick file downloads or when you need recursive mirroring. For API testing and network diagnosis, curl is almost always the right choice.

#!/bin/bash
# Test each networking layer independently to isolate where failures occur

# ─── DNS TESTING WITH dig ───────────────────────────────────────────────

# Basic A record lookup — what IP does this hostname resolve to?
dig api.example.com A

# Query a SPECIFIC nameserver (e.g., Google's 8.8.8.8) to bypass your local resolver
# Useful when debugging 'works on my machine' DNS issues caused by local caching
dig @8.8.8.8 api.example.com A

# Trace the full DNS delegation from root servers down to the authoritative nameserver
# This is the gold standard for debugging DNS propagation issues
dig +trace api.example.com

# Check MX records (mail routing) — +short gives just the values, no noise
dig api.example.com MX +short

# Reverse DNS lookup — find the hostname for an IP address
dig -x 93.184.216.34 +short

# ─── HTTP/HTTPS TESTING WITH curl ───────────────────────────────────────

# Follow redirects (-L), show verbose output (-v) — see every step of the connection
curl -Lv https://api.example.com/health

# Time each phase of the connection — invaluable for performance diagnosis
# This custom format prints timing for each phase on separate lines
curl -o /dev/null -s -w "
DNS lookup:        %{time_namelookup}s
TCP connect:       %{time_connect}s
TLS handshake:     %{time_appconnect}s
Time to first byte: %{time_starttransfer}s
Total time:        %{time_total}s
HTTP status code:  %{http_code}
" https://api.example.com/health

# Test an API endpoint with a JSON POST body — simulates what your app does
curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-token-here" \
  -d '{"username": "alice", "email": "alice@example.com"}' \
  --max-time 10  # fail after 10 seconds instead of hanging forever

# Test TLS certificate details — check expiry, issuer, and SANs
curl -vI https://api.example.com 2>&1 | grep -E '(subject|issuer|expire|SSL)'

# Skip TLS verification (ONLY for debugging self-signed certs — never in production)
curl -k https://internal-service.local/health

ping, traceroute and tcpdump — Tracing the Path and Catching Packets

Once you've confirmed DNS resolves correctly and the service is listening, the next question is whether packets are actually reaching their destination. ping tells you if a host is reachable and measures round-trip time. traceroute shows you every hop between you and the destination. tcpdump lets you actually capture and inspect raw packets on the wire.

ping is often misused as a binary 'is it up?' test, but it's more nuanced than that. ICMP packets (what ping uses) can be blocked by firewalls while TCP traffic flows fine. A failed ping doesn't mean a service is down — it might just mean ICMP is blocked. Always follow up a failed ping with a TCP-level check.

traceroute reveals the routing path and where latency is introduced. Each hop shows you a router, and timing spikes between hops show you where delays occur. When a cloud VM can't reach an external API, traceroute often reveals the packet dying at a NAT gateway or security group that's silently dropping traffic.

tcpdump is the most powerful of the three but also the most complex. It captures actual packet data, which is essential for diagnosing issues that higher-level tools can't see — like retransmissions, RST floods, or malformed HTTP headers. Always combine it with Wireshark for complex analysis.

#!/bin/bash
# Trace packet paths and capture traffic for deep network diagnosis

# ─── PING ───────────────────────────────────────────────────────────────

# Basic connectivity check — send 5 packets, show statistics
ping -c 5 8.8.8.8

# Ping with a larger packet size to test MTU issues (1472 bytes + 28 byte IP/ICMP header = 1500 MTU)
# If this fails but small pings work, you likely have an MTU mismatch (common with VPNs)
ping -c 4 -s 1472 -M do 8.8.8.8  # -M do means 'don't fragment'

# ─── TRACEROUTE ─────────────────────────────────────────────────────────

# Standard traceroute using UDP probes
traceroute api.example.com

# Use TCP SYN probes on port 443 instead of UDP
# More likely to get through firewalls that block ICMP and UDP probes
traceroute -T -p 443 api.example.com

# mtr combines ping and traceroute — shows live packet loss per hop
# This is often the single best tool for diagnosing intermittent routing issues
mtr --report --report-cycles 20 api.example.com

# ─── TCPDUMP ────────────────────────────────────────────────────────────

# Capture all traffic on eth0 — CTRL+C to stop
# -n = don't resolve hostnames (faster), -i = interface
tcpdump -i eth0 -n

# Capture only HTTP traffic (port 80 or 443) to/from a specific host
# 'host' filters by IP in either direction
tcpdump -i eth0 -n host 10.0.1.20 and \( port 80 or port 443 \)

# Capture DNS queries — see what your server is resolving and to which nameserver
tcpdump -i eth0 -n port 53

# Save capture to a file for later analysis in Wireshark
# -w writes raw packets, -C 100 rotates files at 100MB to avoid filling the disk
tcpdump -i eth0 -n -w /tmp/capture.pcap -C 100 host 10.0.1.20

# Capture and print packet payloads in ASCII — see the actual HTTP request/response text
# -A = ASCII output, -s 0 = capture full packet (no truncation)
tcpdump -i eth0 -n -A -s 0 port 8080 | grep -E '(GET|POST|HTTP|Host:)'

Network Performance Monitoring — Bandwidth, Throughput and Bottlenecks

Production servers don't just need connectivity — they need performance. When a service becomes slow, you need to know whether the bottleneck is your server's network interface, the application itself, or somewhere in between. Tools like nload, iftop, iptraf-ng, and nethogs give you real-time bandwidth and per-process traffic data.

nload shows total incoming and outgoing traffic on each interface with a live graph. iftop shows traffic per connection — which remote IPs are consuming the most bandwidth. iptraf-ng adds detailed statistics per protocol and interface. nethogs breaks down traffic per process, so you can see which application is saturating the link.

A common production scenario: a misconfigured backup job or a rogue cron script starts transferring gigabytes of data and saturates the NIC. nethogs reveals 'python3' as the culprit. iftop shows the destination IP is an internal backup server. You then find the backup script is running on the wrong schedule.

For throughput testing, iperf3 is essential. It measures actual TCP/UDP throughput between two hosts, revealing issues like buffer bloat or misconfigured flow control that won't appear in ping.

#!/bin/bash
# Real-time network performance monitoring tools

# ─── INSTALLATION ───────────────────────────────────────────────────────
# sudo apt install nload iftop iptraf-ng nethogs iperf3 -y

# ─── NLOAD: total bandwidth per interface ───────────────────────────────
# Run with interface name (or all interfaces default)
nload eth0

# ─── IFTOP: bandwidth per connection ────────────────────────────────────
# Shows top talkers; press 'p' to toggle port display
sudo iftop -i eth0

# ─── IPTRAF-NG: per-protocol statistics ────────────────────────────────
sudo iptraf-ng
# Then choose 'IP traffic monitor' or 'Detailed interface statistics'

# ─── NETHOGS: bandwidth per process ────────────────────────────────────
sudo nethogs eth0

# ─── IPERF3: throughput test between two hosts ──────────────────────────
# Start server on host B
iperf3 -s

# Connect from host A (client)
iperf3 -c 10.0.1.20 -t 30  # run for 30 seconds

# Test UDP throughput (includes jitter and packet loss)
iperf3 -c 10.0.1.20 -u -b 100M -t 30

# ─── CHECK INTERFACE ERRORS ────────────────────────────────────────────
# Look at 'errors', 'dropped', 'overruns' counters
ip -s link show eth0

DNS Debugging — Why Your App Connects but Resolves to Nothing

DNS looks simple. It isn't. The worst outages I've seen weren't packet loss or firewall rules — they were DNS caching a dead record for 24 hours. Before you blame the network, prove the name resolution is working in isolation.

Start with dig. It bypasses your system resolver and queries the authoritative nameserver directly. If dig google.com returns an IP but ping google.com fails, your local resolver is poisoning the cache. Flush it with resolvectl flush-caches on systemd systems or restart systemd-resolved.

nslookup is the old standby but lies about retries. Use dig +short for scripting. Check TTL values carefully — high TTLs hide failures until clients expire. The host command is great for quick reverse lookups.

For production: always test with dig @1.1.1.1 first. That bypasses your corporate DNS server and tells you if the problem is upstream or local.

// io.thecodeforge
# Test DNS resolution bypassing local cache
dig +short google.com @1.1.1.1

# Check authoritative nameserver and TTL
dig google.com NS +short

# Reverse lookup for a known IP
dig -x 8.8.8.8 +short

Firewall Forensics — iptables Is Dead, nftables Is What You Debug at 3 AM

The old iptables tool is deprecated. Every modern RHEL 9, Debian 12, and Ubuntu 24+ ship with nftables as the default firewall engine. If you still run iptables -L, you're looking at a compatibility layer that lies about actual rules.

Why this matters: I once spent four hours debugging why a Kubernetes node couldn't reach itself on port 8443. The culprit was an nftables set with a typo in the service CIDR. iptables -L showed nothing. nft list ruleset showed the exact broken line.

Learn nft list ruleset to dump all rules. Use nft add rule for temporary fixes during incidents. For persistent changes, edit /etc/nftables.conf and run nft -f.

Key difference: nftables uses tables, chains, and sets. Think of sets as efficient IP address groups. Debug with nft monitor to see packets hitting rules in real time — game changer for chasing phantom drops.

// io.thecodeforge
# Show all active nftables rules
nft list ruleset

# Monitor packets hitting rules in real-time
nft monitor | grep -E 'drop|reject'

# Add a temporary rule to log dropped packets
nft add rule inet filter input log prefix "DROPPED: " drop