Nmap Network Scanning: Host Discovery That Actually Works

# io.thecodeforge — System Design tutorial
# Scenario: Pre-security-review host discovery on an inherited VPC /24 subnet.
# Goal: Maximum coverage, minimum false negatives, logged output for the audit trail.
# Run as root or with sudo — raw socket access required for half-open scans.

# ------ PHASE 1: DISCOVERY WITHOUT PORT SCANNING ------
# -sn              : Skip port scan entirely — host discovery only.
# -PE              : Send ICMP echo requests (ping). Many hosts block this — we add it anyway.
# -PP              : Send ICMP timestamp requests. Gets past firewalls that block echo but allow timestamp.
# -PS443,8443      : TCP SYN to 443 and 8443. A RST reply still proves the host is alive.
# -PA80,8080       : TCP ACK to 80 and 8080. Stateless firewalls pass ACK; host sends RST = alive.
# -PU53            : UDP probe to port 53. Catches DNS servers that block all TCP probes.
# --min-parallelism: Run at least 100 probes in parallel — speeds up /24 significantly.
# --max-retries 2  : Don't retry dropped packets more than twice — keeps scan time sane.
# -oN              : Normal output to file — human-readable, also your audit artifact.
# -oG              : Greppable output — useful for piping into other tools downstream.
sudo nmap \
  -sn \
  -PE -PP \
  -PS443,8443,22,3389 \
  -PA80,8080,3306 \
  -PU53,161 \
  --min-parallelism 100 \
  --max-retries 2 \
  -oN /tmp/discovery_normal.txt \
  -oG /tmp/discovery_greppable.txt \
  10.0.1.0/24

# ------ PHASE 2: PULL LIVE HOSTS FROM GREPPABLE OUTPUT ------
# grep 'Up' extracts only confirmed-alive hosts from the greppable file.
# awk '{print $2}' isolates the IP address column.
# This list feeds directly into Phase 2 port scanning — no guesswork.
grep 'Up' /tmp/discovery_greppable.txt | awk '{print $2}' > /tmp/live_hosts.txt

echo "Confirmed live hosts:"
cat /tmp/live_hosts.txt

# ------ PHASE 3: TARGETED PORT + SERVICE SCAN ON LIVE HOSTS ONLY ------
# -iL              : Read targets from file — only scan confirmed-live hosts.
# -sS              : SYN scan (half-open). Faster, less likely to hit application logs.
# -sV              : Service version detection. Nmap sends probes and matches banner responses.
# --version-intensity 5: Balanced intensity (0-9). 9 is thorough but slow; 5 catches 95% of services.
# -O               : OS fingerprinting. Requires root. Uses TCP/IP stack behaviour to guess OS.
# --osscan-limit   : Only attempt OS detection on hosts with at least one open and one closed port.
#                    Without this flag, Nmap tries (and fails loudly) on filtered-only hosts.
# -p               : Scan common service ports. Adjust to your environment.
# -T3              : Normal timing. T4 is faster but causes packet loss on congested networks.
# --open           : Only show ports that are open — cuts output noise by 80%.
sudo nmap \
  -iL /tmp/live_hosts.txt \
  -sS \
  -sV --version-intensity 5 \
  -O --osscan-limit \
  -p 22,80,443,3306,5432,6379,8080,8443,9200,27017 \
  -T3 \
  --open \
  -oN /tmp/service_scan.txt \
  -oX /tmp/service_scan.xml

# XML output (-oX) feeds into tools like Nessus, Metasploit, or custom parsers.
# Keep both formats — the XML is for machines, the normal is for humans reading the audit report.

echo "Scan complete. Results in /tmp/service_scan.txt"

# io.thecodeforge — System Design tutorial
# Scenario: You've deployed new security group rules in a staging environment
# and need to verify exactly which ports are filtered vs closed vs open
# before promoting to production. This is not pentesting — this is change validation.

# ------ ACK SCAN: MAP FIREWALL RULES, NOT SERVICES ------
# -sA: ACK scan. Sends TCP ACK packets (not SYN).
#      Stateful firewalls drop ACK with no matching SYN in their table = FILTERED.
#      Stateless firewalls pass ACK through = host replies RST = UNFILTERED.
#      This tells you about firewall behaviour, NOT whether a service is running.
# Use case: validate that your firewall is blocking what it should be blocking.
sudo nmap \
  -sA \
  -p 22,80,443,3306,5432,6379,8080,9200 \
  --reason \
  10.0.1.5

# --reason: Shows WHY Nmap assigned each state — the actual packet received.
# Example output lines:
#   22/tcp   unfiltered  ssh  no-response (ACK passed, host replied RST)
#   3306/tcp filtered         no-response (firewall ate the packet)
# Without --reason you're guessing. With it you're debugging.

# ------ SYN SCAN WITH VERBOSE STATE REPORTING ------
# Now compare: what does a SYN scan show on the same host?
# -sS: SYN scan. The standard. Half-open — sends SYN, interprets SYN-ACK (open)
#      or RST (closed) or nothing/ICMP unreachable (filtered).
# -p-: Scan all 65535 ports. Expensive but complete.
#      Only do this on confirmed single targets, not ranges.
# --reason: Essential for debugging. 'syn-ack' = open service. 'rst' = closed port.
# -v: Verbose. Prints results as they arrive instead of waiting for full scan.
sudo nmap \
  -sS \
  -p 1-1024 \
  --reason \
  -v \
  10.0.1.5

# ------ INTERPRETING THE RESULTS IN CONTEXT ------
# After running both scans, compare:
# Port 3306:
#   ACK scan  -> filtered  (firewall is dropping)
#   SYN scan  -> filtered  (service unreachable)
#   Conclusion: MySQL is properly isolated. Security group is working.
#
# Port 9200 (Elasticsearch):
#   ACK scan  -> unfiltered  (ACK passes through)
#   SYN scan  -> open        (service is running and reachable)
#   Conclusion: Elasticsearch is PUBLICLY ACCESSIBLE. This is a P0 incident.
#   Fix: Restrict security group ingress on 9200 to internal CIDRs only.

# I have personally seen Elasticsearch on 9200 exposed to the internet in THREE
# separate production environments during audits. Every single time, the engineer
# thought security groups were applied. They weren't. Verify with Nmap. Always.

# io.thecodeforge — System Design tutorial
# Scenario: Weekly automated security sweep of internal microservices in a
# production VPC. Runs as a cron job. Must not cause application-level disruption.
# Must complete within a 20-minute window during low-traffic period.

# ------ TIMING TUNING FOR PRODUCTION NETWORKS ------
# -T3              : Normal timing. Max 1 probe per open port at a time.
#                    Safe for production. Does not flood network buffers.
# --min-rate 100   : Send at least 100 packets/second.
#                    Without this, T3 can be painfully slow on clean networks.
# --max-rate 300   : Hard cap at 300 packets/second.
#                    Prevents Nmap from ramping up and causing collateral damage.
# --scan-delay 10ms: Minimum 10ms between probes to the same host.
#                    Critical for rate-limited hosts (some cloud instances throttle
#                    at the hypervisor level and drop rapid repeated SYNs).
sudo nmap \
  -sS -sV \
  -T3 \
  --min-rate 100 \
  --max-rate 300 \
  --scan-delay 10ms \
  -p 22,80,443,3000,3306,5432,6379,8080,8443,9090,9200 \
  --open \
  -oX /var/log/security-sweep/$(date +%Y%m%d)_sweep.xml \
  10.0.0.0/20

# ------ IDS-AWARE SCANNING: FRAGMENTATION + DECOYS ------
# This block is for educational understanding — use it only in environments
# you own and only when explicitly testing IDS detection capability.

# -f: Fragment packets into 8-byte chunks.
#     Older IDS systems can't reassemble before matching signatures.
#     Suricata 6+, Snort 3+: fully reassembles before inspection. -f does nothing.
#     Legacy appliances from 2015 and earlier: might slip through.
# -D RND:10: Use 10 random decoy IP addresses alongside your real source IP.
#            IDS sees 11 sources scanning simultaneously — harder to attribute.
#            WARNING: The real source IP is still in the packet list. This is
#            obfuscation, not anonymisation. A careful analyst will find you.
# --source-port 53: Spoof source port as 53 (DNS).
#                   Old stateless firewalls trusted DNS source ports.
#                   Modern stateful firewalls: this does nothing useful.
# --data-length 25: Append 25 random bytes to packets.
#                   Breaks signature matching on payload-length-sensitive rules.
sudo nmap \
  -sS \
  -f \
  -D RND:10 \
  --source-port 53 \
  --data-length 25 \
  -T2 \
  -p 80,443,22 \
  10.0.1.100
# T2 with evasion because -f + decoys slow packet processing significantly.
# Running T4 with fragmentation causes your own OS to choke on reassembly.

# ------ SCRIPT ENGINE FOR SERVICE-SPECIFIC CHECKS ------
# NSE (Nmap Scripting Engine) runs Lua scripts against discovered services.
# These are NOT port scans — they're active probes that touch the application layer.
# Run these only on confirmed non-production hosts unless the scripts are
# explicitly safe (category: safe).

# Check for anonymous FTP access:
sudo nmap --script ftp-anon -p 21 10.0.1.0/24

# Check for HTTP security headers on all discovered web servers:
sudo nmap --script http-headers -p 80,443,8080,8443 10.0.1.0/24

# Enumerate SSL/TLS cipher suites — useful for compliance audits (PCI-DSS):
sudo nmap --script ssl-enum-ciphers -p 443,8443 10.0.1.5

# Check for open MongoDB with no auth (script from default category):
sudo nmap --script mongodb-info -p 27017 10.0.1.0/24

# NEVER run --script=all or --script=vuln against production.
# vuln category scripts actively exploit vulnerabilities. They will crash services.
# I have personally watched --script=vuln take down a Redis instance
# by triggering an unprotected FLUSHALL via the redis-info script chain.

# io.thecodeforge — System Design tutorial
# Scenario: Deciding between Nmap and lighter tools based on the actual question.
# This is the decision tree engineers skip and then wonder why their scan takes
# 10 minutes to tell them something nc would've said in 2 seconds.

# ------ QUESTION: Is a specific port open on a specific host? ------
# RIGHT TOOL: netcat. No root. No raw sockets. No delay.
# -z: Zero I/O mode — just check connectivity, don't send data.
# -v: Verbose — prints 'succeeded' or 'refused'.
# -w 3: Timeout after 3 seconds. Without this, nc hangs on filtered ports forever.
nc -zv -w 3 10.0.1.5 5432
# Output: Connection to 10.0.1.5 5432 port [tcp/postgresql] succeeded!
# If it hangs for 3s and exits: filtered. If 'refused': closed but host is up.

# ------ QUESTION: What version of a service is running on one host? ------
# RIGHT TOOL: curl or openssl for HTTP/HTTPS; nmap -sV only if banner not exposed.
curl -sI https://10.0.1.5 | grep -i 'server\|x-powered-by'
# Faster than nmap -sV for HTTP services. Gets you the same version banner.
# Nmap -sV makes sense when the service doesn't expose version in HTTP headers.

# ------ QUESTION: Who is the network talking to? ------
# RIGHT TOOL: ss (socket statistics). Shows active connections, not listening state.
# -t: TCP sockets only.
# -n: Don't resolve hostnames — faster, avoids DNS delays.
# -p: Show process using each socket.
# -a: All sockets (listening + connected).
ss -tnpa
# This tells you what IS communicating right now.
# Nmap tells you what COULD communicate. Different questions.

# ------ QUESTION: What's traversing the network between hosts? ------
# RIGHT TOOL: tcpdump. Captures actual traffic, not probed state.
# -i eth0: Listen on eth0 interface.
# -n: Don't resolve names — critical in production, DNS lookups slow capture.
# 'host 10.0.1.12 and port 3306': Filter to MySQL traffic from/to target host.
# -w: Write to pcap file for offline analysis with Wireshark.
tcpdump -i eth0 -n 'host 10.0.1.12 and port 3306' -w /tmp/mysql_traffic.pcap

# ------ WHEN NMAP IS ACTUALLY THE RIGHT CALL ------
# 1. You need to survey a /16 or larger and don't know what's there.
# 2. You need OS fingerprinting to understand the fleet's OS diversity.
# 3. You need to confirm firewall rules are working as designed (ACK scan).
# 4. You're doing a formal security audit and need documented, reproducible output.
# 5. You need NSE scripts to check for specific vulnerabilities (carefully).

# For anything else — reach for the lighter tool first.