Node ETIMEDOUT? Diagnose Network Timeouts Fast

Checkout calls to your payment API start failing with ETIMEDOUT. Not all of them — 8% at peak, zero at night. Retries sometimes help, sometimes stack into a bigger pile. The firewall team says nothing changed. Your code hasn't changed either, but the error tracker shows a steady climb that started Tuesday. Welcome to timeout debugging, where the bug lives in the space between two computers.

Timeout errors mislead because they describe a symptom (no answer in time) rather than a cause (firewall drop, saturated pool, DNS stall, slow upstream). Developers infinitely tune the timeout value — 5 seconds, 30 seconds, 60 — without asking why packets go unanswered. Longer timeouts just convert fast failures into slow resource leaks.

This guide gives you the split that matters: ETIMEDOUT versus ECONNREFUSED versus ENOTFOUND, each with its confirm command. You'll learn keep-alive tuning that stops handshake pileups, and retry logic with backoff plus AbortSignal that absorbs blips without amplifying outages. Numbers included throughout.

Timeout vs Refused vs DNS: Read err.code First

Three codes, three owners, three runbooks. ETIMEDOUT means packets went out and nothing came back — suspect firewalls, egress rules, or an overloaded upstream. ECONNREFUSED means the host answered and rejected the port — suspect wrong port, stopped service, or blocked ingress. ENOTFOUND and EAI_AGAIN mean the hostname never resolved — suspect DNS config, VPC resolver, or a typo'd host. Branching on err.code routes each failure to the team that can fix it instead of dumping every network error on the app developers.

Make the split mechanical in code and in triage. Log code, syscall, address, and port on every network failure — address and port distinguish a bad config from a bad network. In triage, reproduce each layer independently: dig for DNS, nc or curl for TCP, then your Node client last. When the first two pass and Node fails, the bug is in your agent or timeout config. When dig fails, no timeout value on earth matters.

Log shape determines triage speed more than any dashboard. Emit one structured line per network failure with code, syscall, host, port, resolved IP, elapsed milliseconds, attempt number, and which deadline fired (connect, response, or total). With those fields, a single log query separates firewall drops (connect deadline, full duration, one IP) from upstream slowness (response deadline, variable durations) from DNS stalls (failure before connect, no IP). Alert on rates per code rather than totals: a spike in ETIMEDOUT pages networking, ECONNREFUSED pages the service owner, EAI_AGAIN pages platform DNS — no more all-hands for every blip. Retain the raw fields for 30 days; post-mortems routinely need the resolved IP from three weeks ago to prove a vendor migrated ranges. Good logs make the err.code split automatic instead of tribal knowledge.

async function classifyFetch(url) {
  try {
    const res = await fetch(url, { signal: AbortSignal.timeout(8000) });
    return { ok: res.ok, status: res.status };
  } catch (err) {
    const code = err.cause?.code ?? err.code ?? err.name;
    if (code === 'ETIMEDOUT' || code === 'TimeoutError') return { layer: 'network-drop' };
    if (code === 'ECONNREFUSED') return { layer: 'nothing-listening' };
    if (code === 'ENOTFOUND' || code === 'EAI_AGAIN') return { layer: 'dns' };
    return { layer: 'unknown', code };
  }
}

Firewalls and Egress: The Silent Packet Droppers

Firewalls that drop packets (rather than rejecting them) are invisible by design — your client waits the full timeout with zero feedback. Cloud security groups, Kubernetes NetworkPolicies, and corporate proxies all default to drop. The signature is unmistakable once you know it: instant success from an unconstrained network, full-timeout failure from production, and a vendor dashboard insisting everything is green.

Confirm from the production host itself, never from your laptop. curl with a short timeout plus dig separates DNS from transit. Compare security-group egress rules against the vendor's published IP ranges, and check whether the vendor migrated ranges recently — allowlisted IPs are a standing invitation for this outage. Log the resolved remote IP on timeouts so the next migration announces itself in your dashboards before revenue notices.

Vendor IP migrations are the recurring villain, so build defenses assuming the next one is unannounced. Subscribe to vendor status and network-change feeds, and maintain allowlists from their published ranges via infrastructure-as-code so diffs review like code. Better yet, prefer hostname-based egress (NAT gateway domain allowlists or service-mesh EgressPolicies) where your platform supports it — IPs churn, hostnames persist. Keep a runbook entry per critical vendor with its current ranges, the last-change date, and the synthetic probe URL, so the 2 AM responder isn't discovering the dependency map mid-incident. Test egress changes in staging with identical rules before promoting; security-group edits deserve the same pipeline as application code. When the next silent migration lands, your probe pages in two minutes and the fix is a reviewed range update, not a four-hour revenue investigation.

curl -m 10 -o /dev/null -w '%{http_code} in %{time_total}s from %{remote_ip}\n' https://api.vendor.com/health
time dig +short api.vendor.com
aws ec2 describe-security-groups --group-ids sg-0abc123 --query 'SecurityGroups[0].IpPermissionsEgress'

Keep-Alive Agents: Stop Paying Handshake Tax

Without keep-alive, every HTTPS request pays a full TCP handshake plus TLS negotiation — 3-4 round trips before the first byte. At 500 requests per second that's 500 concurrent handshakes competing for ephemeral ports and CPU, and tail latency climbs exactly when traffic peaks. The symptom pattern is diagnostic: clean at night, timeouts at noon, with server CPU healthy on both sides.

A shared http.Agent with keepAlive:true reuses established connections, collapsing per-request cost to one round trip. Size maxSockets to peak concurrency plus headroom (100-200 for mid-size APIs), set keepAliveMsecs around 30 seconds to survive brief lulls, and give slow dependencies their own small agent so one sluggish host can't starve the pool. Verify with socket counters: TIME_WAIT should fall and p99 latency should drop visibly on the next peak.

Pool sizing deserves real math, not defaults. Count peak concurrent outbound requests per host (concurrency = throughput × latency: 500 rps × 0.2s = 100 sockets), add 50% headroom for bursts, and set maxSockets per host accordingly — then verify with socket counters during load tests, not production peaks. FreeSockets versus active sockets in agent.getName stats reveal whether the pool is reuse-healthy (mostly active, low churn) or thrashing (constant create-destroy). Tune keepAliveMsecs to outlast typical request gaps (30s covers most API rhythms) but stay under upstream idle timeouts — a load balancer culling at 60s turns longer keep-alives into ECONNRESET surprises. Disable Nagle (noDelay) for latency-sensitive RPC, and set scheduling priorities so bulk exports can't starve interactive checkouts sharing the pool. Measure p50, p99, and TIME_WAIT before and after; keep-alive done right shows up in all three.

const http = require('http');
const https = require('https');

const fastAgent = new https.Agent({
  keepAlive: true,
  keepAliveMsecs: 30000,
  maxSockets: 100,
  timeout: 8000,
});

const slowVendorAgent = new https.Agent({
  keepAlive: true,
  maxSockets: 10,
  timeout: 8000,
});

module.exports = { fastAgent, slowVendorAgent };

Retry With Backoff, Not With Hope

Transient blips deserve retries; persistent outages deserve fast failure. The discipline is threefold: retry only transient signals (ETIMEDOUT, ECONNRESET, 503, 429), space attempts with exponential backoff plus jitter so clients don't thundering-herd the recovering server, and cap the total budget under the caller's patience. A retry policy without a budget just moves the queue from the vendor to your event loop.

Bound every attempt with AbortSignal.timeout so a hung socket can't consume the whole budget. Add jitter (randomize each delay by 20-50%) to desynchronize fleet-wide retries. Count attempts in logs and export retry-rate metrics — a rising retry rate is the earliest warning of upstream trouble, visible minutes before timeouts breach SLO. When retries exhaust, return 502 with Retry-After rather than hanging: a fast honest failure beats a slow ambiguous one.

Idempotency keys make retries safe, so design them per operation. Derive keys deterministically from business identity — order ID plus attempt scope for payments, event ID for webhooks, request hash for reads — so replays collapse onto the original instead of duplicating side effects. Send keys on every mutating call, not just payments: inventory decrements, email sends, and provisioning calls all double-fire under retry without them. Store key-to-result mappings server-side with TTLs covering your maximum retry window, and return the stored result on key replay with a replayed: true flag for observability. Test the path explicitly: simulate a timeout-after-commit in staging (proxy that drops the response but forwards the request) and confirm the retry returns the original without duplication. Without this test, idempotency is a header you hope works rather than a guarantee you've proven.

const TRANSIENT = new Set(['ETIMEDOUT', 'ECONNRESET', 'EAI_AGAIN']);

async function fetchWithRetry(url, { attempts = 3, baseMs = 400 } = {}) {
  for (let i = 1; i <= attempts; i++) {
    try {
      return await fetch(url, { signal: AbortSignal.timeout(8000) });
    } catch (err) {
      const code = err.cause?.code ?? err.code;
      if (!TRANSIENT.has(code) || i === attempts) throw err;
      const wait = baseMs * 2 ** (i - 1) * (0.8 + Math.random() * 0.4);
      await new Promise((r) => setTimeout(r, wait));
    }
  }
}

AbortSignal.timeout: Bound Every Wait

A request without a timeout is a resource leak with optimistic branding. Hung sockets pin pool slots, accumulate memory, and cascade one slow dependency into every route sharing the pool. Node's AbortSignal.timeout(8000) gives fetch a clean per-attempt deadline; for raw http requests, set request.setTimeout plus socket timeouts. The value should reflect user patience (5-10 seconds for interactive paths) rather than vendor promises.

Keep three timeouts distinct: connect timeout (can we reach them), response timeout (do they answer), and total budget (how long do we keep trying). Conflating them produces 30-second hangs on paths users abandon in 3. Log which deadline fired — connect timeouts implicate the network, response timeouts implicate the upstream — and size pools so a burst of timeouts degrades one route, not the server.

Layer timeouts like defense in depth. Set a connect timeout (2-3s: can we reach them), a response timeout per attempt (5-8s: do they answer), and a total budget across retries (under the caller's patience — 15s for interactive paths). Each layer implicates a different owner when it fires, and the total budget is what prevents retry logic from converting a vendor outage into your own. In Node's http module, setTimeout on the request plus destroy on timeout approximates what fetch gets for free with AbortSignal; for streams, abort the stream and release the socket or the pool leaks exactly as if no timeout existed. Propagate deadlines downstream: pass the remaining budget to chained calls via headers so depth-three fan-outs can't each spend the full allowance. Review timeout values quarterly against actual latency percentiles — budgets set during an outage tend to stay generous forever.

node -e "fetch('https://httpbin.org/delay/10', { signal: AbortSignal.timeout(3000) }).then(r => console.log(r.status)).catch(e => console.error('deadline fired:', e.name))"
node -e "console.log(typeof AbortSignal.timeout === 'function' ? 'abort-timeout ok' : 'upgrade node')"

DNS: The Timeout Impostor

Slow or broken DNS looks exactly like a connection timeout when you only watch total duration. Resolution happens before any packet leaves, so a 5-second resolver stall plus a fast connection still reports as a timeout against your 8-second budget. In containers the usual culprits are missing VPC DNS, ndots misconfiguration causing search-domain storms, and resolver IPs that answer intermittently.

Separate DNS from TCP explicitly: time dig independently, log resolution duration in your client, and cache successful lookups with a TTL-aware cache. Prefer all:false plus family hints to avoid Happy Eyeballs delays on misconfigured IPv6. When EAI_AGAIN clusters across services simultaneously, page platform DNS — no application retry policy fixes a resolver outage, and fast failure preserves your pools for recovery.

Resolver hygiene prevents the impostor class entirely. Run a local caching resolver (Node's built-in cache with TTL, or a sidecar like dnsmasq) so transient upstream blips don't stall every new connection. Audit ndots and search domains in container resolv.conf — ndots:5 with three search domains turns one external lookup into a storm of doomed queries, each adding latency before the real resolution even starts. Pin critical dependencies to dual-stack-aware lookup (verbatim ordering, explicit family) to avoid Happy Eyeballs delays on half-broken IPv6. Monitor DNS resolution time as its own metric with alerts at p99 thresholds; when it degrades fleet-wide, the dashboard should say DNS before anyone blames application timeouts. And keep a static-host fallback for the two dependencies that would halt revenue — written by hand, reviewed quarterly, and worth every line during a resolver outage.

time dig +short api.vendor.com
cat /etc/resolv.conf
node -e "const t=Date.now(); require('dns').lookup('api.vendor.com', e => console.log(e && e.code, Date.now() - t + 'ms'))"