OSPF Metric Blind Spot — Why 10Gbps and 100Mbps Look Equal
Default OSPF cost=1 on both 10Gbps and transatlantic 100Mbps caused >200ms latency.
20+ years shipping production systems from the metal up. Lessons pulled from things that broke in production.
- ✓Deep production experience
- ✓Understanding of internals and trade-offs
- ✓Experience debugging complex systems
- RIP uses Bellman-Ford for distance vector routing with hop count metric
- OSPF uses Dijkstra's SPF for link state routing with cost based on bandwidth
- BGP is a path vector protocol that exchanges reachability and policy attributes
- Convergence: RIP can take minutes (count-to-infinity), OSPF converges in under 10s
- Production insight: metric misconfiguration causes silent suboptimal routing
Routing protocols are the distributed consensus engines that determine how packets traverse networks, from a small office LAN to the global internet. They solve the fundamental problem of dynamic path selection: when a link or router fails, the network must automatically recalculate viable routes without human intervention.
Without them, every router would need static routes manually configured for every possible destination — an impossible task at internet scale. These protocols are essentially algorithms that allow routers to exchange reachability information and build a shared, consistent view of the network topology.
In the networking stack, routing protocols operate at the control plane, distinct from the data plane that actually forwards packets. They fall into two architectural camps: Interior Gateway Protocols (IGPs) like OSPF and RIP, which run within a single autonomous system (AS), and Exterior Gateway Protocols like BGP, which connect ASes.
The choice depends on scale and requirements — OSPF converges in seconds and handles hundreds of routers, while BGP handles hundreds of thousands of routes but prioritizes policy over raw speed. RIP, the oldest, is limited to 15 hops and is effectively obsolete outside of legacy or lab environments.
The critical blind spot this article addresses is OSPF's default metric behavior: it uses a reference bandwidth of 100 Mbps, meaning a 10 Gbps link and a 100 Mbps link both get a cost of 1. This breaks path selection on modern hardware, forcing engineers to manually adjust the reference bandwidth or use explicit cost overrides.
Understanding this quirk is essential for anyone designing or troubleshooting OSPF networks, as it directly impacts traffic engineering and convergence behavior in production environments.
Imagine your city has thousands of roads and you need to drive a package from New York to Los Angeles. A routing protocol is like a GPS system that every intersection uses to talk to its neighbours — each junction shares what roads it knows about, how congested they are, and updates its map when a road closes. The 'protocol' is just the agreed language all those intersections use to gossip with each other so every driver always takes the best available path.
Every time you load a webpage, your request hops across dozens of routers spanning continents, undersea cables, and data-centres owned by completely different companies. None of those routers were pre-programmed with a static map of the entire internet — that would be impossible to maintain. Instead, they run routing protocols: living, breathing algorithms that continuously discover the network topology, elect the best paths, and heal themselves when links go dark. Understanding this machinery isn't academic; it's what separates an engineer who can debug a production outage from one who just restarts the router and hopes.
The core problem routing protocols solve is dynamic reachability at scale. A static route you add manually works fine for a lab with five subnets. It collapses the moment a link fails or a new site comes online, because nothing automatically redistributes that knowledge. Routing protocols replace human intervention with distributed consensus — every router converges on the same view of the network without a central coordinator, and they do it in seconds or milliseconds depending on the protocol.
By the end of this article you'll understand exactly how Bellman-Ford powers RIP and why it causes count-to-infinity, how Dijkstra's SPF algorithm inside OSPF builds a loop-free topology, why BGP is a policy engine masquerading as a routing protocol, and how to reason about convergence time and route selection in production networks. You'll also walk away with concrete Python simulations you can run locally to watch these algorithms think.
Why Routing Protocols Are the Internet's Distributed Consensus Engine
A routing protocol is a distributed algorithm that lets routers discover and maintain paths across a network without a central controller. The core mechanic: routers exchange reachability information (prefixes) and path cost metrics, then each independently runs a shortest-path computation (e.g., Dijkstra for OSPF, Bellman-Ford for RIP) to build its forwarding table. The result is a loop-free, converged topology that adapts when links fail or costs change.
Key properties that matter in practice: convergence time (how fast all routers agree after a change), metric type (hop count vs. bandwidth-delay composite), and scalability (link-state protocols like OSPF handle hundreds of routers; distance-vector like EIGRP scales differently). OSPF uses cost = reference bandwidth / interface bandwidth, defaulting to 100 Mbps reference — meaning any link faster than 100 Mbps gets cost 1, making 10 Gbps and 100 Mbps indistinguishable.
Use routing protocols in any multi-hop IP network where manual static routes are impractical — data centers, enterprise WANs, ISP backbones. They matter because they automate failover, load-balance across equal-cost paths, and enforce policy (e.g., BGP communities). Without them, a single link failure would require human intervention to re-route traffic.
Where Routing Protocols Fit in Networking
Every router maintains a routing table — the list of every destination network it knows about and how to reach it. But routers don't build that table by magic. They rely on routing protocols to dynamically discover paths, adapt to failures, and distribute reachability across an autonomous system or the entire internet.
The distinction between routing protocols and routed protocols is crucial for production thinking. Routed protocols like IP carry user data. Routing protocols like OSPF, BGP, and RIP carry routing information between routers. If the routing protocol goes down, the IP traffic doesn't necessarily stop — the routers still have stale routes until they time out. That stale state is a common root cause of traffic blackholes.
# Show routing table on a Cisco router show ip route # Example output (abbreviated) C 192.168.1.0/24 is directly connected, GigabitEthernet0/0 O 10.0.0.0/8 [110/2] via 192.168.1.1, 00:00:34, GigabitEthernet0/0 B 172.16.0.0/12 [20/0] via 10.0.0.1, 00:10:22, Serial0/0/0 R 192.168.2.0/24 [120/3] via 192.168.1.2, 00:00:12, GigabitEthernet0/1
RIP — Distance Vector Routing and Its Production Limits
RIP (Routing Information Protocol) is the simplest routing protocol. Every router sends its entire routing table to neighbours every 30 seconds. Each route carries a hop count — a metric that counts how many routers you must cross to reach the destination. The maximum is 15 hops; 16 means unreachable.
The algorithm behind RIP is Bellman-Ford — a distributed version where each router updates its table based on received advertisements. RIP converges slowly because of the count-to-infinity problem: when a link fails, the announcement takes time to propagate, and routers may temporarily believe a path exists through a router that just lost that route, incrementing the hop count each time. This slowly increases until it hits 16.
In production, RIP is almost never used today. The 15-hop limit is too restrictive for any network with more than a few routers. Convergence can take minutes — unacceptable for modern applications. However, RIP's simplicity makes it an excellent teaching tool. Understanding its flaws explains why OSPF and BGP exist.
# io.thecodeforge.routing.bellman_ford import itertools def rip_simulate(routers, initial_links, fail_link=None): """Simplified RIP-like Bellman-Ford simulation with count-to-infinity.""" INF = 16 # Initialize distances: self=0, direct neighbours=1, else INF dist = {r: {t: (1 if t in initial_links.get(r, []) else 0 if r == t else INF) for t in routers} for r in routers} for iteration in range(5): # simulate 5 update cycles new_dist = {r: d.copy() for r, d in dist.items()} for r in routers: for neighbor in initial_links.get(r, []): if fail_link and (r, neighbor) in [fail_link, (fail_link[1], fail_link[0])]: continue # link is down for dest in routers: via_neighbor = dist[neighbor][dest] + 1 if via_neighbor < new_dist[r][dest] and via_neighbor <= 15: new_dist[r][dest] = via_neighbor dist = new_dist print(f"After iteration {iteration+1}: Router A to D = {dist['A']['D']}") return dist # Test: 4 routers A-B-C-D routers = ['A', 'B', 'C', 'D'] links = {'A': ['B'], 'B': ['A', 'C'], 'C': ['B', 'D'], 'D': ['C']} print("Initial distances (A->D):") result = rip_simulate(routers, links) print("After iteration 5:", result['A']['D'])
- When a link fails, the router that discovers it sets the metric to 16 and announces it.
- Neighbors may have a better path? No — they might have learned that route from the failed router, so they still think it's reachable.
- The failed router hears the neighbor's advertisement and thinks there's a path through them, so it updates its own metric to (neighbor's metric + 1).
- This cycle repeats, each time incrementing the metric, until it reaches 16 and is finally considered unreachable.
- Protocols use split-horizon and route poisoning to mitigate this, but poison reverse only helps for directly connected routers.
OSPF — Link State Routing with Fast Convergence
OSPF (Open Shortest Path First) is a link-state routing protocol. Instead of exchanging routing tables, OSPF routers flood Link State Advertisements (LSAs) to all routers in the same area. Every router builds an identical Link State Database (LSDB) of the entire network topology. Then each router runs Dijkstra's Shortest Path First (SPF) algorithm on this database to compute the shortest path tree to every destination.
OSPF uses a metric called cost, which defaults to reference_bandwidth / interface_bandwidth. This makes path selection sensitive to bandwidth — a 1Gbps link gets cost=1 (if reference is 100Gbps), a 100Mbps link gets cost=1 as well. That's why you must adjust the reference bandwidth to match your fastest links.
OSPF converges in seconds because each router independently calculates paths from the consistent LSDB — no hop-by-hop propagation delay. Link failures trigger immediate LSA floods, and the SPF tree is recalculated. However, SPF recalculations can be CPU-intensive in large topologies; OSPF mitigates this with areas (hierarchical design) and incremental SPF (iSPF).
# io.thecodeforge.routing.ospf.dijkstra import heapq def dijkstra_spf(graph, start): """Standard Dijkstra's algorithm returns shortest distances from start.""" distances = {node: float('inf') for node in graph} distances[start] = 0 priority_queue = [(0, start)] visited = set() while priority_queue: current_dist, current_node = heapq.heappop(priority_queue) if current_node in visited: continue visited.add(current_node) for neighbor, cost in graph[current_node].items(): new_dist = current_dist + cost if new_dist < distances[neighbor]: distances[neighbor] = new_dist heapq.heappush(priority_queue, (new_dist, neighbor)) return distances # Example topology: routers with OSPF costs network_graph = { 'Router1': {'Router2': 10, 'Router3': 5}, 'Router2': {'Router1': 10, 'Router4': 20}, 'Router3': {'Router1': 5, 'Router4': 15}, 'Router4': {'Router2': 20, 'Router3': 15} } spf_result = dijkstra_spf(network_graph, 'Router1') print("Shortest path costs from Router1:") for dest, cost in spf_result.items(): print(f" {dest}: cost = {cost}")
BGP — The Internet's Policy Engine
BGP (Border Gateway Protocol) is not a typical routing protocol. It doesn't find the shortest path based on bandwidth or hops. Instead, it exchanges reachability information and applies policy. BGP is a path vector protocol: each route advertisement carries the entire AS_PATH — the list of autonomous systems the route has traversed. This prevents loops (an AS will reject its own AS in the path).
BGP's primary metric is not latency or bandwidth but administrative policy. Network operators use BGP attributes like Local Preference (LOCAL_PREF), Multi-Exit Discriminator (MED), AS_PATH length, and community tags to influence inbound and outbound traffic. BGP decision process has 10+ tie-breaking steps, from highest LOCAL_PREF to lowest IGP metric to the router ID.
BGP convergence is complex — especially after a failure that creates a withdrawn route. The path exploration problem can cause significant delays (minutes) as BGP speakers try alternative paths before giving up. Techniques like BGP prefix independent convergence (PIC) and BGP add-path mitigate this.
# Check BGP table entry for a prefix on a Cisco router show ip bgp 192.0.2.0/24 # Sample output (abbreviated) BGP routing table entry for 192.0.2.0/24, version 12345 Paths: (2 available, best #2) Path #1: (metric 100) via 203.0.113.1 AS_PATH: 65001 65002 65003 Local preference: 100 MED: 50 Path #2: (metric 10) via 198.51.100.1 AS_PATH: 65001 65004 65003 Local preference: 200 MED: 20 # Best path is Path #2 due to higher local-preference.
- A route is like a treaty: 'I will send traffic to network X through AS Y because we have a peering agreement.'
- Local Preference is your internal policy: 'I prefer routes learned from my transit provider over my backup.'
- AS_PATH is a trust measure: shorter path = less intermediation, but longer path may be more reliable.
- MED is a suggestion: 'I'd prefer you enter my AS through this specific border router.'
- Communities are tags that convey intent across administrative boundaries.
Convergence and Performance: Engineering for Fast Recovery
Convergence is the time it takes for all routers to agree on a consistent view of the network after a change. The required convergence time depends on the application: voice traffic can tolerate sub-second outage, email can handle seconds, but financial trading systems require sub-50ms recovery.
RIP converges in tens of seconds to minutes due to hold-down timers and count-to-infinity. OSPF converges in 5-10 seconds with default timers; with BFD this drops to <1 second. BGP convergence is trickier: after a route withdrawal, BGP may explore alternative paths for up to minutes (path exploration).
- BFD (Bidirectional Forwarding Detection): sub-second link failure detection independent of routing protocol.
- BGP PIC (Prefix Independent Convergence): pre-compute backup paths.
- LFA (Loop-Free Alternate) in OSPF/IS-IS: install a backup next-hop to avoid waiting for SPF.
- Graceful Restart: allows router to continue forwarding while restarting, if neighbor cooperates.
The trade-off: faster convergence often means more state, more CPU or memory. BFD adds packet overhead. LFA doubles the FIB table size. Choose based on your failure rate and tolerance.
# Simplified convergence test using ping and cron # Step 1: On a monitoring server, ping the far end IP continuously ping -i 0.1 -c 500 far-end-router > /tmp/ping.log & # Step 2: On the upstream router, shut down the primary link conf t interface GigabitEthernet0/0 shutdown # Step 3: Check ping.log for the longest gap between responses # Count lost packets, divide by 10 (ping interval 100ms) to estimate convergence time. # For OSPF with default timers, expect around 5-10 seconds of loss. # With BFD, loss should be under 1 second. # Step 4: Restore the link, and repeat for BGP or RIP.
Routing Table Decoder: Protocol Codes You'll See in a Crash
When you type show ip route on a Cisco router after a BGP flap, the output isn't just noise. Each prefix has a letter code that tells you how it was learned. That code determines trust, path selection, and failover behavior. L means local — the router's own interface IP. C means directly connected — layer-2 adjacency. S is static, set by a human (or automation) with administrative distance of 1. O is OSPF, R is RIP, D is EIGRP, and B is BGP. Codes like IA (OSPF inter-area) or EX (EIGRP external) tell you the route was redistributed — a common source of suboptimal routing. Memorize these. When a route flips from O to E2 or B to D, you're watching a convergence event in real time. The code is your first diagnostic clue.
// io.thecodeforge # Simulate a routing table dump during a BGP maintenance window cat << EOF Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route Gateway of last resort is 203.0.113.1 to network 0.0.0.0 O 10.10.0.0/16 [110/2] via 192.168.1.1, 00:12:34, GigabitEthernet0/0 B 172.16.0.0/16 [20/0] via 203.0.113.2, 00:05:22, GigabitEthernet0/1 S 10.0.0.0/8 [1/0] via 10.0.0.1 EOF
show ip route tells you the route's origin, trust level, and how it will behave during convergence — never ignore it.Static vs. Dynamic vs. Default: When to Use Each at Scale
Static routing is for the paranoid. You hardcode every path. Zero overhead. Zero adaptability. At 10 routers, it's manageable. At 100, it's a career-limiting mistake. One typo and you blackhole traffic. Dynamic routing is the default for any network with redundancy. Protocols like OSPF and BGP auto-discover neighbors and converge around failures. The cost is CPU cycles and convergence time measured in seconds. Default routing is your escape hatch. It's the "I don't know where this packet is going, ship it to the gateway" rule. Production networks need all three. Use static for critical loopbacks and management interfaces. Use dynamic for internal transit links. Use a default route pointed at your WAN edge. Never default-route everything into a datacenter unless you enjoy congestion collapse.
// io.thecodeforge // Simulates routing decision logic based on type public class RoutingDecision { private static final int STATIC_PREFERENCE = 1; private static final int DYNAMIC_PREFERENCE = 20; private static final int DEFAULT_PREFERENCE = 200; public static Route selectBestRoute(List<Route> routes, IpPrefix destination) { // Production note: prefer most specific prefix first, then lowest preference return routes.stream() .filter(r -> r.matches(destination)) .min(Comparator.comparingInt(Route::getPreference) .thenComparing(Route::getPrefixLength)) // longest prefix match .orElse(null); } public static void main(String[] args) { List<Route> table = List.of( new Route("10.0.0.0/8", "203.0.113.1", STATIC_PREFERENCE), new Route("10.0.0.0/16", "192.168.1.1", DYNAMIC_PREFERENCE), new Route("0.0.0.0/0", "198.51.100.1", DEFAULT_PREFERENCE) ); Route best = selectBestRoute(table, new IpPrefix("10.0.1.0/24")); System.out.println("Selected next hop: " + best.getNextHop()); } } class Route { private final String prefix; private final String nextHop; private final int preference; // constructor, getters, matches() omitted for brevity }
OSPF Metric Misconfiguration Caused East-West Traffic to Traverse a Transatlantic Link
- Never assume default OSPF costs reflect real path quality.
- Always tune the reference bandwidth to match your fastest links.
- Use administrative distance or route-maps when simple cost isn't enough.
- Monitor traffic flows after any routing configuration change — a silent diversion can be worse than a visible outage.
show running-config | section router ospf/rip/bgp — to see what was appliedshow ip route — to check if default routes are missingshow ip route x.x.x.x — check for multiple next hops pointing back to itselftraceroute x.x.x.x — see where TTL expiresshow ip bgp neighbors x.x.x.x — look for state and last errorping x.x.x.x source <loopback> — verify path MTU| Feature | RIP | OSPF | BGP |
|---|---|---|---|
| Type | Distance Vector | Link State | Path Vector |
| Metric | Hop Count (max 15) | Cost (bandwidth-based) | Policy (AS_PATH, Local Pref, MED) |
| Algorithm | Bellman-Ford | Dijkstra (SPF) | Best Path Selection (10 steps) |
| Convergence | Minutes (count-to-infinity) | Seconds (SPF recalc) | Seconds to minutes (path exploration) |
| Scalability | Small (<15 hops) | Medium (areas for scale) | Internet-scale (global tables) |
| Loop Prevention | Split horizon, poison reverse | SPF guarantees loop-free tree | AS_PATH loop detection |
| Hierarchy | Flat only | Areas (backbone, normal, stub) | ASes, confederations, reflectors |
| Use Case | Legacy/Lab | Enterprise IGP | Inter-AS / Internet |
| Production Risk | Slow convergence, bandwidth waste | LSDB explosion, CPU spikes | Misconfiguration leaks, convergence delay |
| File | Command / Code | Purpose |
|---|---|---|
| show-route.sh | show ip route | Where Routing Protocols Fit in Networking |
| bellman_ford_simulation.py | def rip_simulate(routers, initial_links, fail_link=None): | RIP |
| dijkstra_ospf_simulation.py | def dijkstra_spf(graph, start): | OSPF |
| bgp-path-selection.sh | show ip bgp 192.0.2.0/24 | BGP |
| measure-convergence.sh | ping -i 0.1 -c 500 far-end-router > /tmp/ping.log & | Convergence and Performance |
| decode_routing_table.sh | cat << EOF | Routing Table Decoder |
| RoutingDecision.java | public class RoutingDecision { | Static vs. Dynamic vs. Default |
Key takeaways
Common mistakes to avoid
5 patternsAssuming OSPF cost default works for all link speeds
Not filtering BGP routes inbound and outbound
Using RIP in production with default timers
Forgetting to configure OSPF network type on point-to-point links
Mismatching BGP timers or update source interface
Interview Questions on This Topic
Explain the difference between distance vector and link state routing protocols. Give one advantage and one disadvantage of each.
Describe the BGP best path selection process. How does LOCAL_PREF influence inbound traffic?
How would you debug a routing loop in an OSPF environment?
Frequently Asked Questions
A routing protocol is a set of rules that routers use to exchange reachability information and build routing tables automatically. Without one, you must manually add static routes for every destination—impractical beyond a few subnets. Routing protocols enable dynamic adaptation to failures and traffic changes.
OSPF is an Interior Gateway Protocol (IGP) used within a single autonomous system, designed for fast convergence and link-state awareness. BGP is an Exterior Gateway Protocol (EGP) used between autonomous systems, focused on policy control and scalability. They serve different roles: OSPF inside your network, BGP to the internet.
Yes, many modern data centres use eBGP as the sole routing protocol (RFC 7938). BGP provides greater flexibility for load balancing, additive paths, and policy control. However, BGP convergence after a failure can be slower without enhancements like BGP PIC. OSPF remains simpler for typical enterprise campuses.
Consider these factors: network size (RIP for tiny labs, OSPF for <500 routers interior, BGP for external connections), required convergence speed (OSPF with BFD for sub-second), need for policy control (BGP only), and team expertise. Most enterprises use OSPF as IGP and BGP for internet edge.
20+ years shipping production systems from the metal up. Lessons pulled from things that broke in production.
That's Computer Networks. Mark it forged?
5 min read · try the examples if you haven't