Circuit Breaker Pattern — Timeouts Alone Kill Thread Pools

Distributed systems fail in ways that monoliths never do. A single slow database call can hold a thread. A hundred slow calls can hold a thread pool. At that point your entire service — which is otherwise perfectly healthy — is completely unavailable, brought down not by its own bugs but by something it was talking to. This is called cascading failure, and it's responsible for some of the most spectacular production outages in the industry.

The Circuit Breaker pattern exists to break that cascade. Instead of letting your service hammer a failing dependency indefinitely, it interposes a state machine between your code and the remote call. When failures breach a threshold, the breaker opens and subsequent calls fail fast — immediately, without touching the network — giving the downstream system breathing room to recover and protecting your own thread pool from exhaustion.

By the end of this article you'll understand exactly how the three-state machine works under the hood, how to tune failure thresholds and timeout windows without guessing, how to implement a production-grade breaker in Java from scratch, and the real-world gotchas that bite teams even when they think they've set it up correctly. We'll also compare the two dominant counting strategies — count-based and time-based sliding windows — so you can choose the right one for your traffic pattern.

What Is the Circuit Breaker Pattern?

The Circuit Breaker pattern is a state machine that monitors remote calls and opens when failures exceed a threshold. Its primary job: fail fast when a dependency is unhealthy, not slow — and give that dependency time to recover without being flooded with requests.

Think of it as a safety valve. In a closed state, all requests pass through normally. Each failure increments a counter. When the counter hits the configured threshold, the breaker trips to open, and subsequent requests are rejected immediately with an exception. After a recovery timeout, the breaker transitions to half-open, allowing a limited number of probe requests. If these succeed, the breaker closes again. If they fail, it reopens.

The pattern decouples error handling from business logic. You don't have to write try-catch blocks in every method that calls an external service. Instead, the circuit breaker centralises failure detection and recovery.

package io.thecodeforge.circuitbreaker;

public enum CircuitBreakerState {
    CLOSED,
    OPEN,
    HALF_OPEN
}

public class CircuitBreaker {
    private final int failureThreshold;
    private final long recoveryTimeoutMs;
    private CircuitBreakerState state = CircuitBreakerState.CLOSED;
    private int failureCount = 0;
    private Instant lastFailureTime;

    public CircuitBreaker(int failureThreshold, long recoveryTimeoutMs) {
        this.failureThreshold = failureThreshold;
        this.recoveryTimeoutMs = recoveryTimeoutMs;
    }

    public synchronized boolean isRequestAllowed() {
        if (state == CircuitBreakerState.CLOSED) {
            return true;
        }
        if (state == CircuitBreakerState.OPEN) {
            if (Duration.between(lastFailureTime, Instant.now()).toMillis() >= recoveryTimeoutMs) {
                state = CircuitBreakerState.HALF_OPEN;
                return true;
            }
            return false;
        }
        // half-open: allow exactly one probe (simplified)
        if (state == CircuitBreakerState.HALF_OPEN) {
            // In reality, track probe count
            return true;
        }
        return false;
    }

    public synchronized void recordFailure() {
        failureCount++;
        lastFailureTime = Instant.now();
        if (failureCount >= failureThreshold) {
            state = CircuitBreakerState.OPEN;
        }
    }

    public synchronized void recordSuccess() {
        if (state == CircuitBreakerState.HALF_OPEN) {
            state = CircuitBreakerState.CLOSED;
            failureCount = 0;
        }
    }

    public CircuitBreakerState getState() { return state; }
}

The Three States and Their Transitions

The circuit breaker operates in three distinct states:

CLOSED — Normal operation. All requests pass through. Each failure increments an internal counter. When the counter reaches the threshold, the breaker transitions to OPEN. In a count-based window, failures are counted within a fixed number of requests (e.g., 5 failures out of the last 10 requests). In time-based windows, failures are counted within a time window (e.g., 5 failures in the last 10 seconds).

OPEN — Requests are rejected immediately without calling the downstream service. The breaker remains open for a configurable recovery timeout. After this timeout, it transitions to HALF_OPEN.

HALF_OPEN — A limited number of probe requests are allowed through. If a probe succeeds, the breaker transitions back to CLOSED (and resets the failure count). If the probe fails, the breaker returns to OPEN and resets the recovery timeout. The number of probes and the success threshold are configurable.

The transition from HALF_OPEN to CLOSED should require a minimum number of consecutive successes (e.g., 3) to prevent flaps. A single success is not enough — one probe could succeed by luck while the downstream is still degraded.

package io.thecodeforge.circuitbreaker;

public class StateMachineTransition {
    public enum Transition {
        CLOSED_TO_OPEN,
        OPEN_TO_HALF_OPEN,
        HALF_OPEN_TO_CLOSED,
        HALF_OPEN_TO_OPEN
    }

    public Transition evaluate(CircuitBreakerState current, int failureCount, int threshold, long elapsedSinceLastFailure, long timeout) {
        switch (current) {
            case CLOSED:
                if (failureCount >= threshold) return Transition.CLOSED_TO_OPEN;
                break;
            case OPEN:
                if (elapsedSinceLastFailure >= timeout) return Transition.OPEN_TO_HALF_OPEN;
                break;
            case HALF_OPEN:
                // simplified: after one probe, decide based on success/failure
                // in production track probe results
                if (failureCount == 0) return Transition.HALF_OPEN_TO_CLOSED;
                else return Transition.HALF_OPEN_TO_OPEN;
        }
        throw new IllegalStateException("Unhandled state: " + current);
    }
}

Implementing a Circuit Breaker in Java: Production-Grade Approach

Building a circuit breaker from scratch is educational, but for production you should use a battle-tested library. Two popular choices in Java: Resilience4j and Spring Cloud Circuit Breaker. The following example uses Resilience4j, which provides sliding window counters, thread pool isolation, and event listeners.

Resilience4j's circuit breaker supports two counting strategies: - count-based: failures in the last N calls (e.g., last 10 calls) - time-based: failures within a time window (e.g., last 10 seconds)

Each strategy has its own internal sliding window implementation. The count-based strategy uses a circular buffer of size N, while the time-based strategy uses a sliding timestamp list. Both are efficient — O(1) for recording calls — but consume memory proportional to the window size.

package io.thecodeforge.circuitbreaker;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;
import java.time.Duration;
import java.util.function.Supplier;

public class PaymentServiceWithBreaker {

    private final CircuitBreaker circuitBreaker;
    public PaymentServiceWithBreaker() {
        CircuitBreakerConfig config = CircuitBreakerConfig.custom()
                .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
                .slidingWindowSize(10)
                .minimumNumberOfCalls(5)
                .failureRateThreshold(50)  // 50% failures -> open
                .recordExceptions(TimeoutException.class, IOException.class)
                .waitDurationInOpenState(Duration.ofSeconds(30))
                .permittedNumberOfCallsInHalfOpenState(3)
                .build();

        this.circuitBreaker = CircuitBreakerRegistry.ofDefaults()
                .circuitBreaker("paymentService", config);
    }

    public PaymentResult processPayment(PaymentRequest request) {
        Supplier<PaymentResult> decorated = CircuitBreaker.decorateSupplier(
            circuitBreaker, () -> callPaymentGateway(request));
        return decorated.get();
    }

    private PaymentResult callPaymentGateway(PaymentRequest request) {
        // actual HTTP call
        return paymentClient.charge(request);
    }
}

Count-Based vs Time-Based Sliding Windows: The Right Strategy for Your Traffic

The sliding window strategy determines how failures are aggregated. Count-based windows consider the last N requests. Time-based windows consider all requests within the last T duration. Both have trade-offs that matter in production.

Count-based is simple: keep a circular buffer of the last N call results. Each new call overwrites the oldest. Failure rate = failures / N. Works well when request rate is roughly constant. But during low traffic, the window is 'empty' for long periods, and a burst of failures near the end of the window may not trigger the breaker if earlier successes dilute the rate.

Time-based uses a sliding timestamp list. Each call records its result and timestamp. Old records are evicted when they're older than the window duration. This adapts naturally to traffic variations: during a spike, the window fills quickly; during a lull, it decays. The memory overhead is higher because every call's timestamp is stored — O(windowSize) in the count-based case vs O(requestsInWindow) in time-based.

Which one should you use? If your traffic is uniform (e.g., 100 req/s constantly), count-based is fine. If your traffic is bursty (e.g., periodic batch jobs that drive request spikes), time-based is more accurate because it measures real time, not request count.

Production Gotchas: What Bites Teams That Think They've Set It Up Correctly

Even with a working circuit breaker, teams hit common pitfalls that cause outages. Here are the six most dangerous ones.

1. Circuit breaker on timeout only, not on exception type Many configurations only count timeouts as failures. But network errors, 5xx responses, and even 429 rate limits should also be counted. If you only count timeouts, a service returning 503 errors will never trip the breaker.

2. Half-open probes that don't match real traffic The probe request is often a simple health check. But the real failure could be a specific endpoint that's slow. Configuration: configure the circuit breaker's probe to use a representative call, or use the same method call with a decorator that records success/failure on every call (even when half-open).

3. Not isolating thread pools per circuit breaker If all circuit breakers share one thread pool for their downstream calls, one open breaker reduces the pool's available threads for other dependencies. Separate thread pools (using Resilience4j's Bulkhead) prevent this.

4. Recovery timeout too short Setting the open state duration to 5 seconds on a database that takes 30 seconds to restart causes continuous open/half-open flapping. Recovery timeout should be at least the P99 recovery time of the downstream service, plus 50%.

5. Forgetting to reset failures on success Some custom implementations never reset the failure count on a successful call while in CLOSED state. This causes the breaker to open after X total failures, even if they occurred days apart. Always reset the failure count after a successful call if you're using a count-based approach (or rely on sliding window).

6. No fallback mechanism Circuit breakers reject requests when open. If you don't provide a fallback (e.g., a cached response or a default value), the user gets an error. Combine circuit breaker with a fallback method for a better user experience.

package io.thecodeforge.circuitbreaker;

import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;
import java.time.Duration;
import java.util.function.Supplier;

public class GotchaExample {
    // CORRECT: records both exceptions and HTTP errors
    CircuitBreakerConfig config = CircuitBreakerConfig.custom()
        .recordExceptions(IOException.class, TimeoutException.class)
        .recordStatusCodes(500, 502, 503, 504, 429)
        .build();

    // WRONG: only records timeout
    CircuitBreakerConfig wrongConfig = CircuitBreakerConfig.custom()
        .recordExceptions(TimeoutException.class)
        .build();
}