Spring Cloud Consul: Service Discovery & Configuration with HashiCorp Consul

You've built a handful of microservices, and they're talking to each other. But hardcoding IP addresses and ports? That's a disaster waiting to happen. When a service instance goes down or a new one spins up, you don't want to redeploy everything. That's where service discovery comes in. And if you're tired of managing configuration files across a dozen services, you need a centralized configuration server.

Spring Cloud Consul combines both: service discovery and distributed configuration, backed by HashiCorp Consul. It's battle-tested, production-hardened, and far more flexible than Eureka for real-world deployments.

In this guide, I'll walk you through setting up Spring Cloud Consul for service discovery and configuration management. I'll share war stories from production where Consul saved our bacon—and where it almost burned the house down. By the end, you'll know not just how to use it, but how to avoid the pitfalls that the official docs gloss over.

Why Consul Over Eureka?

If you've used Spring Cloud Eureka, you know it works—until it doesn't. Eureka is a Netflix relic that lacks health checking, multi-datacenter support, and a consistent key-value store. Consul, on the other hand, is a first-class citizen in the HashiCorp ecosystem, designed for production.

Here's the hard truth: Eureka is fine for a small cluster in a single region. But if you need health checks that actually remove dead instances, or if you need configuration management built-in, Consul is the better choice. Consul uses the Raft consensus protocol for strong consistency, so you never get stale data. Eureka uses eventual consistency, which can lead to routing to dead instances.

I've seen teams migrate from Eureka to Consul after a production incident where a zombie instance kept receiving traffic for 15 minutes because Eureka's heartbeats were too slow. Consul's TTL-based health checks catch failures in seconds.

That said, Consul adds operational complexity. You need to run a Consul cluster (or use the cloud-managed version). But the trade-off is worth it for any serious microservices deployment.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>

Setting Up Service Discovery with Consul

Enough theory—let's get our hands dirty. First, add the Consul Discovery starter to your pom.xml. Then configure your bootstrap.yml (or application.yml) with the Consul host and port.

Here's a minimal configuration that registers a service named 'payment-service' on port 8080. Consul will automatically health-check the /actuator/health endpoint every 10 seconds.

One gotcha: if you don't set a unique instance-id, the second instance will fail to register because Consul sees a duplicate. Use a combination of application name, port, and random value.

spring:
  application:
    name: payment-service
  cloud:
    consul:
      host: localhost
      port: 8500
      discovery:
        instance-id: ${spring.application.name}:${spring.application.instance_id:${random.value}}
        prefer-ip-address: true
        health-check-interval: 10s
        health-check-critical-timeout: 30s

Distributed Configuration with Consul KV

Consul's key-value store is a perfect place to centralize configuration. Instead of scattering application.yml files across services, you store configs in Consul under a path like config/{application}/{profile}/{property}. Spring Cloud Consul watches these paths and refreshes beans annotated with @RefreshScope.

To enable config, add the Consul Config starter. Then in bootstrap.yml, set the prefix (default 'config'), default context (application name), and profile-separator.

Here's the real power: you can have a common config for all services (under config/application/) and service-specific overrides (under config/payment-service/). The profile-specific ones (e.g., config/payment-service,dev/) take precedence.

But there's a catch: the watch mechanism polls Consul every 55 seconds by default. If you need faster updates, set spring.cloud.consul.config.watch.delay=1000 (milliseconds). But be careful—too frequent polling can overwhelm Consul.

spring:
  cloud:
    consul:
      config:
        enabled: true
        prefix: config
        default-context: application
        profile-separator: ','
        watch:
          enabled: true
          delay: 10000

What the Official Docs Won't Tell You

The Spring Cloud Consul documentation is decent, but it leaves out some critical details that you'll only learn in production.

1. Connection Pool Tuning: The default Consul client uses a connection pool of 20 connections. In a microservices environment with many instances and frequent health checks, you'll exhaust that pool. Set spring.cloud.consul.client.connection-pool.max-total to at least 200.
2. Health Check Timeouts: The default health check interval is 10 seconds, but if your /actuator/health endpoint takes longer (e.g., because it checks downstream services), Consul will mark your service as unhealthy. Set health-check-critical-timeout to 30s or more, and consider making health checks lightweight.
3. Config Watch Delay: The default watch delay is 55 seconds. That's too slow for critical config changes. Set it to 5-10 seconds, but monitor Consul server load.
4. Instance ID Collisions: If you run multiple instances on the same host (e.g., during development), they'll have the same hostname and port, causing registration failures. Use random.value in instance-id.
5. Ribbon vs. Spring Cloud LoadBalancer: Spring Cloud Consul uses Ribbon by default (deprecated) or Spring Cloud LoadBalancer. If you're on Spring Cloud 2020+, it uses LoadBalancer. Make sure you have the correct dependencies.
6. Multi-Datacenter: Consul supports multiple datacenters, but Spring Cloud Consul doesn't natively support cross-datacenter service discovery. You'll need to configure the Consul agent to join datacenters and then use DNS or custom code.

spring:
  cloud:
    consul:
      client:
        connection-pool:
          max-total: 200
          max-per-route: 50
      discovery:
        health-check-critical-timeout: 30s
        health-check-interval: 10s
        instance-id: ${spring.application.name}:${random.value}

Integrating Consul with Spring Cloud LoadBalancer

Once services are registered, you need a client-side load balancer to distribute requests. Spring Cloud Consul integrates with Spring Cloud LoadBalancer (or the legacy Ribbon). The load balancer queries Consul for healthy instances of a service and picks one using a rule (e.g., round-robin).

To use it, add spring-cloud-starter-loadbalancer to your dependencies. Then, when you make a REST call to a service by its application name (e.g., http://payment-service/api/pay), the load balancer resolves it via Consul.

But here's a nuance: the load balancer caches the service list. By default, it refreshes every 30 seconds. If a service goes down, you might still route to it for up to 30 seconds. To reduce that, set spring.cloud.loadbalancer.cache.enabled=false or tune the cache TTL.

In production, I prefer to enable caching with a short TTL (e.g., 5 seconds) to balance freshness and performance. Also, use health checks to remove unhealthy instances faster.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Health Checks: The Unsung Heroes

Consul health checks are what make service discovery reliable. Each registered service has a health check that Consul runs periodically. If the check fails, Consul marks the service as unhealthy and stops routing traffic to it.

Spring Cloud Consul automatically registers an HTTP health check against /actuator/health. But the default configuration can be problematic. The health check interval is 10 seconds, and the timeout is 5 seconds. If your health endpoint takes longer than 5 seconds (e.g., because it checks a database), the check will timeout and fail.

My advice: make your health endpoint fast. Use liveness and readiness probes appropriately. For Consul, you want a lightweight check that just verifies the application is alive, not that the database is reachable. Consider creating a custom health indicator that doesn't check downstream dependencies.

Also, set health-check-critical-timeout. If a check fails for this duration, Consul will deregister the service. This prevents zombie instances from lingering.

@Component
public class ConsulHealthIndicator implements HealthIndicator {
    @Override
    public Health health() {
        // Only check if the app is alive, not external dependencies
        return Health.up().build();
    }
}

Production Hardening: Connection Pool, Timeouts, and Retries

Running Consul in production requires careful configuration of the client. The defaults are meant for development, not for high-traffic environments.

First, the connection pool. As mentioned, increase max-total and max-per-route. I've seen production apps with 100+ instances that need 500 connections. Monitor the pool usage via Micrometer metrics (consul.client.connection-pool.*).

Second, timeouts. Set a read timeout and connect timeout: spring.cloud.consul.client.read-timeout=5000, connect-timeout=3000. If Consul is slow, your app will hang waiting for a response.

Third, retries. The Consul client retries on failures by default, but you can configure the number of retries: spring.cloud.consul.client.retry.max-attempts=3. Combine with exponential backoff.

Fourth, circuit breaker. Consider wrapping Consul calls with a circuit breaker (e.g., Resilience4j) to prevent cascading failures if Consul goes down.

Finally, consider using a dedicated Consul agent on each host (the 'agent' mode) instead of connecting directly to the server. This reduces load on the server and provides local caching.

spring:
  cloud:
    consul:
      client:
        read-timeout: 5000
        connect-timeout: 3000
        retry:
          max-attempts: 3
        connection-pool:
          max-total: 500
          max-per-route: 100