Spring Boot Actuator: Custom Health Indicators, Metrics, and Endpoints for Production

After 15 years of building and operating payment-processing systems and SaaS billing platforms, I've seen more production outages caused by blind spots than by bad code. Your application might handle millions of dollars in transactions, but if you can't see its internal health, you're flying blind. Spring Boot Actuator 3.x, combined with Micrometer 1.12.x, gives you that visibility — but only if you use it correctly. The default configuration is a starting point, not a production-ready setup. In this article, we'll go beyond the basics. You'll learn how to write custom health indicators that actually catch failures, create business metrics that alert you before revenue drops, and build custom endpoints that expose the data your operations team needs at 3 AM. I'll share real war stories from production, including the time a misconfigured health check took down an entire microservice cluster. We'll cover version-specific details for Spring Boot 3.2.x and Java 21, with code you can adapt to any domain. By the end, you'll have a production-grade Actuator setup that your on-call team will thank you for.

Custom HealthIndicator: Beyond the Built-In Checks

Spring Boot 3.2.x ships with several built-in health indicators for common data sources (DataSourceHealthIndicator), Redis (RedisHealthIndicator), Elasticsearch, and more. But in a real-world payment-processing system, you need to check things like: is the payment gateway reachable? Is the fraud detection service responsive? Is the queue for pending transactions not backed up? A custom HealthIndicator is a Spring bean that implements the HealthIndicator interface and returns a Health object. The key is to make it fast, resilient, and meaningful. Never block for more than a second. Use timeouts. Cache results if the check is expensive. Here's a pattern I've used in production for checking a third-party payment API:

import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
import java.time.Duration;

@Component
public class PaymentGatewayHealthIndicator implements HealthIndicator {

    private final RestTemplate restTemplate;

    public PaymentGatewayHealthIndicator(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
        this.restTemplate.setConnectTimeout(Duration.ofMillis(500));
        this.restTemplate.setReadTimeout(Duration.ofMillis(1000));
    }

    @Override
    public Health health() {
        try {
            // Perform a lightweight ping, not a full transaction
            String response = restTemplate.getForObject(
                "https://api.paymentgateway.com/v1/ping", String.class);
            if ("OK".equals(response)) {
                return Health.up()
                    .withDetail("gateway", "reachable")
                    .withDetail("latencyMs", 45)
                    .build();
            }
            return Health.down()
                .withDetail("gateway", "unexpected_response")
                .withDetail("response", response)
                .build();
        } catch (Exception e) {
            return Health.down()
                .withDetail("gateway", "unreachable")
                .withDetail("error", e.getMessage())
                .build();
        }
    }
}

What the Official Docs Won't Tell You

The official Spring Boot documentation for Actuator is comprehensive but misses critical production patterns. First: health groups. You can define custom health groups like /actuator/health/db or /actuator/health/external to expose only specific indicators. But the docs don't emphasize that you should NEVER expose the full health endpoint to the public internet — it leaks internal system details. Use management.endpoints.web.exposure.include=health,info and exclude everything else. Second: the liveness and readiness probes in Kubernetes are separate concepts. Spring Boot 2.3+ added the /actuator/health/liveness and /actuator/health/readiness endpoints, but many teams still use the generic /actuator/health for both, causing cascading failures. Third: Micrometer's MeterRegistry is a singleton, but if you create multiple registries (e.g., one for Prometheus, one for Graphite), you can double-count metrics. Always use the auto-configured MeterRegistry. Fourth: custom endpoints must be registered as beans, but the endpoint ID must be unique. If you create an endpoint with the same ID as a built-in one (like 'health'), it will override it silently. Finally: the info endpoint can be enhanced with environment properties, but the docs don't warn that exposing spring.datasource.password via info is a security risk. Always filter sensitive properties.

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus
      base-path: /actuator
  endpoint:
    health:
      show-details: when-authorized
      show-components: when-authorized
      roles: ADMIN
      probes:
        enabled: true
  info:
    env:
      enabled: true
      # Never expose sensitive keys
      keys-to-sanitize: password,secret,key,token,.*credentials.*

Custom Micrometer Metrics: Counters, Gauges, and Timers for Business Logic

Micrometer 1.12.x is the metrics facade in Spring Boot 3.2.x. While built-in metrics cover JVM, CPU, and database pools, custom business metrics are what save you at 3 AM. In a payment-processing system, I track: payment.attempts (counter), payment.success.rate (gauge), payment.latency (timer), and queue.depth (gauge). The trick is to use the MeterRegistry injected by Spring Boot. Never create your own MeterRegistry. Use tags (dimensions) to slice metrics by important attributes like payment method, currency, or region. But beware: high-cardinality tags (like user ID) will blow up your metrics storage. Limit tags to fewer than 10 unique values. For timers, use the Timer.Sample pattern to measure latency correctly across threads. Here's how to instrument a payment processing method:

import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;
import org.springframework.stereotype.Service;

@Service
public class PaymentMetricsService {

    private final MeterRegistry meterRegistry;

    public PaymentMetricsService(MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;
    }

    public void recordPaymentAttempt(String currency, String paymentMethod) {
        // Counter: increment per attempt
        meterRegistry.counter("payment.attempts",
            "currency", currency,
            "method", paymentMethod
        ).increment();
    }

    public void recordPaymentSuccess(String currency, String paymentMethod, boolean success) {
        // Gauge: current success rate (0.0 to 1.0)
        // This should be updated from a scheduled job or event listener
        meterRegistry.gauge("payment.success.rate",
            Tags.of("currency", currency, "method", paymentMethod),
            success ? 1.0 : 0.0
        );
    }

    public Timer.Sample startTimer() {
        return Timer.start(meterRegistry);
    }

    public void stopTimer(Timer.Sample sample, String currency, String paymentMethod) {
        sample.stop(Timer.builder("payment.latency")
            .tag("currency", currency)
            .tag("method", paymentMethod)
            .publishPercentiles(0.5, 0.95, 0.99)
            .register(meterRegistry));
    }
}

Creating Custom Actuator Endpoints for Business Data

Sometimes health indicators and metrics aren't enough. You need to expose business data directly — like the current state of a batch job, the number of pending refunds, or the last 10 failed transactions. Spring Boot lets you create custom endpoints by implementing the @Endpoint annotation. You can expose read operations with @ReadOperation, write operations with @WriteOperation, and delete operations with @DeleteOperation. The endpoint ID becomes the URL path (e.g., /actuator/pending-refunds). Always return a Map or a custom POJO — Spring Boot serializes it to JSON. Important: custom endpoints are not secured by default. You must add Spring Security or use the management.endpoints.web.exposure.include to control access. Also, avoid heavy database queries in read operations — they should be fast. Cache results if needed. Here's an example for exposing pending payment retries:

import org.springframework.boot.actuate.endpoint.annotation.Endpoint;
import org.springframework.boot.actuate.endpoint.annotation.ReadOperation;
import org.springframework.stereotype.Component;
import java.util.HashMap;
import java.util.Map;

@Component
@Endpoint(id = "pending-retries")
public class PendingRetriesEndpoint {

    private final PaymentRetryService retryService;

    public PendingRetriesEndpoint(PaymentRetryService retryService) {
        this.retryService = retryService;
    }

    @ReadOperation
    public Map<String, Object> getPendingRetries() {
        Map<String, Object> result = new HashMap<>();
        result.put("totalPending", retryService.countPending());
        result.put("oldestRetry", retryService.getOldestPendingTime());
        result.put("recentFailures", retryService.getRecentFailures(10));
        result.put("status", retryService.isRetryQueueHealthy() ? "HEALTHY" : "DEGRADED");
        return result;
    }

    // Optionally: @WriteOperation to manually trigger a retry
}

Integrating with Prometheus and Grafana: Production-Ready Configuration

Spring Boot Actuator with Micrometer can export metrics to Prometheus via the micrometer-registry-prometheus dependency. But the default configuration is not production-ready. First, you must add the dependency (io.micrometer:micrometer-registry-prometheus:1.12.x). Then, expose the prometheus endpoint: management.endpoints.web.exposure.include=prometheus. The /actuator/prometheus endpoint outputs text in Prometheus exposition format. But here's the gotcha: Prometheus scrapes on a schedule (typically 15s). If your metrics change rapidly (like payment latency), you might miss spikes. Use Micrometer's step meter registries to handle this, but be aware that step registries buffer metrics and export them on the step boundary. For Grafana, you'll create dashboards that query Prometheus. But don't just import a generic JVM dashboard — build business-specific panels. I always include: payment throughput (rate of payment_attempts_total), error rate (rate of payment_attempts_total with status=error), latency percentiles (payment_latency_seconds), and queue depth (queue_depth gauge). Also, set up alerting rules in Prometheus for: high error rate (>5% over 5 minutes), high latency (p99 > 1s), and queue depth growing. Finally, use the management.metrics.export.prometheus properties to configure pushgateway, histogram buckets, and step size.

groups:
  - name: payment-service
    rules:
      - alert: HighPaymentErrorRate
        expr: rate(payment_attempts_total{status="error"}[5m]) / rate(payment_attempts_total[5m]) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Payment error rate > 5%"
      - alert: HighPaymentLatency
        expr: payment_latency_seconds{quantile="0.99"} > 1.0
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "P99 latency > 1 second"
      - alert: QueueDepthGrowing
        expr: queue_depth > 1000
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Queue depth > 1000 for 10 minutes"

Testing Custom Health Indicators and Endpoints

Your custom health indicators and endpoints are only useful if they work correctly under load and failure. Unit testing is not enough — you need integration tests that simulate real conditions. Spring Boot 3.2.x provides @WebMvcTest and @SpringBootTest with auto-configured MockMvc for testing Actuator endpoints. But the real challenge is testing health indicators that depend on external services. Use Mockito to mock the external dependency, but also write a test that simulates timeout and connection failure. For custom endpoints, test both the happy path and the case where the underlying service is down. Also, test the security configuration: verify that unauthenticated requests get 401 or 403. Here's a pattern for testing a custom health indicator:

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.bean.MockBean;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.web.client.RestTemplate;

import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@SpringBootTest
@AutoConfigureMockMvc
class PaymentGatewayHealthIndicatorTest {

    @Autowired
    private MockMvc mockMvc;

    @MockBean
    private RestTemplate restTemplate;

    @Test
    void testHealthUp() throws Exception {
        when(restTemplate.getForObject(anyString(), any())).thenReturn("OK");

        mockMvc.perform(get("/actuator/health"))
            .andExpect(status().isOk())
            .andExpect(jsonPath("$.components.paymentGateway.status").value("UP"));
    }

    @Test
    void testHealthDown() throws Exception {
        when(restTemplate.getForObject(anyString(), any()))
            .thenThrow(new RuntimeException("Connection refused"));

        mockMvc.perform(get("/actuator/health"))
            .andExpect(status().isServiceUnavailable())
            .andExpect(jsonPath("$.components.paymentGateway.status").value("DOWN"));
    }

    @Test
    void testHealthTimeout() throws Exception {
        when(restTemplate.getForObject(anyString(), any()))
            .thenAnswer(inv -> {
                Thread.sleep(2000);
                return "OK";
            });

        mockMvc.perform(get("/actuator/health"))
            .andExpect(status().isServiceUnavailable())
            .andExpect(jsonPath("$.components.paymentGateway.status").value("DOWN"));
    }
}

Advanced: Conditional Health Indicators and Composite Health

In complex microservice architectures, you often need health checks that are conditional on the environment or the presence of certain beans. Spring Boot's @ConditionalOnBean and @ConditionalOnProperty annotations let you register health indicators only when certain conditions are met. For example, you might have a health indicator for a legacy payment system that only exists in the production environment. Use @ConditionalOnProperty(name = "payment.legacy.enabled", havingValue = "true") to register it conditionally. Another advanced pattern is composite health indicators — where you aggregate multiple health checks into a single logical check. Spring Boot's CompositeHealthContributor allows you to group related health indicators. For instance, you could have a 'payment-system' composite that includes the payment gateway, the fraud service, and the transaction queue. If any of them is DOWN, the composite is DOWN. This simplifies the health endpoint output and makes it easier for load balancers to interpret. But be careful: composite health indicators can mask individual failures if not implemented correctly. Always expose the details of each sub-component. Here's an example of a conditional health indicator:

import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.stereotype.Component;

@Component
@ConditionalOnProperty(name = "payment.legacy.enabled", havingValue = "true", matchIfMissing = false)
public class LegacyPaymentHealthIndicator implements HealthIndicator {

    @Override
    public Health health() {
        // Only registered if payment.legacy.enabled=true
        try {
            // Check legacy SOAP endpoint
            return Health.up()
                .withDetail("service", "legacy-payment")
                .withDetail("version", "1.0")
                .build();
        } catch (Exception e) {
            return Health.down(e)
                .withDetail("service", "legacy-payment")
                .build();
        }
    }
}

Debugging Actuator in Production: Tools and Techniques

When things go wrong in production, you need to debug Actuator itself. Common issues: health endpoint returns 404, metrics missing, custom endpoint not showing up. First, check the management.endpoints.web.exposure.include property — if it's not set, only health and info are exposed by default. Second, verify that your custom endpoint bean is actually created. Use the /actuator/beans endpoint (if exposed) to see all beans. Third, check the logs for 'Mapped' messages — Spring Boot logs each endpoint mapping at INFO level on startup. Fourth, use curl with verbose output to see the exact HTTP response. Fifth, for metrics issues, check if the MeterRegistry has the expected meters by calling /actuator/metrics. If a metric is missing, verify that the MeterRegistry is not a different instance (e.g., you injected a custom one instead of the auto-configured one). Sixth, for health indicators, check if the health indicator class is in a package that's component-scanned. Finally, use Spring Boot's Actuator endpoints to debug themselves: /actuator/health shows the overall status, /actuator/info shows build and git info, /actuator/metrics lists all available metrics. Here's a production debugging checklist:

#!/bin/bash
# Production debugging commands for Spring Boot Actuator

# 1. Check which endpoints are exposed
curl -s http://localhost:8080/actuator | jq .

# 2. Check health status with details (if authorized)
curl -s -u admin:admin http://localhost:8080/actuator/health | jq .

# 3. List all metrics
curl -s http://localhost:8080/actuator/metrics | jq .

# 4. Check a specific metric
curl -s http://localhost:8080/actuator/metrics/payment.attempts | jq .

# 5. Check custom endpoint
curl -s http://localhost:8080/actuator/pending-retries | jq .

# 6. Check environment properties (sensitive!)
curl -s -u admin:admin http://localhost:8080/actuator/env | jq '.propertySources[] | select(.name | test("payment"))'

# 7. Check if bean exists
curl -s -u admin:admin http://localhost:8080/actuator/beans | jq '.contexts."application".beans | keys[] | select(contains("PaymentGatewayHealth"))'