Spring Cloud AWS Messaging: SQS and SNS Integration

Messaging is the backbone of any resilient microservices architecture. If your services talk to each other via synchronous HTTP calls, you're building a house of cards. One service goes down, and the whole chain collapses. That's where Spring Cloud AWS messaging comes in, giving you battle-tested integration with Amazon SQS and SNS.

I've been building microservices since the Netflix stack days, and I've seen teams butcher messaging patterns more times than I care to admit. They treat SQS like a database, poll every second, and wonder why their bill is astronomical. Or they use SNS without idempotency and duplicate payments all over the place.

In this article, I'll show you how to integrate SQS and SNS the right way using Spring Cloud AWS. We'll cover configuration, production patterns, and the gotchas that the official docs gloss over. By the end, you'll know how to build messaging that scales, handles failures, and doesn't bankrupt you on AWS costs.

We'll use realistic examples from payment processing and event-driven architectures. No more generic User/Order nonsense. You'll see how to process payment events reliably, handle retries, and debug issues when things go sideways at 2 AM.

Setting Up Spring Cloud AWS Messaging

First things first: add the right dependencies. Spring Cloud AWS has evolved. If you're still using the old spring-cloud-aws-messaging from the 2.x line, you're missing out. The 3.x line (Spring Cloud AWS 3.0+) is a complete rewrite backed by the AWS SDK v2. It's faster, supports reactive, and has sane defaults.

Add this to your pom.xml:

```xml <dependencyManagement> <dependencies> <dependency> <groupId>io.awspring.cloud</groupId> <artifactId>spring-cloud-aws-dependencies</artifactId> <version>3.1.0</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>

<dependency> <groupId>io.awspring.cloud</groupId> <artifactId>spring-cloud-aws-starter-sqs</artifactId> </dependency> <dependency> <groupId>io.awspring.cloud</groupId> <artifactId>spring-cloud-aws-starter-sns</artifactId> </dependency> ```

Now configure your AWS credentials. I've seen teams hardcode them in properties files and then commit to GitHub. Don't be that person. Use IAM roles if on EC2/EKS, or environment variables locally.

``properties # application.yml spring: cloud: aws: region: static: us-east-1 credentials: access-key: ${AWS_ACCESS_KEY_ID} secret-key: ${AWS_SECRET_ACCESS_KEY} sqs: listener: poll-timeout: 20s max-concurrent-messages: 10 ``

The poll-timeout enables long polling, which reduces empty receives and saves money. max-concurrent-messages controls how many messages your listener processes in parallel. Tune this based on your downstream dependencies.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.awspring.cloud</groupId>
      <artifactId>spring-cloud-aws-dependencies</artifactId>
      <version>3.1.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependency>
  <groupId>io.awspring.cloud</groupId>
  <artifactId>spring-cloud-aws-starter-sqs</artifactId>
</dependency>
<dependency>
  <groupId>io.awspring.cloud</groupId>
  <artifactId>spring-cloud-aws-starter-sns</artifactId>
</dependency>

Consuming SQS Messages with @SqsListener

The @SqsListener annotation is your bread and butter. It automatically polls the queue, deserializes the message body (JSON by default), and invokes your method. But here's where most people screw up: they treat the listener method like a regular controller method.

```java @Component public class PaymentEventListener {

private static final Logger log = LoggerFactory.getLogger(PaymentEventListener.class); private final PaymentService paymentService;

public PaymentEventListener(PaymentService paymentService) { this.paymentService = paymentService; }

@SqsListener("payment-events") public void handlePaymentEvent(PaymentEvent event) { log.info("Processing payment event: {}", event.paymentId()); paymentService.processPayment(event); } } ```

This looks innocent, but it's blocking. If processPayment takes 10 seconds, your listener thread is tied up. With max-concurrent-messages=10, you can handle 10 messages concurrently, but if each takes 10s, throughput is 1 msg/sec. Not great.

Instead, make your listener asynchronous:

``java @SqsListener("payment-events") public void handlePaymentEvent(PaymentEvent event) { CompletableFuture.runAsync(() -> { try { paymentService.processPayment(event); } catch (Exception e) { log.error("Failed to process payment {}", event.paymentId(), e); // Let the framework handle retries throw e; } }); } ``

But be careful: if you throw an exception inside the async task, the framework won't know about it. The message will be deleted from the queue (since the listener returned successfully). You need to handle errors explicitly.

A better approach: use the @Async annotation on the service method and configure a thread pool. Or use reactive programming with Spring WebFlux.

What the docs don't tell you: the default error handler deletes messages on success and redrives on exception. If you catch the exception and don't rethrow, the message is gone forever. Always rethrow runtime exceptions for transient failures.

@Component
public class PaymentEventListener {

    private static final Logger log = LoggerFactory.getLogger(PaymentEventListener.class);
    private final PaymentService paymentService;

    public PaymentEventListener(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    @SqsListener("payment-events")
    public void handlePaymentEvent(PaymentEvent event) {
        log.info("Processing payment event: {}", event.paymentId());
        try {
            paymentService.processPayment(event);
        } catch (NonRetryableException e) {
            log.error("Non-retryable error for payment {}", event.paymentId(), e);
            // Do NOT rethrow - message will be deleted
        } catch (Exception e) {
            log.error("Retryable error for payment {}", event.paymentId(), e);
            throw new RuntimeException(e); // Redrive to DLQ after maxReceiveCount
        }
    }
}

Publishing to SNS Topics

SNS is great for fan-out: one event, many subscribers. Spring Cloud AWS makes publishing trivial with SnsTemplate or NotificationMessagingTemplate. But here's the catch: SNS is fire-and-forget by default. If your subscriber is down, the message is lost. That's why the production pattern is SNS -> SQS -> Consumer.

```java @Service public class PaymentEventPublisher {

private final NotificationMessagingTemplate messagingTemplate;

public PaymentEventPublisher(NotificationMessagingTemplate messagingTemplate) { this.messagingTemplate = messagingTemplate; }

public void publishPaymentEvent(PaymentEvent event) { messagingTemplate.sendNotification("payment-events-topic", event, event.paymentId()); } } ```

The third parameter is the subject (optional). Under the hood, it serializes the object to JSON and publishes to the topic ARN (or name if you configured a prefix).

But what if the SNS topic doesn't exist? Spring Cloud AWS can auto-create topics if you enable it:

``properties spring.cloud.aws.sns.enable-sns-topic-creation=true ``

Don't rely on this in production. Create topics via CloudFormation or Terraform. Auto-creation can lead to unexpected costs and permissions issues.

Now, the critical part: idempotency. SNS does not guarantee exactly-once delivery to subscribers. If your subscriber is an SQS queue, the queue might receive duplicates. Always design your consumers to be idempotent.

One more thing: SNS message filtering. You can set filter policies on subscriptions so that only certain messages reach the queue. This reduces noise and saves costs. For example, only forward events with type=payment.

@Service
public class PaymentEventPublisher {

    private final NotificationMessagingTemplate messagingTemplate;

    public PaymentEventPublisher(NotificationMessagingTemplate messagingTemplate) {
        this.messagingTemplate = messagingTemplate;
    }

    public void publishPaymentEvent(PaymentEvent event) {
        messagingTemplate.sendNotification("payment-events-topic", event, event.paymentId());
    }
}

What the Official Docs Won't Tell You

The official Spring Cloud AWS documentation is decent, but it glosses over several production realities. Here are the gotchas I've encountered:

1. FIFO Queue Limitations FIFO queues guarantee exactly-once processing, but they have a throughput limit of 3000 messages per second (with batching). If you need more, you're out of luck. Also, the consumer group concept doesn't exist in SQS. If you have multiple consumers on a FIFO queue, each message is processed by exactly one consumer, but ordering is per message group ID. Choose your group ID wisely: use customer ID or order ID to maintain ordering per entity.

2. Deserialization Failures By default, Spring Cloud AWS uses Jackson to deserialize JSON. If your message payload changes (e.g., new field added), deserialization might fail silently. The message will be redriven to DLQ. To handle schema evolution, use @JsonIgnoreProperties(ignoreUnknown = true) on your DTO or use a version field.

3. Long Polling vs Short Polling Short polling (default) returns immediately even if the queue is empty. This causes high AWS costs due to empty receives. Always enable long polling with poll-timeout=20s. But don't set it too high (max 20s) or your threads will be blocked.

4. The DLQ Configuration If you don't configure a dead-letter queue, messages that fail processing are retried indefinitely (up to maxReceiveCount, default 3). After that, they disappear. Always set a DLQ with a reasonable maxReceiveCount (3-5). Monitor the DLQ for anomalies.

5. SNS Subscription Confirmation When you subscribe an SQS queue to an SNS topic, AWS sends a confirmation message to the queue. Spring Cloud AWS auto-confirms subscriptions by default, but if the queue ARN is wrong, it fails silently. Always verify subscriptions in the AWS Console.

6. CloudWatch Metrics Don't rely on logs alone. Monitor ApproximateNumberOfMessagesVisible, NumberOfMessagesReceived, and NumberOfMessagesDeleted. A sudden drop in deleted messages indicates processing failures.

@JsonIgnoreProperties(ignoreUnknown = true)
public record PaymentEvent(
    String paymentId,
    String customerId,
    BigDecimal amount,
    String currency,
    @JsonProperty("event_version") int eventVersion
) {}

Error Handling and Retries with DLQ

Let's talk about the redrive policy. When your listener throws an exception, Spring Cloud AWS increments the receive count. After maxReceiveCount (default 3), the message is sent to the dead-letter queue (DLQ). But here's the nuance: if you catch the exception and don't rethrow, the message is considered successfully processed and deleted. That's bad if you want to retry.

Configure your DLQ in the AWS Console or via CloudFormation:

```yaml PaymentQueue: Type: AWS::SQS::Queue Properties: QueueName: payment-events RedrivePolicy: deadLetterTargetArn: !GetAtt PaymentDLQ.Arn maxReceiveCount: 5

PaymentDLQ: Type: AWS::SQS::Queue Properties: QueueName: payment-events-dlq ```

Now, in your listener, decide which exceptions are retryable. For transient errors (network timeout, database deadlock), rethrow a RuntimeException. For permanent errors (invalid message format), catch and log, but don't rethrow so the message is deleted.

``java @SqsListener("payment-events") public void handle(PaymentEvent event) { try { paymentService.process(event); } catch (InvalidPaymentException e) { log.error("Invalid payment, discarding: {}", event.paymentId(), e); // No rethrow -> message deleted } catch (RetryableException e) { log.warn("Retryable error for payment {}", event.paymentId(), e); throw new RuntimeException(e); // Redrive } } ``

But what if you want to retry with exponential backoff? SQS doesn't support that natively. You can implement a custom retry by re-queuing the message with a delay. Or use a more sophisticated approach: send the message to a separate retry queue with a delay, and after max attempts, move to DLQ.

Alternatively, use Spring Retry:

```java @SqsListener("payment-events") @Retryable(value = {RetryableException.class}, maxAttempts = 3, backoff = @Backoff(delay = 2000, multiplier = 2)) public void handle(PaymentEvent event) { paymentService.process(event); }

@Recover public void recover(RetryableException e, PaymentEvent event) { log.error("Payment failed after retries: {}", event.paymentId(), e); // Send to DLQ manually or just let it go } ```

This works, but note: the @Recover method is called after all retries fail, and the message is still considered processed (deleted) because the method returned without exception. So you lose the message unless you manually send it to DLQ.

@SqsListener("payment-events")
@Retryable(value = {RetryableException.class}, maxAttempts = 3, backoff = @Backoff(delay = 2000, multiplier = 2))
public void handle(PaymentEvent event) {
    paymentService.process(event);
}

@Recover
public void recover(RetryableException e, PaymentEvent event) {
    log.error("Payment failed after retries: {}", event.paymentId(), e);
    // Optionally send to DLQ manually
    sqsTemplate.send("payment-events-dlq", event);
}

Testing SQS/SNS Integration Locally

You can't always hit real AWS queues during development. That's where local emulators come in. ElasticMQ is a lightweight SQS emulator. LocalStack provides both SQS and SNS.

For ElasticMQ, run it with Docker:

``bash docker run -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native ``

Then configure Spring:

``properties spring.cloud.aws.sqs.endpoint=http://localhost:9324 spring.cloud.aws.sqs.region=us-east-1 spring.cloud.aws.credentials.access-key=test spring.cloud.aws.credentials.secret-key=test ``

ElasticMQ doesn't support SNS. For that, use LocalStack:

``bash docker run -p 4566:4566 localstack/localstack ``

Configure:

``properties spring.cloud.aws.sqs.endpoint=http://localhost:4566 spring.cloud.aws.sns.endpoint=http://localhost:4566 spring.cloud.aws.region.static=us-east-1 ``

Now you can create queues and topics programmatically or via AWS CLI pointing to localhost.

Here's a test example using @SpringBootTest and @TestContainers:

```java @SpringBootTest @Testcontainers class PaymentEventIntegrationTest {

@Container static LocalStackContainer localstack = new LocalStackContainer(DockerImageName.parse("localstack/localstack:3.0")) .withServices(SQS, SNS);

@DynamicPropertySource static void configureProperties(DynamicPropertyRegistry registry) { registry.add("spring.cloud.aws.sqs.endpoint", () -> localstack.getEndpointOverride(SQS).toString()); registry.add("spring.cloud.aws.sns.endpoint", () -> localstack.getEndpointOverride(SNS).toString()); registry.add("spring.cloud.aws.region.static", () -> localstack.getRegion()); }

@Autowired private SqsTemplate sqsTemplate;

@Autowired private NotificationMessagingTemplate snsTemplate;

@Test void testPublishAndConsume() { // Create queue and topic CreateQueueResult queue = sqsTemplate.createQueue("test-queue"); CreateTopicResult topic = snsTemplate.createTopic("test-topic"); // Subscribe snsTemplate.subscribe(topic.topicArn(), "sqs", queue.queueUrl()); // Publish snsTemplate.sendNotification(topic.topicArn(), new PaymentEvent("1", "cust1", BigDecimal.TEN, "USD", 1), "test"); // Poll List<Message> messages = sqsTemplate.receive(queue.queueUrl()); assertThat(messages).hasSize(1); } }

@SpringBootTest
@Testcontainers
class PaymentEventIntegrationTest {

    @Container
    static LocalStackContainer localstack = new LocalStackContainer(DockerImageName.parse("localstack/localstack:3.0"))
            .withServices(SQS, SNS);

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.cloud.aws.sqs.endpoint", () -> localstack.getEndpointOverride(SQS).toString());
        registry.add("spring.cloud.aws.sns.endpoint", () -> localstack.getEndpointOverride(SNS).toString());
        registry.add("spring.cloud.aws.region.static", () -> localstack.getRegion());
    }

    @Autowired
    private SqsTemplate sqsTemplate;

    @Autowired
    private NotificationMessagingTemplate snsTemplate;

    @Test
    void testPublishAndConsume() {
        CreateQueueResult queue = sqsTemplate.createQueue("test-queue");
        CreateTopicResult topic = snsTemplate.createTopic("test-topic");
        snsTemplate.subscribe(topic.topicArn(), "sqs", queue.queueUrl());
        snsTemplate.sendNotification(topic.topicArn(), new PaymentEvent("1", "cust1", BigDecimal.TEN, "USD", 1), "test");
        List<Message> messages = sqsTemplate.receive(queue.queueUrl());
        assertThat(messages).hasSize(1);
    }
}

Performance Tuning and Best Practices

Let's squeeze performance out of your messaging system. Here are the knobs you should tune:

1. Concurrency max-concurrent-messages controls how many messages your listener processes in parallel. Default is 10. If your processing is I/O-bound (calling external APIs), increase it to 20-50. But be careful: each thread consumes memory. Monitor heap usage.

2. Long Polling poll-timeout should be set to 20s (max). This reduces empty receives. But if you need lower latency, set it to 5-10s. Balance cost vs latency.

3. Batch Size When publishing, batch messages to SQS. Use SqsTemplate.sendBatch() to send up to 10 messages at once. This reduces API calls and costs.

4. Message Size SQS supports up to 256KB per message. If you need larger payloads, use S3 with a pointer in the message (S3 link). Or compress the payload.

5. Consumer Idempotency As stressed earlier, always implement idempotency. Use a database table to track processed message IDs. For high throughput, use Redis with TTL.

6. Monitoring Set up CloudWatch dashboards for queue depth, age of oldest message, and DLQ depth. Alert on anomalies.

Here's a production-ready configuration:

``properties spring.cloud.aws.sqs.listener.poll-timeout=20s spring.cloud.aws.sqs.listener.max-concurrent-messages=20 spring.cloud.aws.sqs.listener.queue-name=payment-events spring.cloud.aws.sqs.listener.max-number-of-messages=10 ``

And a batch publisher:

```java @Bean public SqsTemplate sqsTemplate(SqsAsyncClient sqsAsyncClient) { return SqsTemplate.builder().sqsAsyncClient(sqsAsyncClient).build(); }

public void publishBatch(List<PaymentEvent> events) { List<SqsBatchMessage> messages = events.stream() .map(event -> SqsBatchMessage.builder() .messageBody(event.toString()) .id(UUID.randomUUID().toString()) .build()) .toList(); sqsTemplate.sendBatch("payment-events", messages); } ```

@Service
public class BatchPublisher {

    private final SqsTemplate sqsTemplate;

    public BatchPublisher(SqsTemplate sqsTemplate) {
        this.sqsTemplate = sqsTemplate;
    }

    public void publishBatch(List<PaymentEvent> events) {
        List<SqsBatchMessage> messages = events.stream()
            .map(event -> SqsBatchMessage.builder()
                .messageBody(event.toString())
                .id(UUID.randomUUID().toString())
                .build())
            .toList();
        sqsTemplate.sendBatch("payment-events", messages);
    }
}