Spring Batch: Enterprise Batch Processing with Chunks and Tasks (2025 Guide)

Batch processing isn't dead — it's the silent workhorse behind every end-of-day settlement, payroll run, and data warehouse ETL. I've seen teams waste weeks trying to reinvent chunk processing with raw JDBC loops, only to hit the same wall: no restartability, no skip logic, no metrics. Spring Batch 5.2 (the latest stable as of early 2025) handles all of that out of the box. In this guide, we'll build a real payment settlement batch job that processes 500,000 transactions per run. You'll learn chunk-oriented processing with ItemReader, ItemProcessor, and ItemWriter; how to configure fault tolerance with skip and retry; and how to debug jobs in production using the JobExplorer. We'll also cover Tasklet-based steps for operations like archiving processed files. If you're coming from Spring Batch 4.x, the key changes are: Java 17 baseline, removal of deprecated classes (like JobBuilderFactory), and new Micrometer-based metrics. This is not a "hello world" tutorial — we're going deep into the patterns that survive production incidents.

Setting Up Spring Batch 5.2 with Spring Boot 3.4

Let's start with the Maven dependencies. Spring Boot 3.4.x auto-configures Spring Batch with sensible defaults, but you need to opt into the batch starter. Here's the minimal pom.xml setup with H2 for development and PostgreSQL for production. Note that Spring Batch 5 removed the deprecated JobBuilderFactory and StepBuilderFactory — you now inject JobRepository and PlatformTransactionManager directly. The @EnableBatchProcessing annotation is still required to trigger auto-configuration. In production, you must configure a JDBC-based JobRepository (default is in-memory Map-based, which doesn't survive restarts). I've seen teams lose hours of processing because they forgot to switch to JDBC repository. Use the provided schema-*.sql scripts for your database — they create the BATCH_JOB_INSTANCE, BATCH_JOB_EXECUTION, BATCH_STEP_EXECUTION, and BATCH_JOB_EXECUTION_CONTEXT tables.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>

What the Official Docs Won't Tell You

The official Spring Batch reference documentation is excellent for the happy path, but it glosses over three critical gotchas. First: chunk size is not just a performance knob — it's a transaction boundary. A chunk of 1000 means your transaction commits every 1000 items. If you have a writer that calls a remote API and that API is slow, you'll hold a database transaction open for too long, causing deadlocks. Second: the default commit-interval is not configurable via properties — you must set it programmatically in the step builder. Third: skip limits are per-step, not per-chunk. If you set skipLimit(10), the entire step will fail after 10 skipped items across all chunks. This is a common source of confusion when a single bad record causes a cascade of skips. Also, the docs don't emphasize that ItemStream.open() is called before the first chunk, and close() after the last chunk — if your reader opens a file, it must implement ItemStream to get proper lifecycle management. I've debugged too many issues where a reader left file handles open because the developer didn't implement close().

@Configuration
@EnableBatchProcessing
public class PaymentSettlementJobConfig {

    @Bean
    public Job paymentSettlementJob(JobRepository jobRepository,
                                     Step settlementStep) {
        return new JobBuilder("paymentSettlementJob", jobRepository)
                .start(settlementStep)
                .build();
    }

    @Bean
    public Step settlementStep(JobRepository jobRepository,
                                PlatformTransactionManager transactionManager,
                                ItemReader<Payment> reader,
                                ItemProcessor<Payment, Settlement> processor,
                                ItemWriter<Settlement> writer) {
        return new StepBuilder("settlementStep", jobRepository)
                .<Payment, Settlement>chunk(1000, transactionManager)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .faultTolerant()
                .skipLimit(10)
                .skip(DataAccessException.class)
                .retryLimit(3)
                .retry(NetworkTimeoutException.class)
                .build();
    }
}

Building a Payment Settlement Job with Chunk Processing

Let's build a real job: payment settlement. The reader fetches unsettled payments from a database, the processor calculates settlement amounts and applies fees, and the writer updates the database and sends a notification. We'll use JdbcCursorItemReader for the reader — it's efficient for large result sets because it uses a database cursor instead of loading everything into memory. The processor is where business logic lives: we compute the net settlement amount (transaction amount minus processing fee) and set the settlement date. The writer uses JdbcBatchItemWriter for batch updates — it's 10x faster than individual updates. Note that JdbcBatchItemWriter does not support transactions by itself; it relies on the chunk's transaction manager. Also, we must ensure idempotency: the job should be restartable. If it fails after writing 5000 items, on restart it should skip those 5000. Spring Batch handles this via the execution context, which stores the current position of the reader. For JdbcCursorItemReader, this works automatically if saveState=true (the default).

@Bean
public JdbcCursorItemReader<Payment> paymentReader(DataSource dataSource) {
    return new JdbcCursorItemReaderBuilder<Payment>()
            .name("paymentReader")
            .dataSource(dataSource)
            .sql("SELECT id, amount, fee, status FROM payments WHERE status = 'PENDING' ORDER BY id")
            .rowMapper((rs, rowNum) -> new Payment(
                    rs.getLong("id"),
                    rs.getBigDecimal("amount"),
                    rs.getBigDecimal("fee"),
                    rs.getString("status")))
            .build();
}

@Bean
public ItemProcessor<Payment, Settlement> paymentProcessor() {
    return payment -> {
        BigDecimal netAmount = payment.amount().subtract(payment.fee());
        return new Settlement(payment.id(), netAmount, LocalDate.now());
    };
}

@Bean
public JdbcBatchItemWriter<Settlement> settlementWriter(DataSource dataSource) {
    return new JdbcBatchItemWriterBuilder<Settlement>()
            .dataSource(dataSource)
            .sql("INSERT INTO settlements (payment_id, net_amount, settlement_date) VALUES (:paymentId, :netAmount, :settlementDate)")
            .beanMapped()
            .build();
}

Fault Tolerance: Skip and Retry Strategies

In production, failures are inevitable. A network timeout, a deadlock, a malformed record. Spring Batch provides two mechanisms: skip and retry. Skip is for non-fatal errors where you can safely ignore a record (e.g., a null field). Retry is for transient errors that might succeed on retry (e.g., network timeout). Configure both in the step builder. Important: skip and retry are applied at the chunk level. If an item fails during processing, the entire chunk is rolled back, and the failed item is excluded. The remaining items in the chunk are re-processed. This means your processor must be idempotent — calling it twice on the same item should produce the same result. Also, be careful with skipLimit: if you set it too high, you might skip legitimate errors. I recommend starting with 10 and monitoring. For retry, use exponential backoff via the RetryTemplate. Spring Batch 5 integrates with Spring Retry, so you can configure a RetryTemplate with backoff policy. Never retry on the same thread without backoff — you'll overwhelm the resource.

@Bean
public Step faultTolerantStep(JobRepository jobRepository,
                               PlatformTransactionManager transactionManager) {
    return new StepBuilder("faultTolerantStep", jobRepository)
            .<Payment, Settlement>chunk(500, transactionManager)
            .reader(paymentReader(dataSource))
            .processor(paymentProcessor())
            .writer(settlementWriter(dataSource))
            .faultTolerant()
            .skipLimit(10)
            .skip(DataIntegrityViolationException.class)
            .noSkip(NullPointerException.class)  // never skip NPE
            .retryLimit(3)
            .retry(NetworkTimeoutException.class)
            .backOff(new ExponentialBackOffPolicy())  // 1s, 2s, 4s
            .build();
}

Tasklet-Based Steps for One-Off Operations

Not every step needs chunk processing. Sometimes you need to perform a single operation: delete a temporary file, send a notification, or archive processed records. That's where Tasklet comes in. A Tasklet is a simple interface with one method: execute(). It returns RepeatStatus.FINISHED when done, or RepeatStatus.CONTINUABLE if you want it to repeat (rarely used). Tasklets run in a transaction — if the Tasklet throws an exception, the transaction rolls back. This is useful for operations that must be atomic. For example, after a chunk step processes payments, you might want to move the input file to an archive folder. If the move fails, you want the entire job to fail. Tasklets are also useful for cleanup steps before or after chunk processing. One caveat: Tasklets don't have built-in skip/retry logic. If you need fault tolerance in a Tasklet, you must implement it manually with try-catch. In practice, I use Tasklets only for operations that are idempotent and simple. For anything complex, use a chunk step with a reader that returns a single item.

@Component
public class ArchiveTasklet implements Tasklet {

    @Override
    public RepeatStatus execute(StepContribution contribution,
                                 ChunkContext chunkContext) throws Exception {
        String inputFile = (String) chunkContext.getStepContext()
                .getJobParameters().get("inputFile");
        Path source = Paths.get(inputFile);
        Path target = Paths.get("/archive/" + source.getFileName());
        try {
            Files.move(source, target, StandardCopyOption.ATOMIC_MOVE);
        } catch (IOException e) {
            throw new ArchiveException("Failed to archive file: " + inputFile, e);
        }
        return RepeatStatus.FINISHED;
    }
}

// In config:
@Bean
public Step archiveStep(JobRepository jobRepository,
                         PlatformTransactionManager transactionManager,
                         ArchiveTasklet archiveTasklet) {
    return new StepBuilder("archiveStep", jobRepository)
            .tasklet(archiveTasklet, transactionManager)
            .build();
}

Job Parameters, Execution Context, and Restartability

Job parameters are the input to your job — they differentiate job instances. Spring Batch uses them to determine if a job instance already exists. If you run a job with the same parameters twice, it returns the existing execution (unless you set 'run.id' parameter or use JobParametersIncrementer). The execution context stores state that survives restarts: reader position, processed count, etc. For chunk steps, Spring Batch manages this automatically. For Tasklets, you must save and restore state manually via StepExecution.getExecutionContext(). This is critical for restartability. If your Tasklet fails after moving 50 files, on restart it should skip those 50. You can store the list of processed files in the execution context. Also, never store large objects in the execution context — it's serialized to the database and can cause performance issues. I've seen a team store a 10MB JSON blob in the context, causing the BATCH_STEP_EXECUTION_CONTEXT table to bloat and queries to slow down. Use the context only for small state (strings, numbers, dates).

@Component
public class RestartableArchiveTasklet implements Tasklet {

    @Override
    public RepeatStatus execute(StepContribution contribution,
                                 ChunkContext chunkContext) throws Exception {
        StepExecution stepExecution = chunkContext.getStepContext().getStepExecution();
        ExecutionContext context = stepExecution.getExecutionContext();
        
        Set<String> archived = (Set<String>) context.get("archivedFiles");
        if (archived == null) {
            archived = new HashSet<>();
        }
        
        String inputFile = stepExecution.getJobParameters().getString("inputFile");
        if (!archived.contains(inputFile)) {
            // archive the file
            Files.move(Paths.get(inputFile), Paths.get("/archive/" + inputFile));
            archived.add(inputFile);
            context.put("archivedFiles", archived);
        }
        return RepeatStatus.FINISHED;
    }
}

Monitoring and Observability with Micrometer and Actuator

Spring Batch 5 integrates with Micrometer to expose metrics: items read, processed, written, skipped, and retried. These metrics are available via Spring Boot Actuator's /actuator/metrics endpoint. You can also expose them to Prometheus or Datadog. In production, I always create a custom metrics dashboard that shows: chunk processing time, skip rate, retry rate, and job duration. The most important metric is 'spring.batch.chunk.write.count' — if it drops to zero while reads are happening, you have a writer bottleneck. Also, monitor 'spring.batch.job.duration' to detect regressions. Spring Batch Admin (now deprecated) is replaced by Micrometer + Actuator. For alerting, set up a threshold on skip rate: if skip rate exceeds 1% of total items, trigger an alert. Also, log the job parameters and execution ID at the start and end of each job. This helps correlate logs with specific job runs. I've seen teams waste hours trying to debug a job without knowing which execution ID caused the issue.

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,prometheus
  metrics:
    tags:
      application: payment-settlement
spring:
  batch:
    job:
      enabled: true
    jdbc:
      initialize-schema: always
  datasource:
    url: jdbc:postgresql://localhost:5432/payments
    username: batch_user
    password: ${DB_PASSWORD}

Testing Batch Jobs with Spring Batch Test

Testing batch jobs is tricky because of the database state and transaction boundaries. Spring Batch Test provides the JobLauncherTestUtils and StepMetaDataObjectFactory to simplify testing. Always test with an embedded database (H2) that mirrors your production schema. Test three scenarios: the happy path (all items succeed), the skip path (some items fail), and the retry path (transient failure). For skip testing, create a reader that returns a mix of valid and invalid items. Verify that the skip count matches expectations. For retry testing, mock a writer that fails the first two times and succeeds on the third. Also, test restartability: start a job, simulate a failure, then restart and verify it picks up where it left off. Use JobLauncherTestUtils.launchJob() for full integration tests. For unit tests, test the processor and writer in isolation. One common mistake: not cleaning the database between tests. Use @SpringBatchTest which automatically cleans the batch tables. Also, set spring.batch.job.enabled=false in test properties to prevent jobs from running on startup.

@SpringBatchTest
@SpringBootTest
class PaymentSettlementJobTest {

    @Autowired
    private JobLauncherTestUtils jobLauncherTestUtils;

    @Test
    void testHappyPath() throws Exception {
        JobParameters params = new JobParametersBuilder()
                .addString("inputFile", "test-payments.csv")
                .addLong("run.id", 1L)
                .toJobParameters();
        JobExecution execution = jobLauncherTestUtils.launchJob(params);
        assertEquals(ExitStatus.COMPLETED, execution.getExitStatus());
    }

    @Test
    void testSkipScenario() throws Exception {
        // Inject a reader that throws DataIntegrityViolationException on every 10th item
        JobExecution execution = jobLauncherTestUtils.launchJob();
        assertEquals(ExitStatus.COMPLETED, execution.getExitStatus());
        assertEquals(5, execution.getStepExecutions().stream()
                .mapToInt(StepExecution::getSkipCount).sum());
    }
}