Laravel Jobs — Stop Silent Retry Storms

Every production Laravel app hits the same wall eventually: a controller action that sends a welcome email, resizes an uploaded avatar, and pings a third-party webhook starts taking 800ms or more. Users feel it, bounce rates climb, and your infrastructure bill quietly grows because slow requests hold PHP-FPM workers hostage. The fix isn't a faster server — it's getting that work out of the request lifecycle entirely.

Laravel's Queue system is the engine that makes that possible. It lets you serialise a 'job' — a self-contained unit of work — push it onto a queue driver (Redis, SQS, database, etc.), and have a long-running worker process pick it up milliseconds later, completely outside any HTTP request. The framework handles serialisation, retry logic, failure tracking, delayed dispatch, job chaining, and concurrent batches out of the box. But like any powerful engine, it has sharp edges that bite teams who don't understand what's happening under the hood.

By the end of this article you'll understand how the queue worker event loop actually works, why model serialisation silently causes stale-data bugs, how to tune concurrency without melting your database, how to use job batching for fan-out workflows, and what to monitor in production so failures surface before your users notice them.

What is Laravel Queues and Jobs?

Laravel Queues and Jobs is the framework's mechanism for deferring time-consuming tasks out of the HTTP request lifecycle. Instead of processing an email send or image resize inline, you package the work into a Job class and dispatch it. The queue worker — a long-running CLI process — pops jobs from a storage backend (Redis, database, Amazon SQS, or Beanstalkd), unserializes them, and executes the handle() method. This keeps your API responses snappy and allows horizontal scaling of background work.

A typical job class looks like this: it defines a handle() method that receives any dependencies via dependency injection. The constructor stores the data needed to run the job later (e.g., the model ID, not the entire model). You dispatch with dispatch() or dispatchIf() and can specify a queue name, delay, or whether it runs on a specific connection. The framework serializes the job instance to JSON using its __sleep/__wakeup or the SerializesModels trait — and it's that serialization that causes the most common production bugs.

<?php

namespace App\Jobs;

use App\Models\Podcast;
use App\Services\AudioProcessor;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class ProcessPodcast implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 5;                     // max attempts
    public $backoff = [10, 30, 60, 120];  // exponential backoff
    public $deleteWhenMissingModels = true;

    public function __construct(
        public int $podcastId,
    ) {}

    public function handle(AudioProcessor $processor): void
    {
        $podcast = Podcast::findOrFail($this->podcastId);
        $processor->transcode($podcast);
    }
}

How the Queue Worker Loop Actually Works

You run php artisan queue:work and suddenly jobs start getting processed. What's happening inside? The worker is an event loop that:

1. Pops a job from the queue driver (Redis: BLPOP, database: SELECT FOR UPDATE, SQS: ReceiveMessage).
2. Unserializes the JSON payload back into a job instance.
3. Fires the Queue.before event.
4. Calls the job's handle() method via the container.
5. If handle() throws an exception: checks retry count. If within $tries, releases the job back to the queue with a delay ($backoff or retry_after). If exceeded, moves the job to failed_jobs.
6. Fires the Queue.after event.
7. Loops to step 1.

Each iteration is called a 'job processing cycle'. The worker sleeps for the --sleep duration when the queue is empty. The --timeout option sets the maximum seconds allowed for one job to run. If a job exceeds the timeout, the worker kills the process and moves on — the job will be retried later (because it was never deleted).

Crucially, the worker does NOT restart between jobs. That means any memory leaked during a job's handle() accumulates. The --memory option forces a restart when the worker's memory exceeds that limit (in MB). For long-running workers, always set --memory=128 or similar.

// Example: supervisor config to run workers with memory guard
// /etc/supervisor/conf.d/laravel-worker.conf
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600 --memory=128
numprocs=4
autostart=true
autorestart=true
user=forge
stdout_logfile=/var/log/worker.log
stderr_logfile=/var/log/worker-error.log

Job Batching: Fan-Out and Aggregation Patterns

Laravel's job batch system lets you dispatch a group of jobs and execute a callback when all of them finish. This is ideal for workflows like: generate 100 report PDFs, then upload a combined archive. Or: process 10,000 records in chunks, then send a notification.

To use batching, define a batchable job that implements ShouldBeUnique — but that's not required. The key is to dispatch the batch via Bus::batch([...])->then(function(Batch $batch) { ... })->catch(...)->finally(...)->dispatch(). The batch ID is stored in the job_batches table. Each job in the batch can read $this->batch() to get the batch instance, and optionally cancel the whole batch by calling $this->batch()->cancel().

Important: batch progress is stored in the database, not in memory. That means if you restart the worker, the batch state persists. However, if the batch table is truncated mid-execution, the batch is lost and callbacks never run.

<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class ProcessChunk implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public array $recordIds) {}

    public function handle(): void
    {
        // If batch was cancelled, bail early
        if ($this->batch() && $this->batch()->cancelled()) {
            return;
        }

        // Process each record...
        foreach ($this->recordIds as $id) {
            // expensive operation
        }

        // Track progress on batch
        $this->batch()->incrementProgress(count($this->recordIds));
    }
}

// In controller:
use Bus;
$batch = Bus::batch([
    new ProcessChunk(range(1, 1000)),
    new ProcessChunk(range(1001, 2000)),
])->then(function () {
    // All chunks processed
})->catch(function () {
    // At least one chunk failed
})->finally(function () {
    // Always runs
})->dispatch();

Redis vs Database Queue Driver: Real-World Trade-offs

Laravel offers multiple queue drivers. The two most common for self-hosted apps are database and redis. Each has distinct operational characteristics.

Database driver: Uses a jobs table with SELECT ... FOR UPDATE for atomic pop. Simple to set up — no extra service required. But performance degrades under load because each pop acquires a database lock. For high-throughput apps (thousands of jobs per second), the database quickly becomes the bottleneck. Also, delayed jobs require WHERE available_at <= NOW() queries, which can be slow without proper indexing.

Redis driver: Uses Redis lists and blocking pop (BLPOP) for near-instant job retrieval. Supports priority queues natively via ZADD with score. Delayed jobs are stored in sorted sets and only become available after the score time passes. Redis handles high throughput easily, and with Redis Sentinel or Cluster you get HA. The downside: Redis memory is limited. If jobs pile up and memory fills, Redis starts evicting keys. Configure maxmemory-policy noeviction or use a separate Redis instance for queues.

Hybrid approach: Use Redis for your fast queue and database for your reliable queue. The database driver can fall back if Redis goes down.

'connections' => [
    'redis' => [
        'driver' => 'redis',
        'connection' => 'queue',           // separate Redis connection
        'queue' => 'default',
        'retry_after' => 90,
        'block_for' => null,
        'after_commit' => true,
    ],

    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
        'after_commit' => true,
    ],
];

Production Gotchas: Stale Models, Exceptions, and Monitoring

The most common production queue failures all stem from the same root: the world changes between when you dispatch a job and when a worker processes it.

Stale Models: If you pass a model instance to a job, SerializesModels will reload it from the database on handle(). But if another process deleted that model, you get a ModelNotFoundException. Alternatively, if the model's attributes changed (e.g., user email updated), the job uses stale data. Fix: always pass the model ID and query fresh in handle(). Or use $deleteWhenMissingModels = true to auto-cancel stale jobs.

Unhandled Exceptions: If your job throws an exception you don't catch, the worker will retry according to $tries. Eventually the job goes to failed_jobs. But if you don't have a failed job handler (queue:failed-table migration not run), the job is just lost. Always run php artisan queue:failed-table and set up a listener for Queue::failing() to alert your team.

Worker Crashes: If the worker process is killed (OOM, deploy, supervisor restart) while running a job, that job is considered released — it will be retried after retry_after seconds. If you have multiple workers, another worker may pick it up before the original worker finishes, causing duplicate processing. Mitigate by using a job lock or idempotency key in the job's data.

Queue Length Bloat: Monitor queue size with a tool like Horizon or a custom metric (Laravel Pulse, Prometheus). If a queue grows faster than workers can drain it, you need more workers or a faster driver.

use Illuminate\Queue\Events\JobFailed;
use Illuminate\Queue\QueueManager;
use Illuminate\Support\Facades\Log;

// In boot() method:
Queue::failing(function (JobFailed $event) {
    Log::critical('Job failed', [
        'job' => $event->job->resolveName(),
        'exception' => $event->exception->getMessage(),
        'payload' => $event->job->getRawBody(),
    ]);

    // Send alert to Slack/PagerDuty
    // e.g., Slack::send("Job {$event->job->resolveName()} failed: {$event->exception->getMessage()}");
});

Frequently Asked Questions