Laravel Jobs — Stop Silent Retry Storms

Every production Laravel app hits the same wall eventually: a controller action that sends a welcome email, resizes an uploaded avatar, and pings a third-party webhook starts taking 800ms or more. Users feel it, bounce rates climb, and your infrastructure bill quietly grows because slow requests hold PHP-FPM workers hostage. The fix isn't a faster server — it's getting that work out of the request lifecycle entirely.

Laravel's Queue system is the engine that makes that possible. It lets you serialise a 'job' — a self-contained unit of work — push it onto a queue driver (Redis, SQS, database, etc.), and have a long-running worker process pick it up milliseconds later, completely outside any HTTP request. The framework handles serialisation, retry logic, failure tracking, delayed dispatch, job chaining, and concurrent batches out of the box. But like any powerful engine, it has sharp edges that bite teams who don't understand what's happening under the hood.

By the end of this article you'll understand how the queue worker event loop actually works, why model serialisation silently causes stale-data bugs, how to tune concurrency without melting your database, how to use job batching for fan-out workflows, and what to monitor in production so failures surface before your users notice them.

What is Laravel Queues and Jobs?

Laravel Queues and Jobs is the framework's mechanism for deferring time-consuming tasks out of the HTTP request lifecycle. Instead of processing an email send or image resize inline, you package the work into a Job class and dispatch it. The queue worker — a long-running CLI process — pops jobs from a storage backend (Redis, database, Amazon SQS, or Beanstalkd), unserializes them, and executes the handle() method. This keeps your API responses snappy and allows horizontal scaling of background work.

A typical job class looks like this: it defines a handle() method that receives any dependencies via dependency injection. The constructor stores the data needed to run the job later (e.g., the model ID, not the entire model). You dispatch with dispatch() or dispatchIf() and can specify a queue name, delay, or whether it runs on a specific connection. The framework serializes the job instance to JSON using its __sleep/__wakeup or the SerializesModels trait — and it's that serialization that causes the most common production bugs.

<?php

namespace App\Jobs;

use App\Models\Podcast;
use App\Services\AudioProcessor;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class ProcessPodcast implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $tries = 5;                     // max attempts
    public $backoff = [10, 30, 60, 120];  // exponential backoff
    public $deleteWhenMissingModels = true;

    public function __construct(
        public int $podcastId,
    ) {}

    public function handle(AudioProcessor $processor): void
    {
        $podcast = Podcast::findOrFail($this->podcastId);
        $processor->transcode($podcast);
    }
}

How the Queue Worker Loop Actually Works

You run php artisan queue:work and suddenly jobs start getting processed. What's happening inside? The worker is an event loop that:

1. Pops a job from the queue driver (Redis: BLPOP

// Example: supervisor config to run workers with memory guard
// /etc/supervisor/conf.d/laravel-worker.conf
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600 --memory=128
numprocs=4
autostart=true
autorestart=true
user=forge
stdout_logfile=/var/log/worker.log
stderr_logfile=/var/log/worker-error.log

Worker Optimization: Key CLI Options

Laravel's queue:work command comes with a dozen CLI options that directly control worker behavior. Misconfiguring these is the most common cause of production queue meltdowns. The table below summarizes the critical flags and their production impact. Always pair these with Supervisor process management for automatic restart and concurrency.

Horizontal Scaling: Multiple Workers and Queue Topology

When a single worker can't keep up — you see queue depth growing and jobs taking minutes to process — the solution is horizontal scaling. Laravel supports multiple workers reading from the same queue driver natively. Each worker is an independent PHP process, often managed by Supervisor. You can also run workers on separate servers, all pointing to the same Redis or SQS queue. The diagram below shows a typical production topology with a load balancer distributing traffic, a shared Redis queue, and multiple worker servers consuming jobs.

Key scaling considerations: Workers are stateless and compete for jobs. There's no built-in worker coordination — each worker pops the next available job. This means you can scale out arbitrarily, but you must ensure the queue driver can handle the concurrency. Redis handles high connection counts well; the database driver may hit lock contention. For SQS, set the visibility timeout to at least 5x your maximum job runtime to prevent duplicate processing when workers are scaled up quickly.

Job Batching: Fan-Out and Aggregation Patterns

Laravel's job batch system lets you dispatch a group of jobs and execute a callback when all of them finish. This is ideal for workflows like: generate 100 report PDFs, then upload a combined archive. Or: process 10,000 records in chunks, then send a notification.

To use batching, define a batchable job that implements ShouldBeUnique — but that's not required. The key is to dispatch the batch via Bus::batch([...])->then(function(Batch $batch) { ... })->catch(...)->finally(...)->dispatch(). The batch ID is stored in the job_batches table. Each job in the batch can read $this->batch() to get the batch instance, and optionally cancel the whole batch by calling $this->batch()->cancel().

Important: batch progress is stored in the database, not in memory. That means if you restart the worker, the batch state persists. However, if the batch table is truncated mid-execution, the batch is lost and callbacks never run.

<?php

namespace App\Jobs;

use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class ProcessChunk implements ShouldQueue
{
    use Batchable, Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(public array $recordIds) {}

    public function handle(): void
    {
        // If batch was cancelled, bail early
        if ($this->batch() && $this->batch()->cancelled()) {
            return;
        }

        // Process each record...
        foreach ($this->recordIds as $id) {
            // expensive operation
        }

        // Track progress on batch
        $this->batch()->incrementProgress(count($this->recordIds));
    }
}

// In controller:
use Bus;
$batch = Bus::batch([
    new ProcessChunk(range(1, 1000)),
    new ProcessChunk(range(1001, 2000)),
])->then(function () {
    // All chunks processed
})->catch(function () {
    // At least one chunk failed
})->finally(function () {
    // Always runs
})->dispatch();

Job Middleware: Rate Limiting, Logging, and Custom Guards

Job middleware in Laravel allow you to wrap the execution of a job with custom logic — similar to HTTP middleware. This is useful for cross-cutting concerns like rate limiting, logging start/end times, or checking a circuit breaker before running. You apply middleware by defining a method on the job class that returns an array of middleware objects. Laravel ships with a RateLimited middleware that uses the cache rate limiter to prevent jobs from hitting an API too frequently.

Middleware runs outside the job's handle() method. If middleware throws an exception (e.g., rate limit hit), the job is not executed and will be released back to the queue after a delay. This is more efficient than checking rate limits inside handle() because the job is never actually popped if the rate limit is exceeded — preventing wasted effort.

You can also create custom middleware. For example, a middleware that checks if a maintenance flag is set, or a middleware that logs job execution time to Prometheus.

<?php

namespace App\Jobs\Middleware;

use Illuminate\Support\Facades\Redis;

class RateLimitMiddleware
{
    public function __construct(
        protected string $key,
        protected int $maxAttempts = 10,
        protected int $decaySeconds = 60,
    ) {}

    public function handle($job, $next)
    {\n        $current = Redis::incr($this->key);\n        if ($current === 1) {\n            Redis::expire($this->key, $this->decaySeconds);\n        }

        if ($current > $this->maxAttempts) {
            $job->release($this->decaySeconds);
            return;
        }

        $next($job);
    }
}

// In job class:
public function middleware(): array
{
    return [
        new RateLimitMiddleware('external-api:'.$this->modelId, 10, 60),
    ];
}

Redis vs Database Queue Driver: Real-World Trade-offs

Laravel offers multiple queue drivers. The two most common for self-hosted apps are database and redis. Each has distinct operational characteristics.

Database driver: Uses a jobs table with SELECT ... FOR UPDATE for atomic pop. Simple to set up — no extra service required. But performance degrades under load because each pop acquires a database lock. For high-throughput apps (thousands of jobs per second), the database quickly becomes the bottleneck. Also, delayed jobs require WHERE available_at <= NOW() queries, which can be slow without proper indexing.

Redis driver: Uses Redis lists and blocking pop (BLPOP) for near-instant job retrieval. Supports priority queues natively via ZADD with score. Delayed jobs are stored in sorted sets and only become available after the score time passes. Redis handles high throughput easily, and with Redis Sentinel or Cluster you get HA. The downside: Redis memory is limited. If jobs pile up and memory fills, Redis starts evicting keys. Configure maxmemory-policy noeviction or use a separate Redis instance for queues.

Hybrid approach: Use Redis for your fast queue and database for your reliable queue. The database driver can fall back if Redis goes down.

'connections' => [
    'redis' => [
        'driver' => 'redis',
        'connection' => 'queue',           // separate Redis connection
        'queue' => 'default',
        'retry_after' => 90,
        'block_for' => null,
        'after_commit' => true,
    ],

    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
        'after_commit' => true,
    ],
];

Queue Driver Performance: Throughput and Latency Benchmarks

Choosing the right queue driver for your workload requires understanding real-world performance. The numbers below come from production benchmarks on a standard 8-core VM with 16GB RAM, running 4 workers consuming jobs that do a 50ms sleep (simulating I/O). All drivers were configured with default Laravel settings except where noted.

Key takeaways: Redis delivers the lowest latency and highest throughput for most workloads. Database driver is fine for low-volume applications but bottlenecks quickly under concurrent writes. SQS adds network latency but offers infinite scalability and durability. For CPU-bound jobs (processing-heavy), throughput is limited by CPU, not queue driver.

Use this table to estimate whether your current driver will handle your peak load. If your job arrival rate exceeds the driver's throughput, you'll need to either add more workers (if the driver can handle more connections) or switch to a faster driver.

Production Gotchas: Stale Models, Exceptions, and Monitoring

The most common production queue failures all stem from the same root: the world changes between when you dispatch a job and when a worker processes it.

Stale Models: If you pass a model instance to a job, SerializesModels will reload it from the database on handle(). But if another process deleted that model, you get a ModelNotFoundException. Alternatively, if the model's attributes changed (e.g., user email updated), the job uses stale data. Fix: always pass the model ID and query fresh in handle(). Or use $deleteWhenMissingModels = true to auto-cancel stale jobs.

Unhandled Exceptions: If your job throws an exception you don't catch, the worker will retry according to $tries. Eventually the job goes to failed_jobs. But if you don't have a failed job handler (queue:failed-table migration not run), the job is just lost. Always run php artisan queue:failed-table and set up a listener for Queue::failing() to alert your team.

Worker Crashes: If the worker process is killed (OOM, deploy, supervisor restart) while running a job, that job is considered released — it will be retried after retry_after seconds. If you have multiple workers, another worker may pick it up before the original worker finishes, causing duplicate processing. Mitigate by using a job lock or idempotency key in the job's data.

Queue Length Bloat: Monitor queue size with a tool like Horizon or a custom metric (Laravel Pulse, Prometheus). If a queue grows faster than workers can drain it, you need more workers or a faster driver.

use Illuminate\Queue\Events\JobFailed;
use Illuminate\Queue\QueueManager;
use Illuminate\Support\Facades\Log;

// In boot() method:
Queue::failing(function (JobFailed $event) {
    Log::critical('Job failed', [\n        'job' => $event->job->resolveName(),\n        'exception' => $event->exception->getMessage(),\n        'payload' => $event->job->getRawBody(),\n    ]);

    // Send alert to Slack/PagerDuty
    // e.g.

Why You Actually Need Queues (Stop Pretending It's Optional)

Every Laravel app starts simple. You fire off a mail, resize an image, call an API — and your user stares at a spinner for 3 seconds. That's not a feature, it's a bug.

Queues exist to decouple your HTTP response from work that doesn't need to finish right now. Shoving work into a queue drops your response time from 8 seconds to under 100ms. That's the difference between a user bouncing or converting.

Fault tolerance isn't a luxury — it's a production survival skill. When your payment gateway goes down, a queued job retries automatically instead of showing a 500 page. Your users never see the crash.

Scalability is the real payoff. With a Redis-backed queue, you can spin up ten workers on one box, then throw fifty more across three servers when Black Friday hits. No code changes. No downtime. Just raw throughput.

The alternative? Your web request thread blocks while you wait for an SMTP handshake. That's amateur hour.

Stop treating queues as advanced architecture. They're the baseline for any app that touches a network or does more than a SELECT query.

// io.thecodeforge — php tutorial

// Before queues — your user waits for this
public function register(Request $request)
{
    $user = User::create($request->validated());
    Mail::to($user)->send(new WelcomeMail($user));
    return redirect()->route('dashboard');
    // 4.2 seconds later...
}

// After queues — 80ms response
public function register(Request $request)
{
    $user = User::create($request->validated());
    SendWelcomeMail::dispatch($user);
    return redirect()->route('dashboard');
    // 80ms. User clicks, page loads instantly
}

Setting Up Queues Without the Fluff (Database Driver in 60 Seconds)

You don't need Redis to start. The database driver gets you 80% of the value with zero ops overhead. Here's the exact sequence.

Set your env. Run the migration. Create a job. Dispatch it. Done.

The queue:table migration creates a simple table that stores serialized job data. Your worker polls it every few seconds — that's the whole loop.

Most tutorials stop here. They should tell you the critical config: notBeforeSeconds. If you're dispatching a job that needs data from a freshly committed transaction, slap a 2-second delay. Otherwise your worker grabs the job before the database index flushes.

Also: run php artisan queue:work with --tries=3 from day one. No tries means a failed job sits in the table forever, silently eating memory. You will find this at 3 AM after a runaway job fills your disk.

Database queue is fine for 10,000 jobs/day. For 100,000+ jobs/day, switch to Redis. But don't optimise before you measure.

// io.thecodeforge — php tutorial

# .env
QUEUE_CONNECTION=database

# Terminal
php artisan queue:table
php artisan migrate
php artisan make:job ProcessImageResize

# Terminal — start worker
php artisan queue:work --tries=3 --delay=5

# Dispatching in controller
namespace App\Http\Controllers;

use App\Jobs\ProcessImageResize;

class UploadController
{
    public function store(Request $request)
    {
        $path = $request->file('avatar')->store('avatars');
        ProcessImageResize::dispatch($path)
            ->delay(now()->addSeconds(2));
        return back()->with('status', 'Upload queued');
    }
}

Dispatching Jobs Like a Pro (Dispatchable, Delay, and Chain Pitfalls)

You dispatch a job with ::dispatch(). But the method you chain after it matters.

dispatch() returns nothing. You cannot catch its result. It's fire-and-forget — the worker runs it later, on a different process.

dispatchIf() and dispatchUnless() let you gate behind booleans. Use them in controller logic to avoid if blocks that make intention unclear.

Delay is not a timeout. Calling ->delay(5) tells Laravel "release this job to the queue 5 seconds from now". It does not fail the job if it runs long. That's a separate retryUntil() method.

Job chaining via Bus::chain() is the hidden gem. Three jobs run in sequence. If step 2 fails, step 3 never fires. Perfect for payment flows: validate, capture, send receipt. Don't chain unrelated jobs; one failure kills the whole chain.

On queue fail — never swallow exceptions in your job's handle() method. That hides errors from Horizon and your monitoring. Let it throw. Let it be caught by the retry logic. You want to know when a job fails, not pretend it succeeded.

// io.thecodeforge — php tutorial

// Conditional dispatch
ProcessPayment::dispatchIf($order->total > 0, $order);

// Delayed dispatch + timeout
ProcessOrderExport::dispatch($order)
    ->delay(now()->addMinutes(15))
    ->retryUntil(now()->addHours(2));

// Job chain — step by step failure
use Illuminate\Support\Facades\Bus;

Bus::chain([
    new ValidateInventory($order),
    new ChargeCustomer($order),
    new SendConfirmation($order),
])->dispatch();

// DO NOT do this — swallows errors
public function handle()
{
    try {
        // risky operation
    } catch (\Exception $e) {
        // nothing — job shows as "processed"
    }
}

// DO THIS — let it fail, retry logic handles it
public function handle()
{
    // risky operation — throws naturally
}