Mid-level 6 min · March 06, 2026

Cron Jobs in Linux — Why Your Backup Script Ran Twice

Cron fires jobs even if previous runs are still active.

N
Naren · Founder
Plain-English first. Then code. Then the interview question.
About
 ● Production Incident 🔎 Debug Guide
⚡Quick Answer
  • Cron is a time-based job scheduler that runs commands at specified intervals using a daemon (crond).
  • Five-field syntax: minute hour day-of-month month day-of-week — each field accepts numbers, ranges, steps, or wildcards.
  • Crontab files hold job definitions: user crontabs (crontab -e) and system crontabs (/etc/crontab with an extra user field).
  • Key pitfall: cron runs jobs with minimal environment — never assume PATH or env variables without explicit export.
  • Performance insight: cron wakes every minute to check schedules — O(1) overhead per job, but overlapping runs corrupt data.
  • Production insight: silent failures dominate — without logging redirection (>> log 2>&1) a crashed job produces zero feedback.
Plain-English First

Imagine you set a reminder on your phone to water your plants every Monday at 8am. You don't have to remember it — your phone just does it automatically, every single week, even while you're asleep. Cron is exactly that, but for your Linux server. You tell it 'run this script at 2am every night', and it does it forever without you lifting a finger. That's it. A tireless, never-forgetting robot assistant built into every Linux system.

Every production system has a graveyard of tasks that someone used to do manually — rotating log files, sending weekly reports, backing up databases, clearing temp folders. Done by hand, these tasks get forgotten, delayed, or skipped on holidays. Done wrong, they take down services. Cron is the Linux answer to this problem: a built-in scheduler that's been running quietly on Unix systems since 1975, and still powers millions of automated workflows today.

The core problem cron solves is reliability. Humans forget. Cron doesn't. If you need something to happen at a predictable time — daily, hourly, every 15 minutes, or at 3:47am on the last day of the month — cron handles it without a process manager, without a paid SaaS tool, and without a single line of application code. It lives at the OS level, which means it works regardless of what language your app is written in or whether your app is even running.

By the end of this article you'll be able to write cron expressions confidently, manage crontab files without breaking things, debug jobs that silently fail, and apply the real-world patterns that DevOps engineers actually use in production. You'll also know the three mistakes that catch almost everyone the first time they use cron — and exactly how to avoid them.

How Cron Actually Works — The Daemon, the Crontab, and the Schedule

Cron is a daemon — a background process that starts when your system boots and never stops. Its name comes from 'chronos', the Greek word for time. Every minute, the cron daemon wakes up, checks all the crontab files on the system, and asks: 'Is there anything I should run right now?' If yes, it fires the job. Then it goes back to sleep until the next minute.

A crontab (cron table) is just a plain text file that lists scheduled jobs. Each line in a crontab is one job: five time fields followed by the command to run. You never edit this file directly — you use the crontab command, which validates the format before saving, protecting you from syntax errors that would silently break everything.

There are two types of crontabs you'll work with. The first is user crontabs — each Linux user has their own, and jobs run as that user. The second is the system crontab at /etc/crontab and files dropped into /etc/cron.d/, which include an extra field specifying which user to run the job as. For most application-level automation, user crontabs are the right choice. For system-level jobs like log rotation, the system crontab is used.

The key mental model: cron doesn't track whether a previous job finished. If your job takes longer than its schedule interval, you can end up with two copies running at the same time. That's one of the most dangerous production gotchas, and we'll cover how to handle it.

crontab_basics.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# ─────────────────────────────────────────────────
# MANAGING YOUR CRONTAB — the essential commands
# ─────────────────────────────────────────────────

# Open your crontab in the default editor (usually nano or vi)
# This is the ONLY safe way to edit your crontab
crontab -e

# List all your current cron jobs — great for auditing
crontab -l

# Remove ALL your cron jobs (dangerous — no confirmation prompt!)
crontab -r

# Edit the crontab for a specific user (must run as root)
crontab -u deploy_user -e

# ─────────────────────────────────────────────────
# ANATOMY OF A CRON EXPRESSION
# ─────────────────────────────────────────────────
# Each job line follows this exact structure:
#
# ┌──────────── minute        (0 - 59)
# │  ┌─────────── hour          (0 - 23)
# │  │  ┌──────────── day of month  (1 - 31)
# │  │  │  ┌─────────── month         (1 - 12)
# │  │  │  │  ┌──────────── day of week   (0 - 7, both 0 and 7 = Sunday)
# │  │  │  │  │
# *  *  *  *  *  command_to_run

# ─────────────────────────────────────────────────
# REAL EXAMPLES with plain-English explanations
# ─────────────────────────────────────────────────

# Run database backup every day at 2:30am
30 2 * * * /opt/scripts/backup_postgres.sh

# Clear the application temp directory every hour
0 * * * * rm -rf /var/app/tmp/*

# Send a weekly analytics report every Monday at 9am
0 9 * * 1 /opt/scripts/send_weekly_report.py

# Run a health check every 15 minutes
*/15 * * * * /opt/scripts/health_check.sh

# Archive logs on the 1st of every month at midnight
0 0 1 * * /opt/scripts/archive_logs.sh

# Run a job only on weekdays (Mon-Fri) at 6am
0 6 * * 1-5 /opt/scripts/sync_business_data.sh
Output
# Output of: crontab -l
30 2 * * * /opt/scripts/backup_postgres.sh
0 * * * * rm -rf /var/app/tmp/*
0 9 * * 1 /opt/scripts/send_weekly_report.py
*/15 * * * * /opt/scripts/health_check.sh
0 0 1 * * /opt/scripts/archive_logs.sh
0 6 * * 1-5 /opt/scripts/sync_business_data.sh
Pro Tip: Use crontab.guru
Before you save any cron expression, paste it into crontab.guru — a free visual editor that translates your expression into plain English in real time. It's saved countless engineers from scheduling jobs at the wrong time.
Production Insight
Cron's per-minute wake cycle adds ~0% CPU overhead, but if your job pool grows beyond a few hundred, consider staggering start times to avoid a thundering herd at minute boundaries.
The @reboot nickname is great but remember it runs before network interfaces are always ready — add a sleep 10 if your job depends on external services.
Rule: always verify with crontab -l after editing — one stray space can shift your schedule by an entire day.
Key Takeaway
Cron is a simple line-by-line scheduler — five fields plus a command.
It runs as whatever user owns the crontab, with a stripped-down environment.
The most dangerous assumption: that cron knows about your shell customizations.
Choosing the Right Crontab Type
IfJob runs as a single non-root user (e.g., deploy)
→
UseUse a user crontab via crontab -e — simpler, no need to specify user.
IfJob needs to run as different users at different times
→
UseUse system crontab /etc/crontab with the extra user field.
IfJob is a system utility installed via package (e.g., logrotate)
→
UseDrop a script into /etc/cron.daily/ — no crontab editing needed.

Writing Production-Ready Cron Jobs — Logging, Environments, and Locking

Here's where most tutorials stop — and where most real problems start. A cron job that runs date will work fine. A cron job that runs your Python script will almost certainly fail silently the first time, and here's why: cron runs with a minimal environment. It doesn't load your .bashrc, .bash_profile, or any of the environment variables you set in your shell session. That means PATH, PYTHONPATH, NODE_ENV, database credentials, API keys — none of it is there unless you explicitly provide it.

The second production concern is logging. By default, cron swallows all output. If your script crashes, you'll never know unless you've set up logging. The fix is simple: redirect both stdout and stderr to a log file on every single cron job.

The third concern is job overlap. If your backup script takes 90 minutes but runs every hour, you'll eventually have two copies fighting over the same files. The solution is a lock file — a mechanism where the script checks if another copy of itself is already running and exits gracefully if so. The flock utility makes this one line.

These three patterns — explicit environment, output logging, and job locking — are what separate a toy cron job from a production one. The example below shows all three working together in a realistic database backup script.

backup_postgres.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
#!/bin/bash
# ─────────────────────────────────────────────────────────────────
# backup_postgres.sh
# Production-ready cron job: PostgreSQL backup with logging + locking
# Scheduled via crontab to run daily at 2:30am:
#   30 2 * * * /bin/bash /opt/scripts/backup_postgres.sh
# ─────────────────────────────────────────────────────────────────

# ── 1. EXPLICIT ENVIRONMENT ──────────────────────────────────────
# Cron's PATH is minimal (/usr/bin:/bin). Set it explicitly so
# commands like pg_dump, aws, gzip are all found correctly.
export PATH="/usr/local/bin:/usr/bin:/bin"

# Load app-specific environment variables from a secure file
# This file contains DB_USER, DB_NAME, S3_BUCKET etc.
# Permissions on this file should be 600 (owner read/write only)
source /etc/app/backup.env

# ── 2. LOGGING SETUP ─────────────────────────────────────────────
LOG_DIR="/var/log/app_backups"
LOG_FILE="${LOG_DIR}/backup_$(date +%Y-%m-%d).log"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

# Create log directory if it doesn't exist
mkdir -p "$LOG_DIR"

# Redirect ALL output (stdout + stderr) to the log file
# The tee command also prints to the terminal when run manually
exec > >(tee -a "$LOG_FILE") 2>&1

echo "[$TIMESTAMP] ─── Backup job started ───"

# ── 3. JOB LOCKING — prevent overlapping runs ────────────────────
LOCK_FILE="/tmp/backup_postgres.lock"

# flock acquires an exclusive lock on the lock file.
# -n means "non-blocking" — if the lock is already held, exit immediately.
# 9 is the file descriptor we're using for the lock.
exec 9>"$LOCK_FILE"
if ! flock -n 9; then
    echo "[$TIMESTAMP] Another backup is already running. Exiting."
    exit 1
fi

# ── 4. THE ACTUAL WORK ───────────────────────────────────────────
BACKUP_FILENAME="${DB_NAME}_$(date +%Y%m%d_%H%M%S).sql.gz"
BACKUP_PATH="/tmp/${BACKUP_FILENAME}"

echo "[$TIMESTAMP] Dumping database: $DB_NAME"

# pg_dump connects to PostgreSQL and streams SQL to gzip for compression
# PGPASSWORD is read from the sourced env file above
pg_dump -U "$DB_USER" -h "$DB_HOST" "$DB_NAME" | gzip > "$BACKUP_PATH"

# Check if the dump succeeded — never assume it worked
if [ $? -ne 0 ]; then
    echo "[$TIMESTAMP] ERROR: pg_dump failed! Aborting upload."
    exit 1
fi

echo "[$TIMESTAMP] Dump complete. Uploading to S3: s3://${S3_BUCKET}/"

# Upload to S3 — credentials come from the sourced env file
aws s3 cp "$BACKUP_PATH" "s3://${S3_BUCKET}/postgres-backups/${BACKUP_FILENAME}"

if [ $? -eq 0 ]; then
    echo "[$TIMESTAMP] Upload successful. Cleaning up local file."
    rm -f "$BACKUP_PATH"
else
    echo "[$TIMESTAMP] ERROR: S3 upload failed! Local backup retained at $BACKUP_PATH"
    exit 1
fi

echo "[$TIMESTAMP] ─── Backup job completed successfully ───"
Output
[2024-03-15 02:30:01] ─── Backup job started ───
[2024-03-15 02:30:01] Dumping database: production_db
[2024-03-15 02:31:47] Dump complete. Uploading to S3: s3://my-company-backups/
upload: /tmp/production_db_20240315_023001.sql.gz to s3://my-company-backups/postgres-backups/production_db_20240315_023001.sql.gz
[2024-03-15 02:32:03] Upload successful. Cleaning up local file.
[2024-03-15 02:32:03] ─── Backup job completed successfully ───
Watch Out: Silent Failures Are Cron's Default
Without output redirection, a failing cron job produces zero feedback. Add >> /var/log/myjob.log 2>&1 to every cron line as your absolute minimum. Better yet, build logging directly into the script itself as shown above — that way you get it whether the job is triggered by cron or run manually.
Production Insight
Never store database credentials inside the script file — source them from a separate env file with 600 permissions.
A missing $? check after pg_dump can silently produce empty backups for weeks.
Rule: every script must have exit codes greater than 0 on failure so cron (or a wrapper) can track them via MAILTO or external monitoring.
Key Takeaway
Three things separate a production cron job from a toy: explicit environment, full logging, and locking.
Without all three, you're one silent failure away from a pager at 3am.
The script pattern above works for any language — just replace the work section.
Choosing the Right Cron Job Structure
IfScript is short (<10 lines) and has no dependencies
→
UseInline command in crontab is fine with >> /var/log/job.log 2>&1.
IfScript is complex, uses env variables, files, or external tools
→
UseWrite a separate script file with full environment setup and logging — as shown in the example above.
IfJob may take longer than its interval
→
UseMust include flock locking — never write a job that can overlap.

Special Schedules, System Crontabs, and When to Use Alternatives

Once you're comfortable with the five-field syntax, there are shorthand strings that make common schedules far more readable. Instead of 0 0 * you can write @daily. These are called cron nicknames and every modern cron daemon supports them. They're self-documenting and much harder to misread.

Beyond user crontabs, Linux ships with a set of system-managed cron directories. Drop an executable script into /etc/cron.daily/ and it will run once a day — no crontab editing required. The actual run times are controlled by the run-parts entries in /etc/crontab. These are perfect for system maintenance tasks packaged by software installers.

That said, cron isn't always the right tool. It has real limitations: it has no dependency management (it can't wait for Job A to finish before starting Job B), it doesn't retry on failure, it has no built-in alerting, and it doesn't scale across multiple machines. For anything more complex — multi-step pipelines, distributed systems, or jobs that need retry logic — tools like systemd timers, Apache Airflow, or cloud-native schedulers (AWS EventBridge, GCP Cloud Scheduler) are better fits. Knowing when NOT to use cron is as important as knowing how to use it.

cron_advanced_patterns.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# ─────────────────────────────────────────────────────────────────
# CRON NICKNAMES — readable shorthand for common schedules
# ─────────────────────────────────────────────────────────────────

@reboot   /opt/scripts/start_queue_worker.sh   # Run ONCE at system startup
@hourly   /opt/scripts/refresh_cache.sh         # Same as: 0 * * * *
@daily    /opt/scripts/cleanup_sessions.sh      # Same as: 0 0 * * *
@weekly   /opt/scripts/generate_sitemap.sh      # Same as: 0 0 * * 0
@monthly  /opt/scripts/invoice_generator.sh     # Same as: 0 0 1 * *
@yearly   /opt/scripts/archive_old_records.sh   # Same as: 0 0 1 1 *

# ─────────────────────────────────────────────────────────────────
# SYSTEM CRONTAB (/etc/crontab) — has an extra USER field
# ─────────────────────────────────────────────────────────────────
# Format: minute  hour  day  month  weekday  USER  command

# Run log rotation as root every day at 6:25am
25 6 * * *   root    /usr/sbin/logrotate /etc/logrotate.conf

# Run the app's data sync as the 'deploy' user at midnight
0  0 * * *   deploy  /opt/app/bin/sync_data.sh

# ─────────────────────────────────────────────────────────────────
# RUNNING A CRON JOB MANUALLY FOR TESTING
# ─────────────────────────────────────────────────────────────────
# The most reliable way to test a cron job is to simulate
# cron's environment: no shell customizations, minimal PATH.

# Run your script exactly as cron would — bare environment
env -i HOME=/root LOGNAME=root PATH=/usr/bin:/bin \
    /bin/bash /opt/scripts/backup_postgres.sh

# ─────────────────────────────────────────────────────────────────
# CHECKING CRON LOGS — when a job runs but you don't see output
# ─────────────────────────────────────────────────────────────────

# On systemd-based systems (Ubuntu 20+, RHEL 8+)
journalctl -u cron --since "1 hour ago"

# On older systems using syslog
grep CRON /var/log/syslog | tail -50

# On Red Hat / CentOS systems
grep CRON /var/log/cron | tail -50
Output
# journalctl -u cron --since "1 hour ago" output:
Mar 15 02:30:01 prod-server CRON[14823]: (deploy) CMD (/opt/scripts/backup_postgres.sh)
Mar 15 03:00:01 prod-server CRON[15102]: (root) CMD (/usr/sbin/logrotate /etc/logrotate.conf)
Mar 15 03:15:01 prod-server CRON[15341]: (deploy) CMD (/opt/scripts/health_check.sh)
Interview Gold: @reboot Is Underused
The @reboot nickname runs a command once when the system starts. It's perfect for starting background workers, mounting drives, or seeding caches after a reboot — and interviewers love asking whether you know it exists. Unlike init scripts, it requires zero configuration beyond a single crontab line.
Production Insight
When you drop a script into /etc/cron.daily/, it runs at the system's configured time (often 6:25am via anacron or run-parts). If multiple day jobs take longer than 24h, they'll backlog and eventually crash the system.
Systemd timers offer OnCalendar= which is more precise than cron's minute granularity, and they can catch up missed runs with Persistent=true.
Rule: if you need retries or dependency chains, don't try to simulate them in cron — switch to a proper workflow engine.
Key Takeaway
Cron nicknames make schedules readable — use @daily over 0 0 *.
System crontabs are for system-level jobs, not application scripts.
If you need retries, dependencies, or distributed execution, cron is the wrong tool.
When to Use Alternatives to Cron
IfJob chain with dependencies (Job B only if Job A succeeded)
→
UseUse systemd timer with After= or a workflow engine like Airflow.
IfJob needs retry on failure (e.g., network-bound API call)
→
UseWrap the command in a retry loop, or use systemd Restart=on-failure.
IfJob runs across multiple servers (e.g., once in the fleet)
→
UseUse a distributed scheduler like AWS EventBridge or a CRDT-based approach.
IfSimple periodic command, no dependencies
→
UseCron is perfect — one line, no overhead.

Debugging Cron Jobs — Diagnosing Silent Failures and Common Pitfalls

When a cron job fails silently, there's no error message, no email, no stack trace — just an application that starts degrading while nobody notices. This is the number one reason cron gets a bad reputation in production. The fix is a repeatable debug workflow.

Start with the cron daemon logs. On modern systems, journalctl -u cron shows every job execution — the exact command, the timestamp, and the user. On older systems, /var/log/syslog or /var/log/cron contains the same. Search for your command name with grep.

If the job ran but had no effect, the problem is usually the environment. Simulate cron's minimal environment with env -i HOME=/root PATH=/usr/bin:/bin /bin/bash your_script.sh. This stripped-down shell will reproduce the exact conditions under which cron runs your script. Any errors you see here are the errors cron sees — they just go unlogged.

Another common pitfall: the script has a shebang pointing to the wrong interpreter. If your script starts with #!/usr/bin/python3 but Python 3 is installed at /usr/local/bin/python3, cron will fail with a misleading error. Always use #!/usr/bin/env python3 for portability.

Finally, remember that cron jobs inherit the umask of their parent process — typically 022. If your job creates files that need specific permissions, set umask explicitly in the script.

debug_cron_failures.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# ─────────────────────────────────────────────────────────────────
# DEBUGGING A CRON JOB THAT FAILS SILENTLY
# ─────────────────────────────────────────────────────────────────

# Step 1: Check if cron daemon is even running
ps aux | grep -E 'cron|crond'

# Step 2: List your crontab and verify the schedule
crontab -l

# Step 3: Check cron logs for recent job starts
journalctl -u cron --since '2 hours ago' | grep -E 'CMD|FAILED'

# Step 4: Simulate cron environment and run the script manually
# (Replaces all shell customizations)
env -i HOME="/home/deploy" LOGNAME="deploy" PATH="/usr/bin:/bin" SHELL="/bin/bash" \
    /bin/bash /opt/scripts/your_script.sh

# Step 5: If the script works in Step 4, the issue is PATH or a missing env variable.
# Add explicit exports at the top of the script.

# Step 6: Check file permissions on script and all parent directories
ls -la /opt/scripts/your_script.sh

# Step 7: Test with a very basic cron job to confirm cron itself works
# (Run for 1 minute from now using 'at' or temporary crontab line)
# Then check cron log
echo "* * * * * echo 'cron works' >> /tmp/cron_test.log" | crontab -
# Wait 1 minute then check
cat /tmp/cron_test.log
# Clean up
crontab -r
crontab -l 2>/dev/null || echo "crontab cleaned"
Output
# Successful debug output:
# journalctl -u cron --since '2 hours ago' | grep -E 'CMD|FAILED'
Mar 15 14:30:01 prod-server CRON[17234]: (deploy) CMD (/opt/scripts/your_script.sh)
Mar 15 14:31:02 prod-server CRON[17235]: (deploy) CMD (/opt/scripts/your_script.sh) # shows duplicate run if no lock
# env -i simulation output:
Error: command not found: pg_dump
# -> Missing PATH for postgres binaries
Cron Debugging Mental Model
  • Every cron job runs in a fresh, isolated shell session with a minimal environment.
  • Any customisation you rely on must be re-declared inside the script.
  • Theenv -i command reproduces that isolation exactly — use it to see what cron sees.
  • When a job works manually but fails under cron, the first suspect is always the environment.
Production Insight
A common scenario: a Python script using os.getenv('ENV') fails because cron doesn't set it. The script exits 0 but does nothing.
Always add a preamble that logs the environment: echo "PATH=$PATH" >> $LOG_FILE at the top of every script.
Rule: if you can't reproduce the failure with env -i, it's not a cron problem — check for race conditions or resource limits.
Key Takeaway
The cron log is your first line of defense — check it before anything else.
env -i replicates the cron environment perfectly — use it to expose missing variables.
Most cron failures are environment and permissions — not code bugs.
Cron Debug Decision Tree
IfJob appears in cron log but no output
→
UseScript ran but crashed silently — add stderr redirection and test manually.
IfJob does not appear in cron log at all
→
UseCron expression is wrong, crontab corrupted, or daemon stopped.
IfJob appears in log with exit code non-zero
→
UseScript returned an error — check script logic, but also check lock files and permissions.

Real-World Cron Patterns — Database Backups, Cache Warming, and Health Checks

In production, cron jobs fall into a handful of common patterns. Getting these right means the difference between a reliable automation pipeline and a pager at 3am.

Database Backups — The most critical cron job. Use the production-ready pattern from section 2: explicit env, logging, locking. Always validate the backup: after pg_dump, run pg_restore --list on the dump file to catch corruption early. Store backups off-server (S3, NFS) and retain multiple copies.

Cache Warming — Many apps rely on a cache that expires at midnight. Instead of serving slow responses to the first users, run a cron job just before peak traffic (e.g., 0 5 *) to pre-compute and populate the cache. Use flock to prevent overlap if the warming takes longer than the interval.

Health Checks — These run every 5 or 15 minutes and report system health to monitoring (e.g., check disk space, process up, API endpoint response). The script should output metrics in a parseable format (JSON) so Prometheus or a custom collector can ingest them. Unlike backups, health checks should NOT use locking — you want to run even if the previous check is still going (but alert if that happens).

Log Rotation — Cron handles log rotation via /etc/cron.daily/logrotate. When building custom rotation for app logs, never use rm -rf — use logrotate with compression and date-stamped filenames. Accidentally removing a log file that a running process is writing to can cause process hang or data loss.

Data Syncing — For ETL or replication between systems, schedule the sync during off-peak hours. Use idempotent scripts (sync only what changed) and monitor for latency. If a sync fails, cron won't retry — wrap it in a retry loop with exponential backoff.

real_world_patterns.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# ─────────────────────────────────────────────────────────────────
# PATTERN 1: Database Backup Validation
# ─────────────────────────────────────────────────────────────────
# Inside the backup script, after dump:
pg_dump -U $DB_USER $DB_NAME | gzip > $BACKUP_PATH.gz

# Validate the backup structure (quick check — doesn't load data)
if ! gzip -dc $BACKUP_PATH.gz | pg_restore --list > /dev/null 2>&1; then
    echo "[$TIMESTAMP] ERROR: Backup validation failed!"
    exit 1
fi

# ─────────────────────────────────────────────────────────────────
# PATTERN 2: Cache Warming (e.g., pre-generate popular product pages)
# ─────────────────────────────────────────────────────────────────
# Scheduled at 5am daily
0 5 * * * /opt/scripts/warm_cache.sh

# warm_cache.sh
#!/bin/bash
for url in $(cat /opt/scripts/top_urls.txt); do
    curl -s -o /dev/null -w "%{http_code}" "$url" >> /var/log/cache_warm.log 2>&1
done

# ─────────────────────────────────────────────────────────────────
# PATTERN 3: Health Check with JSON Output
# ─────────────────────────────────────────────────────────────────
# Every 15 minutes
*/15 * * * * /opt/scripts/health_check.sh

# health_check.sh
#!/bin/bash
cat <<EOF
{
  "timestamp": "$(date -Iseconds)",
  "disk_free_pct": $(df / | awk 'NR==2 {print $5}' | tr -d '%'),
  "mem_free_mb": $(free -m | awk 'NR==2 {print $7}'),
  "nginx_running": $(pgrep nginx | wc -l)
}
EOF
# The output goes to stdout; redirect in crontab to a log or a file readable by monitoring.
Output
# Output of health_check.sh:
{
"timestamp": "2024-03-15T06:30:01+00:00",
"disk_free_pct": 72,
"mem_free_mb": 1024,
"nginx_running": 1
}
Don't Validate Backups with File Size Alone
A corrupted backup can be exactly the same size as a valid one. Use pg_restore --list as shown above — it catches schema corruption, partial dumps, and truncation in milliseconds. For MySQL, use mysqlcheck --databases $DB_NAME after dumping.
Production Insight
Cache warming sounds great until your cache refresh takes longer than 5 minutes and overlaps with itself. Always add flock to cache warmers too — stale cache is better than a runaway refresh loop.
Health checks should never block on external dependencies. If your check pings an external API and that API is slow, your cron job may pile up. Add a timeout: timeout 10 curl ....
Rule: for any cron job that touches production data, add a dry-run mode (--dry-run) that prints what it would do — run it after every code change.
Key Takeaway
Different cron job types have different patterns: backups need locking, health checks need timeouts, cache warmers need staggering.
Always validate the job's output — a silent success that produces broken data is worse than a crash.
Dry-run mode for every production-affecting cron job. No exceptions.
Choosing a Real-World Cron Job Pattern
IfJob operates on production data (backups, migrations)
→
UseMust include locking, validation, offsite storage, and a dry-run mode. Never test on prod without dry-run.
IfJob improves performance (cache warming, precomputation)
→
UseMust handle gracefully if it takes too long — use locking and consider staggering with a jitter.
IfJob provides observability (health check, metric push)
→
UseDo NOT lock — you want the check to run even if the previous one is stalled. Set timeouts instead.
IfJob is idempotent (sync data, clean up temp files)
→
UseIdempotency is great, but still log everything — you'll need the logs when something unexpected happens.
● Production incidentPOST-MORTEMseverity: high

When Database Backup Overlaps Destroyed a Week of Transactions

Symptom
The backup script silently completed without errors, but a week later when a restore was needed, the backup file contained partial, interleaved data from two simultaneous dumps.
Assumption
The engineer assumed cron would wait for the previous job to finish before starting the next one. They'd never seen two copies of the script running at the same time.
Root cause
Cron has no built-in job tracking — it fires the command at the scheduled time regardless of whether a previous invocation is still running. The backup used pg_dump which writes to a temporary file; two concurrent dumps wrote to the same temp file, corrupting it.
Fix
Add an exclusive lock using flock as shown in the production-ready example above. Also switch the backup interval to once daily (the job runtime was well under 24h after optimisation) and set the cron expression to 30 2 * so it runs once per day at 2:30am, when system load is lowest.
Key lesson
  • Cron will never wait for a previous run to finish — you must implement locking yourself.
  • Always verify backup integrity with automated restoration tests (e.g., restore to a staging DB and run a checksum).
  • Log the PID at the start of each run so you can identify overlapping executions retroactively.
Production debug guideSymptom → Action table for the most common cron production issues4 entries
Symptom · 01
Cron job works when run manually but does nothing under cron
→
Fix
Simulate cron's environment: run env -i HOME=$HOME PATH=/usr/bin:/bin /bin/bash your_script.sh and check errors. Compare env output vs cron log.
Symptom · 02
No output from cron job, no log file created
→
Fix
Check cron log: grep your_command /var/log/syslog or journalctl -u cron --since today. Ensure redirection is present: >> /var/log/myjob.log 2>&1.
Symptom · 03
Job runs but produces no effect (e.g., backup file is empty)
→
Fix
Check for overlapping runs: verify lock file existence. If lock file present, script may have exited early due to an unhandled error. Add explicit error handling with exit codes.
Symptom · 04
Job fails with 'command not found' but script uses absolute paths
→
Fix
Check script's shebang line (#!/bin/bash or #!/usr/bin/python3). If shebang is missing, cron assumes shell script and may fail. Also verify script is executable (chmod +x).
★ Quick Debug Cheat Sheet — Cron JobsFive commands to diagnose any cron failure in under 60 seconds
No job output at all−
Immediate action
Open cron log
Commands
sudo journalctl -u cron --since '1 hour ago'
grep CRON /var/log/syslog | tail -30
Fix now
Add >> /var/log/cron_output.log 2>&1 to the crontab line and re-run the job manually.
Job runs manually but fails under cron+
Immediate action
Simulate cron environment
Commands
env -i HOME=$HOME PATH=/usr/bin:/bin /bin/bash script.sh
crontab -l | grep script.sh
Fix now
Set PATH explicitly inside the script: export PATH="/usr/local/bin:/usr/bin:/bin"
Job runs but too many instances+
Immediate action
Check for lock file and running PIDs
Commands
ps aux | grep script_name.sh
ls /tmp/*.lock
Fix now
Wrap the script body with exec 9>/tmp/mylock.lock; flock -n 9 || exit 1
Job fails with permission denied+
Immediate action
Check file permissions
Commands
ls -la /path/to/script.sh
sudo -u deploy_user /bin/bash script.sh
Fix now
Run chmod +x /path/to/script.sh and ensure the cron user has read/execute on the script and all parent directories.
Cron vs Systemd Timers — Which Should You Use?
FeatureCron JobsSystemd Timers
Setup complexitySingle crontab lineTwo unit files required (.service + .timer)
LoggingManual — redirect to fileAutomatic via journald
Missed job handlingSilently skipped if system was offCan catch up missed runs with Persistent=true
Dependency managementNone — runs regardlessCan depend on other systemd units
Run on bootVia @reboot keywordNative with OnBootSec=
Environment variablesMinimal — must set explicitlySet in unit file with Environment= directive
Retry on failureNot supportedConfigurable with Restart= directive
Best forSimple, single-command schedulesComplex system-level tasks with dependencies
Visibilitycrontab -l onlysystemctl list-timers — shows next run time
PortabilityWorks on every Linux/Unix systemOnly on systemd-based systems (most modern Linux)

Key takeaways

1
Cron runs with a stripped-down environment
never assume PATH, env variables, or shell customizations are available. Use absolute paths and source env files explicitly inside every script.
2
Always redirect both stdout and stderr to a log file (>> /path/to/job.log 2>&1)
without this, silent failures are guaranteed and you'll have no evidence a job even ran.
3
Use flock to create a lock file when your job's runtime could exceed its schedule interval
running two copies of a backup or migration job simultaneously can corrupt data silently.
4
Know when cron is the wrong tool
if you need retries, job dependencies, distributed execution, or failure alerting, reach for systemd timers, Airflow, or a cloud scheduler instead of bolting complexity onto cron.
5
Validate every backup
size checks are not enough. Use pg_restore --list or similar to catch corruption immediately after dump.
6
Add a dry-run mode to every production-affecting cron job. Test it on every code change before enabling the real schedule.

Common mistakes to avoid

3 patterns
×

Relying on your shell's PATH

Symptom
The job works perfectly when you run it manually but does nothing (or logs 'command not found') when cron triggers it.
Fix
Always use absolute paths in cron commands (/usr/bin/python3 not python3) and set PATH explicitly at the top of your script, or source your env file.
×

Forgetting to redirect stderr

Symptom
Your script throws an error, cron emails root (if mail is configured) or discards it entirely, and you have zero visibility into what went wrong.
Fix
Always append >> /var/log/myjob.log 2>&1 to your crontab line, where 2>&1 merges stderr into stdout so both go to the same log file.
×

Using `crontab -r` when you meant `crontab -e`

Symptom
All your cron jobs disappear instantly with no confirmation prompt and no undo.
Fix
Before you do anything destructive, run crontab -l > ~/crontab_backup.txt to save a copy. Some teams commit their crontab to version control via a provisioning script specifically to prevent this.
INTERVIEW PREP · PRACTICE MODE

Interview Questions on This Topic

Q01SENIOR
What happens to a cron job if the server is rebooted exactly when the jo...
Q02SENIOR
A developer says their cron job works fine when they run it manually but...
Q03SENIOR
Two cron jobs are scheduled to run at the same time and they both write ...
Q01 of 03SENIOR

What happens to a cron job if the server is rebooted exactly when the job was supposed to run? How would you handle that scenario in production?

ANSWER
Cron does not catch up on missed runs. If the system is off at the scheduled time, that run is lost forever. In production, you can handle this by using systemd timers with Persistent=true, which will run the job immediately after boot if the timer was missed. Alternatively, use @reboot to trigger the job on startup, or design your job to be idempotent and run frequently so missing one run doesn't matter. For critical jobs like database backups, consider a monitoring layer that alerts if a backup file hasn't been created in the last 26 hours.
FAQ · 5 QUESTIONS

Frequently Asked Questions

01
How do I run a cron job every 5 minutes in Linux?
02
Why is my cron job not running even though the syntax looks correct?
03
What's the difference between /etc/crontab and a user crontab?
04
Can cron send me an email when a job fails?
05
How do I run a cron job only on weekdays?
Related Next Steps
→Shell Scripting Basics: Writing Scripts That Actually Work in ProductionMaster the scripting patterns that pair with cron — error handling, idempotency, and robust execution.→AutoSys vs Cron vs Control-M — Which Job Scheduler Should You Use?Compare cron with enterprise schedulers so you know when to migrate away from basic cron.→Linux Process Management Explained — ps, kill, jobs and signalsUnderstand how cron's child processes behave, how to kill runaway jobs, and signal handling.→Linux System Performance Tuning: Kernel, Memory & I/O Deep DiveLearn how cron jobs impact system resources and how to configure I/O priorities for background tasks.
🔥

That's Linux. Mark it forged?

6 min read · try the examples if you haven't

←
Previous
Shell Scripting Advanced
7 / 12 · Linux
Next
Linux Networking Commands
→