Systemd Units: Services, Timers, and Socket Activation — Production Patterns That Actually Work

You've been using cron and init scripts for years. They work — until they don't. I've seen a cron job silently fail for six months because the environment wasn't loaded. I've watched a startup script deadlock because it tried to start a database before the network was up. Systemd units fix this with explicit dependencies, resource limits, and logging that doesn't vanish on crash. But most people use them wrong — they copy-paste a template and wonder why their service restarts in a loop or their timer misses a beat. This article shows you the patterns that survive production.

Why Systemd Units Beat Init Scripts and Cron

Before systemd, services were started by shell scripts in /etc/init.d. Those scripts had no dependency management — you'd sleep 5 seconds hoping the database was up. Cron jobs ran with minimal environment — PATH was often empty. Logs went to syslog or files that rotated unpredictably. Systemd units fix all this: declarative dependencies, structured logging via journald, resource limits, and reliable scheduling. The real win is socket activation — a pattern that cron and init scripts can't replicate. But the trade-off is complexity: a mistyped directive can silently break your service. You need to know the sharp edges.

// io.thecodeforge — DevOps tutorial

# /etc/systemd/system/myapp.service
[Unit]
Description=MyApp - production checkout service
After=network-online.target postgresql.service
Wants=postgresql.service

[Service]
Type=notify
ExecStart=/usr/local/bin/myapp --config /etc/myapp/config.yaml
Restart=on-failure
RestartSec=5
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=-/etc/myapp/env
LimitNOFILE=65536
MemoryMax=2G
MemoryHigh=1.5G

[Install]
WantedBy=multi-user.target

Services: The Right Way to Define a Daemon

A service unit defines how to start, stop, and monitor a daemon. The critical decisions are Type, Restart, and resource limits. Type=simple works for short-lived commands that stay in foreground. Type=notify is best for services that can signal readiness via sd_notify(). Type=forking is legacy — avoid unless you're wrapping an old daemon. Restart=on-failure is your friend, but pair it with RestartSec to prevent thrashing. Always set User and Group — running as root is a security hole. Use EnvironmentFile for config — don't hardcode secrets. The gotcha: systemd kills the entire cgroup on stop, not just the main PID. If your service spawns children that don't die, use KillMode=process to avoid zombie processes.

// io.thecodeforge — DevOps tutorial

# /etc/systemd/system/checkout-api.service
[Unit]
Description=Checkout API service
After=network.target redis.service
Requires=redis.service

[Service]
Type=notify
ExecStart=/opt/checkout/bin/api-server --port 8080
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/bin/kill -TERM $MAINPID
Restart=on-failure
RestartSec=10
StartLimitIntervalSec=60
StartLimitBurst=3
User=checkout
Group=checkout
WorkingDirectory=/opt/checkout
EnvironmentFile=-/etc/checkout/env
Environment=GOMAXPROCS=4
LimitNOFILE=65536
MemoryMax=1G
CPUQuota=80%

[Install]
WantedBy=multi-user.target

Timers: Cron's Replacement That Won't Lose Jobs

Cron has a dirty secret: if the system is off at the scheduled time, the job is lost. Systemd timers use monotonic or calendar events, and they can catch up after downtime with Persistent=true. Timers also log to journald, have resource limits, and can depend on other units. The common mistake: forgetting to enable the timer (not the service). The timer unit triggers the service; the service should not be enabled independently unless you want it to run at boot. Use OnCalendar for wall-clock schedules, OnUnitActiveSec for intervals after last run. The gotcha: time zones — systemd uses UTC by default. Set OnCalendar=--* 02:00:00 America/New_York explicitly.

// io.thecodeforge — DevOps tutorial

# /etc/systemd/system/cleanup.service
[Unit]
Description=Cleanup old logs

[Service]
Type=oneshot
ExecStart=/usr/local/bin/cleanup-logs --retention 30
User=cleanup

# /etc/systemd/system/cleanup.timer
[Unit]
Description=Run cleanup daily at 3am local
Requires=cleanup.service

[Timer]
OnCalendar=*-*-* 03:00:00 America/New_York
Persistent=true
RandomizedDelaySec=300
Unit=cleanup.service

[Install]
WantedBy=timers.target

Socket Activation: On-Demand Services That Save Resources

Socket activation lets systemd listen on a port and start the service only when a connection arrives. This is huge for low-traffic services — they consume zero memory until needed. It also enables parallel startup: the socket is created early, so clients can connect before the service is ready (connections queue in the kernel buffer). The pattern: a .socket unit defines the listen address, and a .service unit with the same name (minus .socket) defines the daemon. The service must accept the socket from systemd via sd_listen_fds() or by using a service that supports socket activation natively (like nginx, Apache, or custom daemons). The gotcha: Accept=yes creates one service instance per connection — use for simple inetd-style services. Accept=no passes the listening socket to a single service instance — use for multi-threaded servers.

// io.thecodeforge — DevOps tutorial

# /etc/systemd/system/myapp.socket
[Unit]
Description=MyApp socket

[Socket]
ListenStream=8080
Accept=no

[Install]
WantedBy=sockets.target

# /etc/systemd/system/myapp.service
[Unit]
Description=MyApp service (socket-activated)
Requires=myapp.socket

[Service]
Type=simple
ExecStart=/usr/local/bin/myapp --socket-fd=3
StandardInput=socket

# myapp must read from fd 3 (or use sd_listen_fds)

When Not to Use Systemd Units

Systemd units are overkill for simple one-off scripts — just run them in a terminal. They add complexity for containerized apps where orchestration (Kubernetes, Docker Compose) already handles restart and resource limits. In those cases, use the orchestrator's health checks and resource requests. Also, avoid systemd for user-level services that should follow a user session — use systemd --user units instead. And never use systemd timers for sub-second precision — the granularity is 1 second, and scheduling overhead can cause jitter. For high-frequency tasks, use a dedicated scheduler like cron with second-level precision (via sleep loops) or a purpose-built job queue.

// io.thecodeforge — DevOps tutorial

# Bad: using systemd timer for a task that runs every 500ms
# Use a while loop in the service instead:

[Service]
ExecStart=/bin/bash -c 'while true; do /usr/local/bin/fast-task; sleep 0.5; done'
Restart=always