AutoSys sendevent Command — FORCE_STARTJOB Pitfalls

sendevent is one of the most frequently used AutoSys commands in day-to-day operations. Every manual job control action — starting a job, putting it on hold, killing it, setting a global variable — goes through sendevent. It's the command-line interface to the AutoSys event system. If you've ever triggered a production outage because you sent the wrong event, you know exactly why this reference exists.

What the AutoSys sendevent Command Actually Does

The sendevent command is the sole mechanism for injecting events into an AutoSys event server, which then triggers job state transitions. Its core mechanic is simple: you specify an event type (e.g., FORCE_STARTJOB, KILLJOB, CHANGE_STATUS) and a job name, and the command writes a structured event record into the event server's queue. The event server processes this record asynchronously, applying the requested state change only if the job's current state permits it.

In practice, sendevent is a fire-and-forget operation — it returns immediately after queuing the event, not after the job actually transitions. This means a successful exit code (0) does not guarantee the job started or stopped. The event server's processing delay is typically sub-second, but under load (e.g., 500+ concurrent events) it can stretch to seconds. Also, the command respects job dependencies and machine availability; a FORCE_STARTJOB will fail silently if the job's machine is down or the job is already running.

You use sendevent when you need to manually override AutoSys scheduling logic — for example, forcing a job to run outside its normal schedule during a production incident, or killing a stuck job that has exceeded its max run time. It is the operational equivalent of a manual override switch, and misusing it (e.g., FORCE_STARTJOB on a job with unresolved upstream failures) can cascade into inconsistent system state.

sendevent syntax and key options

The basic syntax is: sendevent -E event_type -J job_name. The -E flag specifies the event type, -J the job name. Most commands also support -S for the AutoSys instance name if you have multiple environments. Always use single quotes around job names if they contain special characters. The event types are case-sensitive — KILLJOB works, killjob doesn't.

# Basic syntax
sendevent -E EVENT_TYPE -J job_name

# For a specific AutoSys instance
sendevent -E EVENT_TYPE -J job_name -S INSTANCE_NAME

# ── The most common sendevent operations ──────────────────────────

# Force start a job (run immediately regardless of conditions)
sendevent -E FORCE_STARTJOB -J daily_report

# Start a job normally (only if conditions are met)
sendevent -E STARTJOB -J daily_report

# Kill a running job
sendevent -E KILLJOB -J daily_report

# Put a job on hold (won't run until released)
sendevent -E JOB_ON_HOLD -J daily_report

# Release a job from hold
sendevent -E JOB_OFF_HOLD -J daily_report

# Put a job on ice (stronger suspension)
sendevent -E JOB_ON_ICE -J daily_report

# Release from ice
sendevent -E JOB_OFF_ICE -J daily_report

# Delete a job via event
sendevent -E DELETEJOB -J old_legacy_job

# Manually change a job's status
sendevent -E CHANGE_STATUS -J job_name -s SUCCESS

# Set a global variable
sendevent -E SET_GLOBAL -G "TRADE_COUNT=5432"

# Stop the Event Processor gracefully
sendevent -E STOP_DEMON

CHANGE_STATUS — manually overriding job status

CHANGE_STATUS lets you manually set a job to SUCCESS or FAILURE. This is used in recovery scenarios: if a job failed but the issue has been resolved manually outside AutoSys, you can mark it SUCCESS so downstream jobs can proceed. But be careful: you're lying to the scheduler, and downstream jobs may trust the data that never arrived. Use it only when you have verified the work is complete.

# Manually mark a job as SUCCESS (to unblock downstream dependencies)
sendevent -E CHANGE_STATUS -J extract_job -s SUCCESS

# Manually mark a job as FAILURE (to block downstream from starting)
sendevent -E CHANGE_STATUS -J questionable_job -s FAILURE

# After CHANGE_STATUS to SUCCESS, downstream jobs that were waiting
# will now evaluate their conditions and may start automatically

Using sendevent in shell scripts for automation

sendevent becomes especially powerful when used inside shell scripts for batch recovery automation. Instead of manually sending events one by one, you can script common operational patterns — like automatically restarting all failed jobs, or setting a global variable and triggering a downstream job in sequence. But there are traps: looping without a sleep can flood the Event Processor, and forgetting to quote variable expansions can break the command.

#!/bin/bash
# Auto-restart all FAILURE jobs matching a pattern

PATTERN="PRD_TRADING_%"

echo "Checking for failed jobs matching: $PATTERN"
FAILED_JOBS=$(autorep -J $PATTERN -s FA | awk 'NR>2 {print $1}' | grep -v '^$')

if [ -z "$FAILED_JOBS" ]; then
    echo "No failed jobs found"
    exit 0
fi

for JOB in $FAILED_JOBS; do
    echo "Restarting: $JOB"
    sendevent -E RESTART -J "$JOB"
    sleep 2   # brief pause between events to avoid overwhelming Event Processor
done

echo "Restart events sent for: $(echo $FAILED_JOBS | wc -w) jobs"

# Pattern 2: Set a global, then trigger downstream job
RECORD_COUNT=$(wc -l < /data/processed.csv)
sendevent -E SET_GLOBAL -G "TRADING_COUNT=${RECORD_COUNT}"
sendevent -E STARTJOB -J PRD_TRADING_VALIDATE_COUNT

sendevent Event Types Deep Dive — When to Use Each

Beyond the common events, there are less frequently used but critical ones: DELETEJOB to remove jobs, RESTART to restart a failed job (respects conditions), STOP_DEMON for graceful shutdown, and SET_GLOBAL for variables. Understanding the nuances — like RESTART only works on FAILURE jobs, or that JOB_ON_ICE prevents the job from running indefinitely until explicitly released — can prevent outages.

# RESTART - only works if job status is FAILURE or has next cycle pending
sendevent -E RESTART -J export_daily   # re-runs with original conditions

# DELETEJOB - removes job definition permanently; use with extreme care
sendevent -E DELETEJOB -J old_job

# SET_GLOBAL - sets a global variable visible to all jobs
sendevent -E SET_GLOBAL -G "ENVIRONMENT=STAGING"

# SENDSTATUS - sends an application status (rarely used, but available)
sendevent -E SENDSTATUS -J my_job -s "Data validated OK"

# STOP_DEMON - graceful shutdown of Event Processor
sendevent -E STOP_DEMON

# START_DEMON - restart Event Processor (usually via script, not sendevent)
# NOTE: There is no START_DEMON event; use the autostart mechanism

sendevent Troubleshooting and Common Failures

When sendevent 'does nothing', the culprit is almost always one of: wrong instance name, job name typo, case mismatch, or the Event Processor is down. Other issues: sending FORCE_STARTJOB to a box job (which may be running), or forgetting that JOB_ON_HOLD doesn't affect a job already running — you need KILLJOB first. This section gives you the debug flow to resolve 90% of issues in under a minute.

# Step 1: Is the Event Processor running?
autorep -W   # returns NOSUCH? If yes, Event Processor is down

# Step 2: What is the job's current status?
autorep -q jobname | head -20

# Step 3: Check event log for recent events
tail -100 $AUTOUSER/events/events.log | grep jobname

# Step 4: Verify job name spelling and case
autorep -J "JOBNAME*" | grep -i jobname

# Step 5: Test connectivity to Event Processor host
ping autoevent_host

# Step 6: Check for multiple instances
sendevent -E PING   # sends a test event; responds with 'PONG' from Event Processor

Firing Multiple Events: The at() Trick for Batch Operations

Running sendevent once per job is for amateurs. When you need to kill, force-start, or change status on fifty jobs simultaneously, you don't write a fifty-line script. You use the at() time specifier.

at() accepts a comma-delimited list of job names inside the parentheses. That means one command, one connection to the Event Processor, one trace log entry. It's faster, cleaner, and less likely to hit race conditions when you're freezing an entire pipeline.

The gotcha: at() only works with event types that accept multiple targets — CHANGE_STATUS, FORCE_START, KILLJOB. It does not work with COMMENT, ALARM, or SEND_SIGNAL. Check the sendevent documentation before you batch, or you'll get a silent partial failure.

// io.thecodeforge — devops tutorial

// Kill three stuck extract jobs in one shot
sendevent -E KILLJOB -J "extract_us_east,extract_us_west,extract_europe" -M "deployment freeze — killing all active extracts"

// Verify
autosyslog -J extract_us_east | tail -5
autosyslog -J extract_us_west | tail -5

Event Log Spam: How to Avoid Flooding the Autosys Event Processor

Every sendevent call generates an entry in the Event Processor log and a record in the autosyslog. That's fine for a handful of events. But when your automation loops over 200 jobs every five minutes, you're generating sixty thousand log entries an hour. The Event Processor doesn't care, but your capacity planning and troubleshooting just became a nightmare.

Filter before you fire. If you're only changing status on jobs that are in a specific state, check first. Don't send a FORCE_START to a job that's already RUNNING — it'll be ignored, but it still gets logged. Use autosys_showjob or a quick grep against the sendevent cache before you issue the command.

Second rule: batch with at() as covered above. Three events instead of three hundred. Your future self, three weeks into a post-mortem, will thank you.

Third rule: use the -T flag to set a termination time for events that might never finish. A CHANGE_STATUS to FORCE_START with no -T means the job runs forever. That's fine for normal batch, but for emergency recovery? Set a max runtime.

// io.thecodeforge — devops tutorial

// Only send FORCE_START to jobs that are in FAILURE state
for job in $(autosys_status -J load_data_* | grep FAILURE | awk '{print $1}'); do
  sendevent -E FORCE_START -J $job -M "recovery restart — pre-check passed"
done

// With a 30 minute hard limit
sendevent -E CHANGE_STATUS -s RUNNING -T 30 -J data_pipeline_primary -M "recovery with timeout"