Senior 3 min · March 19, 2026

AutoSys ON_ICE vs ON_HOLD — Why Release Timing Skips Jobs

ON_ICE released at 2:30 AM skipped the 2 AM run entirely, delaying settlement 24 hours.

N
Naren · Founder
Plain-English first. Then code. Then the interview question.
About
 ● Production Incident 🔎 Debug Guide
Quick Answer
  • ON_HOLD (OH): job is frozen; release means it starts immediately if conditions are currently true
  • ON_ICE (OI): job is deeply suspended; release waits for conditions to reoccur in the next cycle
  • Key difference: time conditions are evaluated ONCE under ON_ICE, but are currently true under ON_HOLD
  • Production impact: using ON_HOLD when you meant ON_ICE can cause a job to run outside its window
  • Most common mistake: assuming both behave the same on release — they don't
Plain-English First

ON_HOLD is like pausing a video — when you press play again, it continues exactly where it left off. ON_ICE is like switching the TV off — when you turn it back on, it waits for the next scheduled programme, not the one you were watching.

ON_HOLD and ON_ICE are the two ways to suspend a job in AutoSys. They're often confused because both stop a job from running — but their release behaviour is completely different, and choosing the wrong one in a production recovery scenario can cause jobs to either run unexpectedly or not run until the next day. Get this distinction right.

ON_HOLD — suspend and resume at current conditions

When a job is ON_HOLD, it is frozen in its current state. If you put a job on hold while its starting conditions were already met, releasing it (OFF_HOLD) will cause it to start immediately — the conditions are still true.

on_hold.shBASH
1
2
3
4
5
6
7
8
9
10
11
# Put a job on hold
sendevent -E JOB_ON_HOLD -J daily_report

# Release from hold — starts immediately if conditions are met
sendevent -E JOB_OFF_HOLD -J daily_report

# Put an entire BOX on hold (and all inner jobs)
sendevent -E JOB_ON_HOLD -J eod_processing_box

# Check if a job is on hold
autorep -J daily_report   # shows OH status
Output
Job Name ST Exit
daily_report OH -- <- ON_HOLD — conditions may already be met
/* After JOB_OFF_HOLD: */
/* If start_times was 02:00 and it's now 03:00 — job starts immediately */
Releasing ON_HOLD can start a job immediately
If you hold a job after its scheduled start time passes, releasing it with OFF_HOLD will start it immediately because the time condition is already past. This catches people off guard. If you want the job to wait until the next day's scheduled run, use ON_ICE instead.
Production Insight
Releasing a held batch job at 3 AM when it normally runs at 2 AM will kick it off right away.
This can trigger unintended downstream processes or hit resource constraints.
Rule: always verify current time against start_times before using OFF_HOLD during production recovery.
Key Takeaway
ON_HOLD is a pause button.
Release runs the job now if conditions are true.
You must know the current time relative to start_times before releasing.
When to use ON_HOLD vs ON_ICE
IfYou need the job to run manually as soon as you release it
UseUse ON_HOLD — it's designed for manual release scenarios
IfYou want to skip the current run and resume at next scheduled time
UseUse ON_ICE — it completely resets the condition cycle
IfYou are not sure when you will lift the hold
UseUse ON_ICE — safer because the job won't run unexpectedly at odd hours
ON_HOLD vs ON_ICE ON_HOLD vs ON_ICE. The release behaviour is everything · ON_HOLD (OH) · ON_ICE (OI) · JOB_ON_HOLD to set · JOB_ON_ICE to set · JOB_OFF_HOLD to release THECODEFORGE.IOON_HOLD vs ON_ICEThe release behaviour is everything ON_HOLD (OH) ON_ICE (OI)VSJOB_ON_HOLD to setJOB_ON_ICE to setJOB_OFF_HOLD to releaseJOB_OFF_ICE to releaseReleases → runs IMMEDIATELYReleases → waits NEXT CYCLEif conditions currently metconditions must reoccurUse: manual release scenariosUse: skip current day's runTHECODEFORGE.IO
thecodeforge.io
ON_HOLD vs ON_ICE
Autosys On Hold Vs On Ice

ON_ICE — suspend and wait for next condition cycle

When a job is ON_ICE, it's more deeply suspended. When you release it (OFF_ICE), the job does NOT start immediately — it waits for its conditions to become true in the next scheduling cycle.

For a job with start_times: '02:00', releasing it from ON_ICE at 3 AM means it won't run until the next scheduled 2 AM — it waits for the condition to reoccur, not to re-evaluate it as currently true.

on_ice.shBASH
1
2
3
4
5
6
7
8
# Put a job on ice
sendevent -E JOB_ON_ICE -J weekly_reconcile

# Release from ice — waits for conditions to reoccur (next cycle)
sendevent -E JOB_OFF_ICE -J weekly_reconcile

# Check if a job is on ice
autorep -J weekly_reconcile   # shows OI status
Output
Job Name ST Exit
weekly_reconcile OI -- <- ON_ICE
/* After JOB_OFF_ICE at 03:00 (job normally runs at 02:00): */
/* Job waits until NEXT scheduled 02:00 — does NOT run at 03:00 */
Production Insight
ON_ICE is forgiving during production incidents.
You can safely put a job on ice and forget it — it won't run until its next normal window.
But be careful: if the job has dependencies, those downstream jobs might also be blocked.
Key Takeaway
ON_ICE is a complete reset.
Release does NOT run the job now — it waits for the next scheduled time.
Use ON_ICE when you want to skip a run and resume normally.

Which to use when

The decision is simple once you understand the release behaviour:

Use ON_HOLD when: You want to delay a job temporarily and start it manually when ready. Maintenance windows. Jobs you want to run manually at a specific moment after releasing.

Use ON_ICE when: You want to skip a job for the current scheduling period and have it resume at its next normal scheduled time. Disabling a job for a few days without permanently modifying its definition. Safe suspension when you're not sure when you'll release it.

hold_vs_ice_decision.shBASH
1
2
3
4
5
6
7
8
9
10
11
# Scenario 1: Emergency maintenance, want to run job manually when done
# Use ON_HOLD — releases will start immediately when ready
sendevent -E JOB_ON_HOLD -J daily_settlement
# ... do maintenance ...
sendevent -E JOB_OFF_HOLD -J daily_settlement   # starts now

# Scenario 2: Job has a bug being fixed, skip today's run
# Use ON_ICE — releases will wait for tomorrow's scheduled run
sendevent -E JOB_ON_ICE -J daily_settlement
# ... fix deployed by end of day ...
sendevent -E JOB_OFF_ICE -J daily_settlement   # runs tomorrow at 02:00 as normal
Production Insight
Mixing up these two in a P1 incident leads to either a job running too early or missing its window.
Teams often apply ON_ICE when they actually need ON_HOLD, then wonder why the job doesn't start.
Rule: decide based on when you want the job to run after release — now or next period.
Key Takeaway
ON_HOLD for manual release now.
ON_ICE for automatic release next cycle.
Pick based on timing, not habit.

How ON_HOLD and ON_ICE affect dependencies

Both statuses affect downstream jobs that depend on the held/frozen job.

When a job is ON_HOLD, it cannot reach SUCCESS or FAILURE, so any downstream jobs that wait for it (via conditions) will remain in a pending state. Releasing from ON_HOLD will start the job, and once it completes, the downstream jobs will then run.

When a job is ON_ICE, same effect — it cannot complete, so dependents wait forever until the job is taken off ice and eventually runs.

The difference: if you put a job ON_HOLD and it already satisfied its own conditions, it will run immediately on release — downstream jobs start soon. If you put it ON_ICE, the job still needs to wait for its next condition cycle, so downstream jobs stay waiting longer.

dependency_impact.shBASH
1
2
3
4
5
6
7
8
9
10
# Job B depends on Job A
autorep -J Job_B   # Shows status: WAITING (because Job A is ON_HOLD or ON_ICE)

# If Job A is ON_HOLD and you release it:
sendevent -E JOB_OFF_HOLD -J Job_A
# Job A runs now -> succeeds -> Job B starts

# If Job A is ON_ICE and you release it:
sendevent -E JOB_OFF_ICE -J Job_A
# Job A waits until next scheduled time -> only then runs -> Job B waits even longer
Dependencies can mask a held job
If you see a downstream job stuck in WAITING, it might not be the job itself — check if any of its direct or indirect predecessors are ON_HOLD or ON_ICE. That's the real root cause.
Production Insight
We once spent two hours debugging a stuck batch chain.
The root cause was a predecessor job that had been ON_ICE for days by a different team.
Rule: always run autorep with dependency recursion when troubleshooting stalled pipelines.
Key Takeaway
Both ON_HOLD and ON_ICE block downstream jobs.
The difference is how soon the blocked job runs after release.
Diagnose stuck chains by checking predecessors' status first.

Debugging jobs that won't start — hold/ice troubleshooting

When a job doesn't start and you suspect a hold or ice status, follow these steps:

  1. Run autorep -J jobname and look for ST column — OH or OI indicates the job is held/iced.
  2. Check the status of any parent box: autorep -J boxname — if the box is OH/OI, inner jobs are blocked even if they look normal.
  3. Review sendevent history to see who set the hold/ice: check autorep logs or event trail.
  4. If a box is held and you release it, inner jobs might start immediately based on their own conditions — be cautious.

To clear a hold: sendevent -E JOB_OFF_HOLD -J jobname To clear ice: sendevent -E JOB_OFF_ICE -J jobname

debug_hold_ice.shBASH
1
2
3
4
5
6
7
8
9
10
11
12
13
# Check job status
autorep -J daily_report
# Output: ... ST: OH ... -> ON_HOLD

# Check box status (inner jobs may be blocked)
autorep -J batch_box

# Check event history (if available)
autorep -J daily_report -w -r

# Force a job ON_HOLD if it's ON_ICE? Not possible directly; must clear ice first.
sendevent -E JOB_OFF_ICE -J daily_report
# Then optionally set hold: sendevent -E JOB_ON_HOLD -J daily_report
Production Insight
A job stuck in OH status for hours is often a sign someone forgot to release it.
Automated monitoring should alert on jobs in OH or OI for more than X minutes.
Don't rely on manual checks — set up a simple cron to autorep and flag held jobs.
Key Takeaway
OH and OI in autorep are easy to spot.
Always check parent box status too.
Automate alerts for held jobs — they are silent failures.
● Production incidentPOST-MORTEMseverity: high

Batch job ran at 3 AM instead of 2 AM because of ON_ICE misuse

Symptom
Daily settlement report did not generate at 2 AM as expected. Downstream jobs failed due to missing input. On investigation, the settlement job was completed but its last successful run was from the previous day.
Assumption
The team assumed that releasing from ON_ICE would behave like ON_HOLD — the job would start immediately since its scheduled time had passed.
Root cause
The job was put ON_ICE during a maintenance window at 1:30 AM. At 2:30 AM, after release, the team expected it to run. But ON_ICE does not evaluate time conditions as 'currently true' — it waits for the next occurrence of the condition (the next day at 2 AM).
Fix
1. Take job off ice: sendevent -E JOB_OFF_ICE -J daily_settlement 2. Force a one-time run using sendevent -E JOB_ON_HOLD? Not possible while on ice. Instead, after OFF_ICE, manually trigger the job with sendevent -E FORCE_STARTJOB -J daily_settlement (if allowed) or wait for next scheduled run. 3. The real fix: use ON_HOLD for manual-release scenarios; ON_ICE only for skipping a period.
Key lesson
  • ON_ICE does NOT run the job immediately on release even if the scheduled window is past.
  • Always ask: do we want this job to run now (ON_HOLD) or next period (ON_ICE)?
  • Train your team on the difference — it's the #1 AutoSys interview question for a reason.
Production debug guideSymptom → Action for stuck jobs5 entries
Symptom · 01
Job shows ST: OH in autorep
Fix
Review why it was put on hold. If safe, release with JOB_OFF_HOLD and monitor if it runs immediately.
Symptom · 02
Job shows ST: OI in autorep
Fix
Understand context. If you want it to run now, release with JOB_OFF_ICE, but be aware it may wait for next cycle.
Symptom · 03
Downstream job stuck in WAITING
Fix
Check all predecessors with autorep -J <jobname> -q. Look for OH or OI status among ancestors.
Symptom · 04
Box job shows OH/OI, inner jobs never start
Fix
Release the box first with appropriate OFF event. Inner jobs may then run based on their own conditions.
Symptom · 05
FORCE_STARTJOB fails when job is ON_ICE
Fix
In most versions, you cannot force start a job that is ON_ICE. Use JOB_OFF_ICE first, then try force start.
★ Hold/Ice Quick Debug Cheat SheetQuick commands and actions for when jobs are stuck due to ON_HOLD or ON_ICE in production.
Job won't start — autorep shows OH
Immediate action
Check if you can release it now.
Commands
autorep -J <jobname> | grep ST
sendevent -E JOB_OFF_HOLD -J <jobname>
Fix now
If conditions are met, job will start immediately after OFF_HOLD.
Job won't start — autorep shows OI+
Immediate action
Assess whether you need it to run now or skip this run.
Commands
autorep -J <jobname> | grep ST
sendevent -E JOB_OFF_ICE -J <jobname>
Fix now
If you need it to run immediately, after OFF_ICE you may need to FORCE_STARTJOB (if allowed).
Downstream pipeline blocked — no job in OH/OI visible+
Immediate action
Check box jobs and ancestors.
Commands
autorep -J <downstream_job> -q
autorep -J <box_name> | head -20
Fix now
If a parent box is OH/OI, release it first.
FORCE_STARTJOB does nothing+
Immediate action
Check if target job is ON_ICE.
Commands
autorep -J <jobname> | grep ST
sendevent -E JOB_OFF_ICE -J <jobname>
Fix now
Then retry FORCE_STARTJOB if needed.
Complete comparison: ON_HOLD vs ON_ICE
AspectON_HOLD (OH)ON_ICE (OI)
Status codeOHOI
How to setsendevent -E JOB_ON_HOLDsendevent -E JOB_ON_ICE
How to releasesendevent -E JOB_OFF_HOLDsendevent -E JOB_OFF_ICE
After release behaviourStarts immediately if conditions currently metWaits for conditions to reoccur in next cycle
Use caseTemporary hold, manual release when readySkip current period, resume at next scheduled time
Risk if misusedJob starts unexpectedly after releaseJob misses current run entirely
Impact on dependenciesDownstream jobs blocked until held job completesSame, but release delay may be longer (next cycle)
Can FORCE_STARTJOB override?Often yes (depending on version)Usually no — must release ice first

Key takeaways

1
ON_HOLD (OH)
releasing runs the job immediately if conditions are currently met — use for manual-release scenarios
2
ON_ICE (OI)
releasing waits for conditions to reoccur in the next cycle — use for skipping the current run and resuming normally
3
This is one of the most common AutoSys interview questions
the release behaviour difference is what matters
4
Setting a box ON_HOLD also holds all inner jobs; same for ON_ICE
5
Always check parent box status when debugging stuck jobs
OH/OI on the box blocks everything inside

Common mistakes to avoid

4 patterns
×

Putting a job ON_HOLD expecting it to wait until the next scheduled run

Symptom
Job runs immediately after OFF_HOLD because conditions are currently met, causing it to execute outside its intended window.
Fix
Use ON_ICE instead if you want the job to skip this run and resume at the next scheduled time.
×

Putting a job ON_ICE when you need it to run manually after release

Symptom
After OFF_ICE, the job does not run; it waits until the next scheduled cycle. You can't force start it either (in most versions).
Fix
Use ON_HOLD if you plan to manually trigger the job after releasing the hold.
×

Forgetting a job is ON_ICE and wondering why it's not running for days

Symptom
Job remains in OI status indefinitely; no one notices because there's no alert.
Fix
Set up monitoring for jobs in OH/OI status for more than a few hours. Send automated reminders.
×

Using ON_HOLD on a box job without understanding it puts all inner jobs on hold too

Symptom
Inner jobs never start even though their individual status looks fine.
Fix
Always check box status when troubleshooting stalled jobs. Release the box with JOB_OFF_HOLD or JOB_OFF_ICE.
INTERVIEW PREP · PRACTICE MODE

Interview Questions on This Topic

Q01JUNIOR
What is the difference between ON_HOLD and ON_ICE in AutoSys?
Q02JUNIOR
If you release a job from ON_HOLD at 3 AM when its scheduled time was 2 ...
Q03JUNIOR
If you release a job from ON_ICE at 3 AM when its scheduled time was 2 A...
Q04SENIOR
When would you use ON_ICE instead of ON_HOLD?
Q05JUNIOR
What AutoSys status code represents ON_HOLD and ON_ICE?
Q06SENIOR
Can you FORCE_STARTJOB a job that is ON_ICE?
Q01 of 06JUNIOR

What is the difference between ON_HOLD and ON_ICE in AutoSys?

ANSWER
ON_HOLD suspends a job but retains the current time condition — releasing it with OFF_HOLD will start the job immediately if the start time has already passed. ON_ICE also suspends the job but resets the time condition — releasing with OFF_ICE makes the job wait for the next occurrence of its start time. In short: ON_HOLD = pause/resume; ON_ICE = skip/cycle wait.
FAQ · 6 QUESTIONS

Frequently Asked Questions

01
What is the difference between ON_HOLD and ON_ICE in AutoSys?
02
Which is safer to use — ON_HOLD or ON_ICE?
03
Can you FORCE_STARTJOB a job that is ON_ICE?
04
Does putting a BOX on hold affect the jobs inside?
05
What is the status code for ON_HOLD and ON_ICE in autorep?
06
How do I check event history to see who put a job on hold?
COMPLETE GUIDE
The Complete AutoSys Workload Automation Guide for Engineers →

JIL syntax, sendevent, autorep, box jobs, file watchers, scheduling, HA, security, cloud workload automation, and 22 interview questions — the definitive AutoSys reference for production engineers.

🔥

That's AutoSys. Mark it forged?

3 min read · try the examples if you haven't

Previous
AutoSys autorep Command
21 / 30 · AutoSys
Next
Force Start and Kill Job in AutoSys