AutoSys WCC Flow View — Cascade Failure Detection

WCC (Workload Control Center) is the browser-based monitoring and management UI for AutoSys. It's hosted on Apache Tomcat (which AutoSys installs automatically) and accessible via a standard web browser. While experienced admins often prefer the command line for speed, WCC is invaluable for understanding the state of a complex job flow at a glance — especially during incident response.

How AutoSys WCC Flow View Reveals Cascade Failures

AutoSys WCC (Web Client Console) Flow View is a dependency graph visualization that maps job-to-job relationships across a schedule. It renders the directed acyclic graph (DAG) of job chains, showing upstream triggers, downstream dependencies, and real-time status (success, failure, running, or queued). The core mechanic is that each node represents a job instance, and edges represent conditions like 'job A completes successfully before job B starts'. This lets you trace failure propagation instantly.

In practice, Flow View refreshes status every 30–60 seconds (configurable) and color-codes nodes: green for success, red for failure, yellow for running, gray for pending. When a job fails, all downstream jobs that depend on its success turn red or remain pending—this is the cascade failure signature. The view also exposes job-level details like exit codes, start times, and resource usage on hover, so you can distinguish a genuine failure from a skipped job due to upstream conditions.

Use Flow View when diagnosing production incidents involving stalled or failed job streams—especially in batch processing pipelines with hundreds of jobs. It matters because a single upstream failure can silently block dozens of downstream jobs, causing SLA breaches. Without this view, teams waste hours grepping logs or manually checking job statuses. With it, you pinpoint the root cause job in under 30 seconds.

Accessing WCC

WCC runs on the AutoSys application server. The default URL is typically http://autosys-server:8080/wcc or https://autosys-server:8443/wcc for SSL.

Before you can access it, verify Tomcat is running. The AutoSys installation includes a bundled Tomcat server configured specifically for WCC. You'll find the startup scripts in $AUTOSYS/wcc/bin/.

If you're running WCC in a clustered environment, each application server hosts its own WCC instance. You'll need to know the hostname or load balancer VIP. Most teams bookmark the URL in their on-call browser profile — don't be the person fumbling for a bookmark at 3 AM.

# Default WCC URL format
# http://autosys-server-hostname:8080/wcc
# https://autosys-server-hostname:8443/wcc  (SSL)

# Check if WCC (Tomcat) is running on the server
ps -ef | grep -i tomcat

# Check the Tomcat port is listening
ss -tlnp | grep 8080
ss -tlnp | grep 8443

# If WCC is down, restart Tomcat
$AUTOSYS/wcc/bin/shutdown.sh
$AUTOSYS/wcc/bin/startup.sh

Key WCC views for day-to-day monitoring

WCC provides several views. These are the ones you'll use most:

Monitor > Jobs: A real-time list of all jobs with their current status, colour-coded. Filterable by status, machine, owner, or name pattern. This is your primary monitoring view.

Monitor > Flow View: Shows jobs as nodes in a dependency graph with connecting arrows. When a job fails, you can immediately see which downstream jobs are blocked. Essential for understanding impact during incidents.

Monitor > Job Activity: A timeline view showing job start and end times. Useful for spotting jobs that are running longer than usual.

Administration > Calendars: View and manage AutoSys calendars.

Administration > Machines: View agent machine status — which are ACTIVE, MISSING, or INACTIVE.

Using WCC during an incident

When the on-call team gets paged at 3 AM because a batch run is failing, WCC is where you start. The flow of an incident investigation:

1. Open WCC and go to Monitor > Jobs. Filter by status = FAILURE to see all failed jobs.
2. Click on the first failed job — it's likely the root cause if multiple jobs failed.
3. Switch to Flow View to see the dependency chain. This shows you exactly which downstream jobs are blocked.
4. Check the failed job's logs by clicking 'View Log' or checking its std_out_file attribute.
5. After fixing the root cause, Force Start the failed job via right-click → Force Start.
6. Monitor the downstream jobs as they cascade from ACTIVATED to RUNNING to SUCCESS.

Pro tip: In large environments, use the filter to show only jobs that are ON HOLD or FAILURE — this reduces noise and highlights the critical path.

# Step 1: Check WCC Monitor > Jobs, filter by status = FAILURE
# Visual: red jobs immediately visible

# Parallel: command-line equivalent
autorep -J % -s FA

# Step 2: Click the failed job in WCC to see its attributes and log paths
# Command-line equivalent:
autorep -J failed_job -d
cat /logs/autosys/failed_job.err

# Step 3: Use Flow View to see which downstream jobs are blocked
# Command-line equivalent:
autorep -J % | grep -E 'AC|IN' | head -20  # show ACTIVATED/INACTIVE jobs

# Step 4: After fixing, restart via WCC (right-click > Force Start)
# Command-line equivalent:
sendevent -E FORCE_STARTJOB -J failed_job

Understanding Monitor > Jobs View

The Monitor > Jobs view is your day-to-day dashboard. It displays a table of all jobs with columns: Job Name, Status, Last Start Time, Last End Time, Machine, Owner, and Exit Code. You can sort by any column and filter by status, machine, owner, or job name pattern.

Color coding: SUCCESS (green), FAILURE (red), RUNNING (blue), ACTIVATED (yellow), ON HOLD (grey), INACTIVE (white). This makes status scanning fast.

You can right-click any job to perform actions: Force Start, Hold, Rerun, Kill, Release. The actions are equivalent to sendevent commands.

One overlooked feature: the 'View Log' popup shows the last N lines of the job's stdout and stderr. However, if the log is on the agent and the agent is unreachable, this won't work. Always have direct SSH access as a fallback.

Pro tip: Save a custom filter for jobs on your team's critical path. For example, filter by job name pattern PAYROLL% and status != SUCCESS to quickly spot non-green jobs in the nightly batch.

Monitor > Flow View: Managing Dependencies Visually

Flow View is WCC's most powerful feature for incident response. It displays jobs as nodes in a directed graph, with arrows representing condition dependencies (SUCCESS, FAILURE, EXIT code, etc.). You can zoom in/out, pan, and click any node to see its details.

When a job fails, its node turns red. All downstream jobs that depend on it are highlighted as blocked (ON HOLD or ACTIVATED with greyed-out arrows). This visual representation immediately answers the question: 'What is affected?'

You can also edit dependencies from Flow View? No — WCC is read-only for dependencies. You must use JIL to change conditions. However, you can view the dependency details by clicking a job and looking at the 'Depends' tab.

Flow View also supports 'Trace' mode: click a job and select 'Trace Upstream' — this highlights all jobs that must succeed before this one can run. Useful for debugging a stuck ACTIVATED job.

Pro tip: In large flows with hundreds of jobs, use the Filter in Flow View to show only jobs with status = FAILURE or ON HOLD. This declutters the graph and highlights the problem chain.

# Find jobs blocked by a specific failed job
autorep -J downstream_job -d | grep -i condition

# Show all jobs that depend on a given job (reverse dependency)
autorep -J % -q | grep "upstream_job_name"

# View the full dependency tree recursively (bash snippet)
autorep -J % -q | awk -F' ' '{print $1}' | while read job; do
  echo "--- $job depends on ---" 
  autorep -J $job -d 2>/dev/null | grep -i "condition"
done

Administering Machines and Calendars

The Administration views in WCC let you manage two critical aspects of AutoSys: machines and calendars.

Administration > Machines: Shows a list of all machines connected to the AutoSys application server. Each machine has a status: ACTIVE (agent is connected and healthy), MISSING (agent not responding after 20 minutes), INACTIVE (agent was deactivated). If a machine is MISSING, jobs assigned to that machine will stay in ACTIVATED state waiting for an available agent. You can right-click a machine to toggle its status or view properties.

Administration > Calendars: View and manage calendar definitions. Calendars control which days jobs run. You can create, edit, delete calendars. Changes take effect immediately — no restart needed. Pro tip: Never delete a calendar that's still referenced by active jobs. WCC will warn you, but double-check with autocal_asc first.

Limitation: WCC does not allow you to add or remove machines — only view and change their status. Machine registration is done via JIL or the AutoSys admin utility.

Pro tip: Use the Machine view during an incident to verify that the agent on the critical machine is ACTIVE. A MISSING agent is a common root cause of stuck jobs.

Troubleshooting WCC Connectivity and Performance

WCC can be temperamental. Here are common issues and what to check:

WCC won't load: Check Tomcat process with ps -ef | grep tomcat. If Tomcat is running, check if the port is listening: ss -tlnp | grep 8080. If Tomcat is not running, start it: $AUTOSYS/wcc/bin/startup.sh. If it crashes immediately, check catalina.out for OutOfMemory errors or port conflicts.

Slow page loads: WCC performance degrades when the database (where AutoSys stores job data) is under load. Also, a large number of jobs (10k+) can slow down the initial page load. Tip: Use filters to reduce the data returned. For example, filter by a specific machine or job name pattern before loading.

Session timeout: WCC has a configurable session timeout (default 30 minutes). If you step away, you'll be logged out. The session is in Tomcat's web.xml. You can extend it, but that's a security trade-off.

Stale data across multiple tabs: If you have multiple WCC tabs open, each may have its own session. They don't auto-sync. Closing and reopening WCC is the quick fix.

WCC vs CLI conflict: Sometimes WCC shows a job as RUNNING but the CLI shows it as SUCCESS. This is a browser cache issue. Press Ctrl+F5 or clear the browser cache.

Log viewer fails: The 'View Log' button in WCC tries to read from the agent's log location. If the agent is unreachable (MISSING), it will fail. Use SSH to the agent directly.

# Check Tomcat process
ps -ef | grep tomcat

# Check if port is listening
ss -tlnp | grep -E '8080|8443'

# Check catalina logs for errors
tail -200 $AUTOSYS/wcc/logs/catalina.out

# Restart Tomcat
$AUTOSYS/wcc/bin/shutdown.sh
$AUTOSYS/wcc/bin/startup.sh

# If restart fails, check disk space
# Tomcat needs free space for logs and temp files
df -h

# If WCC is still down, continue operations via CLI
autorep -J %
sendevent -E FORCE_STARTJOB -J my_job

Why Your WCC Jobs Are Silent-Failing: The JIL Validation Gap

You've seen it. A job shows 'SUCCESS' in WCC, but the data it was supposed to produce never landed. That's not a bug in AutoSys — it's a gap in how you define success in JIL. The exit code of a shell script is not the same as business success.

WCC only reports what AutoSys knows: the process exit code. If your script prints 'All good' after crashing on line 47, but still returns exit 0, WCC paints that green square and calls it a day. You're flying blind.

The fix: instrument your JIL definitions with a condition check on the actual outcome, not just the exit code. Append a post-processing job that validates the artifact — file existence, row count, API response code. Then condition that downstream job against 'SUCCESS' of the validator, not the primary executor.

Map your critical indicators in WCC as custom 'reportable' attributes in the job definition. Pull them into a Monitor view column. You will spot a silent failure before the business calls you.

// io.thecodeforge — devops tutorial
// Validate job output before trusting WCC status

insert_job: data_extract
job_type: cmd
command: /opt/scripts/extract.sh
every: 5
condition: s(ingestion_done)
term_run_time: 60

// Validator job - only this feeds downstream
insert_job: validate_extract
job_type: cmd
command: /opt/scripts/validate_output.sh /data/extract/output.csv
condition: s(data_extract)
term_run_time: 10

// Downstream waits on validator, not extractor
insert_job: load_to_dwh
job_type: cmd
command: /opt/scripts/load.sh
condition: s(validate_extract)
term_run_time: 120

The Machine-Level View WCC Hides: Why Your Boxes Stall

Hot take: WCC's 'Monitor > Jobs' view is a lie. It shows you job status on a machine, but it will never tell you why the machine itself is the bottleneck. I've spent three hours tracing a box that kept showing 'INACTIVE' while all its children were 'RUNNING' — turns out the agent had dropped a heartbeat, but WCC still showed the machine as 'ON'.

WCC polls agents via TCP/7600. If the agent responds with a stale status (e.g., from a zombie process holding the lock), WCC trusts it. You see a cascade of 'FAILURE' on downstream jobs and assume the job broke — but really, the agent needed a restart yesterday.

The answer: build a separate heartbeat job that pokes the agent every 2 minutes and writes to a watchdog box. Then condition your real work on that box being 'SUCCESS'. If the box goes 'FAILURE', your entire pipeline stops hard — and you know it's infrastructure, not code.

Don't rely on the WCC machine dashboard alone. Trust, but verify — with a job.

// io.thecodeforge — devops tutorial
// Machine-level health via job not WCC dashboard

insert_job: agent_heartbeat_box
job_type: box
date_conditions: 1

// Runs every 2 minutes, checks agent responsiveness
insert_job: heartbeat_app_01
job_type: cmd
command: /opt/autosys/bin/autorun %s
box_name: agent_heartbeat_box
run_calendar: always
start_times: "00:00"
interval: 00:02:00
alarm_if_fail: 1

// Downstream work depends on heartbeat box
insert_job: production_pipeline
job_type: box
condition: s(agent_heartbeat_box)

The Hidden Value of WCC's History View: Why You're Debugging Blind

When a job fails after a week of clean runs, most devs open the job definition, tweak something random, and rerun. That's cargo-cult debugging. You need to see what changed — in the environment, not the code. WCC's History view (Administer > Reports > Job History) is the tool nobody uses properly.

Export the history for that job over the past 30 days. Look for patterns: the job consistently runs 3 seconds on success, but 11 seconds on the last run before failure. That's a resource leak. Or the job finishes at 2:01 AM EXACTLY every day, but yesterday it drifted to 2:14 AM — and then at 2:15 AM a calendar-based batch job kicked in and stole CPU.

The History view exposes shift patterns in run times, exit codes, and resource contention windows. Sort by duration, filter by exit code != 0, then look at the preceding run's duration. A sudden slowdown always precedes a failure. That's your debugging window.

Apply this: set up a WCC report that emails you any job whose runtime exceeds 3x its moving average over 7 runs. Catch the slowdown, fix the resource issue, and watch your mean time to resolution drop.

// io.thecodeforge — devops tutorial
// WCC report for jobs exceeding 3x moving average runtime

insert_job: anomaly_detection_report
job_type: cmd
command: /opt/scripts/wcc_anomaly_report.sh
  --window 7
  --threshold 3.0
every: 10
max_run_alarm: 1

// The script queries WCC history API, filters, and alerts
// Sample output: