# Timeouts & Liveness

What kills (or doesn't kill) a live agent. CLAUDE.md → Timeouts & Liveness
keeps the core invariant; the spawn-phase watchdog, steering safety nets,
and stale-orphan detection details live here.

> Source of truth: `engine/timeout.js`, `engine/spawn-phase-watchdog.js`,
> `engine/shared.js` (`ENGINE_DEFAULTS`, `getProcessCpuSeconds`,
> `killImmediate`), `engine/steering-store.js`. See also:
> [engine-restart.md](engine-restart.md). Last verified: 2026-06-09.

## Core invariant

**A live tracked agent is never killed for being silent.** Long builds,
installs, multi-file edits routinely produce no stdout for many minutes.

Only two things kill a live tracked process (`engine/timeout.js`):

1. **Hard wall-clock timeout** `engine.agentTimeout` (default 5h from
   `startedAt`; per-fan-out `meta.deadline`).
2. **Steering kill** — explicit human steering → `killImmediate()` so the
   agent re-spawns with `--resume <session>`.

**Don't add output-silence timers for live tracked processes.**

## Stale-orphan detection

`engine.heartbeatTimeout` (default 5 min) is the **grace window after the
engine loses the tracked process handle** — not a heartbeat timer.

Per-type overrides in `ENGINE_DEFAULTS.heartbeatTimeouts`:
`implement` / `implement:large` / `fix` / `test` / `verify` → 15 min;
`plan` → 10 min.

Orphan declaration requires four checks (all must pass):

1. `isTrackedProcessAlive` returns false.
2. 64 KB tail scan for `[process-exit] code=N`.
3. `isOsPidAliveForDispatch` returns false.
4. Full-log re-scan still shows no completion.

After engine restart, gated on `engineRestartGraceUntil` (default
20 min) — see [engine-restart.md](engine-restart.md).

## Steering safety nets (W-mq066js7000fff1f-c)

Three knobs backstop the steering pipeline. `engine/timeout.js`
defensively requires `./steering-store` and swallows `MODULE_NOT_FOUND`
only, so the gates work even before the store has shipped.

### Kill-retry escalation ladder

`engine.steeringMaxKillRetries` (default `3`, range `1–5`). After a
steering kill, if the process hasn't exited within 30 s:

1. Retry gracefully at 60 s, 120 s (last interval reused past the cap).
2. Fire a platform hard kill:
   - **Windows:** `taskkill /F /T /PID <pid>`.
   - **Unix:** descendant-tree SIGKILL — `pgrep -P` deepest-first +
     `pkill -KILL -P <pid>`.
3. After the cap: `[steering-stuck]` on `live-output.log` +
   `[engine-system]` inbox notice; `_steeringGaveUp = true`.

### Deferred-steering safety net

`engine.steeringDeferredMaxMs` (default `900000` = 15 min, range
`60_000–14_400_000`). Per-tick, any deferred message older than this
without a `sessionId` is **stranded** — `[steering-warn]` +
`_steeringStranded: true` via `mutateDispatch`; steering store →
`status='stranded'`. Re-warn is guarded by
`_deferredSteeringStrandedFiles`.

### Stale-session purge

`onAgentClose` clears `session.json` on `No conversation found`.
`dropSteeringForPurgedSession(agentId, sessionId, liveOutputPath)` runs
BEFORE the unlink, walks `steeringStore.listForAgent(agentId)` dropping
`{queued, live_kill, deferred, re_spawning}` whose `_steeringSessionId`
matches (status `dropped`, last_error `session-purged`) + writes:

```
[steering-failed] Session <id> was purged by runtime; message <id>
dropped, please re-send.
```

## Spawn-phase watchdog (W-mq0e2dae000a003d)

`engine/spawn-phase-watchdog.js#checkSpawnPhaseStalls` runs every tick
alongside `checkTimeouts` / `checkSteering` and kills children wedged
in MCP-init. Four gates, **ALL** required:

1. Fresh spawn (skips `procInfo.reattached`).
2. Elapsed since `procInfo.startedAt` ≥ `engine.spawnPhaseGraceMs`
   (default 120000 ms).
3. The last 16 KB of `live-output.log` contains zero non-startup events:
   - Copilot startup-only set:
     `session.{mcp_server_status_changed, mcp_servers_loaded, skills_loaded, tools_updated, info}`.
   - Claude startup-only set:
     `{type:'system', subtype:'init'|'hook_started'|'hook_response'}`.
   - Non-JSON lines and any other JSON `type` count as real activity.
4. Per-process CPU seconds (via `shared.getProcessCpuSeconds(pid)` —
   PowerShell on Windows, `/proc/<pid>/stat` on Linux, `ps -o cputime=`
   on macOS) ≤ `engine.spawnPhaseMaxCpuSeconds` (default 5).

On fire:

- Kill via `shared.killImmediate`.
- Write a structured `spawn-phase-stall-<id>` inbox note with the
  live-output tail.
- Complete the dispatch with
  `failureClass: SPAWN_PHASE_STALL` + `agentRetryable: true` so the next
  tick re-spawns the agent with a fresh runtime invocation.

`SPAWN_PHASE_STALL` is also in `FORCE_DEMOTE_FAILURE_CLASSES` as
defense-in-depth — a stray completion report can't paper over a wedge.

**CPU sampling fails open:** a null result skips the kill so OS-level
glitches don't masquerade as wedges.

Toggle off via `engine.spawnPhaseWatchdogEnabled: false`.
