# Upgrade Guide — vNEXT

<!-- assembled-by: assemble-next-md -->
<!-- bump: patch -->

## What Changed

Makes the SessionWatchdog's stuck-command escalation **fail CLOSED** instead of fail-open when its LLM "stuck vs legitimate" judge can't run. The watchdog flags a shell command as a candidate after 3 minutes, asks an LLM whether it's genuinely stuck or a legitimate long-running task (build/test/install), and only then sends `Ctrl+C`. Previously, when that judge was UNAVAILABLE (no provider) or ERRORED (rate-limited / circuit-open / timeout — common under load), `isCommandStuck` returned `true` (fail-open) → it Ctrl+C'd **every** command past the 3-minute threshold, interrupting legitimate test suites, builds, and `docs-coverage.mjs` runs (the session shows *"Interrupted · What should Claude do instead?"*). Now: when the judge can't run, the watchdog does NOT interrupt below a deterministic hard ceiling (`monitoring.watchdog.hardCeilingSec`, default **1800s / 30 min**); it only escalates once a command has run past that ceiling — so a genuinely hung command (e.g. `crontab -` waiting on stdin forever) is still recovered deterministically without any LLM. A ceiling of `0` disables it (pure fail-closed — never interrupt without a positive LLM "stuck" verdict). The stdin-consumer fail-closed guard and the positive-LLM-verdict path are unchanged. This is the "No Silent Degradation to Brittle Fallback" standard applied to a destructive action.

## What to Tell Your User

If you saw sessions get interrupted "out of nowhere" — a command suddenly stopping with *"Interrupted · What should Claude do instead?"* — that was my stuck-command watchdog misfiring. It Ctrl+C's commands it thinks are hung, using an AI check to spare legitimate long builds/tests; but when that check couldn't run (which happens exactly when the machine is busy), it assumed the worst and interrupted everything over 3 minutes. Now it does the safe thing instead: if it can't confirm a command is actually stuck, it leaves it alone — and only force-stops something that's been frozen for a very long time (30 min by default) with no other way to tell. So your real work stops getting killed, while a truly hung command is still recovered.

## Summary of New Capabilities

| Capability | How to Use |
|-----------|-----------|
| Watchdog fails closed when its stuck-judge is unavailable | automatic — no config needed |
| Tune/disable the deterministic hard ceiling | set `monitoring.watchdog.hardCeilingSec` (default 1800; `0` disables) |

## Evidence

Reproduction (live, from this agent's `logs/server.log`, 2026-06-09): repeated `[Watchdog] "<session>": stuck command (190s) … — sending Ctrl+C` / `(230s)` entries on legitimate autonomous work, plus the user's screenshot of a session interrupted mid-`docs-coverage.mjs --check`. These coincide with the machine under load (LLM circuit open / rate-limited), i.e. the fail-open path: `isCommandStuck` returned `true` because the judge couldn't run.

After the fix: new unit suite `tests/unit/SessionWatchdog-failclosed.test.ts` (8 tests) pins both sides — judge unavailable/erroring below the ceiling → no interrupt; past the ceiling → interrupt; ceiling 0 → never interrupt; LLM "legitimate"/"stuck" verdicts honored unchanged. `tests/unit/SessionWatchdog-pipeline.test.ts` updated (the test that asserted the old fail-open now asserts fail-closed, plus past-ceiling and LLM-confirmed-stuck cases). All 175 watchdog/triage unit tests green; `tsc` + repo lint clean.

## The problem

You saw sessions suddenly stop mid-command — a build or test or script just
halting with *"Interrupted · What should Claude do instead?"* — for no obvious
reason. It wasn't random. Here's what was actually happening.

I run a safety watchdog that watches the shell commands running inside each
session. Its job is to rescue a session if a command gets truly stuck (hangs
forever). The way it works:

1. If a command has been running more than **3 minutes**, the watchdog gets
   suspicious.
2. Lots of legitimate commands take longer than 3 minutes — installs, builds,
   big test suites. So before doing anything, the watchdog asks an AI helper:
   *"Is this command actually stuck, or is it just a normal long-running job?"*
3. If the AI says "stuck," the watchdog presses **Ctrl+C** to kill the command.
   That Ctrl+C is exactly the "Interrupted" you saw.

The bug was in step 2. When that AI helper **couldn't run** — because the
machine was busy, rate-limited, or its circuit breaker was tripped (which
happens *precisely* when lots of commands are slow) — the old code **assumed
the command was stuck and killed it anyway.** That's called "failing open." So
under load, the watchdog Ctrl+C'd basically every command that ran longer than
3 minutes — including perfectly healthy builds and test suites and the
docs-coverage check in your screenshot.

In other words: the safety net meant to rescue stuck sessions was itself
interrupting healthy ones, and worst exactly when the system was busiest.

## What already exists

- The watchdog, its 3-minute threshold, and the AI "stuck or legitimate?" check.
- A separate, already-correct rule for pipeline commands (`tail`, `grep`): those
  already refused to be killed without a positive AI confirmation. This fix
  brings the *normal* command path in line with that safer behavior.

## What's new

One change in judgment: when the AI helper can't run, the watchdog now does the
**safe** thing instead of the destructive thing. It **does not interrupt** a
command just because it can't confirm it's stuck. It only force-stops a command
once it has been running past a deterministic **hard ceiling** — 30 minutes by
default — which is long enough that a genuinely frozen command (like one waiting
forever for keyboard input that will never come) still gets cleaned up, without
needing the AI at all.

There's a new dial, `monitoring.watchdog.hardCeilingSec`, default 1800 (30 min).
Set it to `0` to turn the ceiling off entirely — then the watchdog will *never*
interrupt a command unless the AI helper positively says "stuck."

## The safeguards in plain terms

- **Safe direction by default.** Killing a real build/test is frequent and
  costly; letting a maybe-stuck command keep running for a while is rare and
  cheap. When unsure, the watchdog now leaves the command alone.
- **Genuinely stuck commands are still rescued.** The 30-minute hard ceiling is
  a deterministic backstop that needs no AI — so a command frozen on input
  (e.g. `crontab -` waiting on stdin) is still caught.
- **Nothing else changes.** When the AI helper *can* run, its "legitimate" or
  "stuck" verdict is honored exactly as before. The change only affects the case
  where it can't run.
- **This change can only kill LESS, never more** — so it can't introduce a new
  way to interrupt healthy work.

## What you need to decide

Nothing required — it ships as a normal patch with a safe default. The only
optional choice is whether to tune `hardCeilingSec` (lower it if you want hung
commands recovered faster, or set `0` to never interrupt without a positive AI
"stuck" verdict). After it deploys, your sessions stop getting Ctrl+C'd out of
nowhere when the machine is under load.
