# Feature Spec: Worker Runtime Watchdog, Liveness Polling, and Timeout Resilience (AOP)

> **Purpose of this document**: Define implementation-ready changes that make live worker execution (`codex`/`claude`/`kiro-cli`/`copilot`/`custom` CLI providers, plus API-backed providers) observable and resilient under hangs, stalls, and timeout-kill scenarios, with deterministic planner/builder/qa behavior.

**Version:** 1.0  
**Date:** 2026-03-05  
**Status:** Draft  
**Roadmap Mapping:** M44

---

## 0. Standards and Dependencies

### 0.1 Required Standards

All implementation MUST preserve:

- deterministic orchestration state transitions and normalized error envelopes
- MCP/in-process parity at tool boundary (no transport-specific semantic drift)
- strict schema and architecture checks (`validate:mcp-contracts`, `validate:architecture`)
- canonical runtime artifacts under `.aop/**`
- existing topology invariants (`1 + 3N` sessions with queued features `unassigned`)

### 0.2 Required Upstream Inputs

Implementing agents MUST read:

- `spec-files/outstanding/agentic_orchestrator_real_worker_provider_execution_spec.md`
- `apps/control-plane/src/providers/providers.ts`
- `apps/control-plane/src/providers/cli-worker-provider.ts`
- `apps/control-plane/src/providers/api-worker-provider.ts`
- `apps/control-plane/src/providers/worker-provider-factory.ts`
- `apps/control-plane/src/supervisor/worker-decision-loop.ts`
- `apps/control-plane/src/supervisor/planning-wave-executor.ts`
- `apps/control-plane/src/supervisor/build-wave-executor.ts`
- `apps/control-plane/src/supervisor/qa-wave-executor.ts`
- `apps/control-plane/src/supervisor/run-coordinator.ts`
- `apps/control-plane/src/application/services/activity-monitor-service.ts`
- `apps/control-plane/src/core/error-codes.ts`
- `agentic/orchestrator/agents.yaml`
- `agentic/orchestrator/schemas/agents.schema.json`
- `agentic/orchestrator/defaults/policy.defaults.yaml`
- `agentic/orchestrator/schemas/policy.schema.json`

### 0.3 Scope

This spec implements:

- deterministic live-worker liveness supervision for `runWorker(...)`
- polling/heartbeat-based stall detection before hard kill
- explicit timeout/stall failure taxonomy (instead of generic runtime failure only)
- role-aware policy actions for planner/builder/qa on runtime timeout/stall failures
- regression tests so `npm run verify` catches this failure class

Out of scope:

- replacing provider CLIs or vendor SDK behavior
- interactive terminal multiplexing for worker runs
- external process supervisors (systemd, PM2, k8s sidecars)

### 0.4 OOP and Architecture Guardrails (Mandatory)

Implementation for this spec MUST follow explicit object-oriented and architectural best practices:

- **KISS**: one watchdog pipeline for all worker runs; avoid provider-specific orchestration branches unless required.
- **SRP**: command execution, watchdog policy, parsing, and orchestration decisions remain separate modules.
- **SOLID-oriented design** (pragmatic):
  - open/closed for new providers via adapter composition, not supervisor branching
  - interface segregation for command runner and watchdog callbacks
  - dependency inversion from supervisor to provider contracts, not process primitives
- **Hexagonal alignment**: provider process mechanics stay in adapter layer (`providers/*`), not in wave executors.
- **Composition over inheritance**: watchdog composed into `CliWorkerProvider`/`ApiWorkerProvider`.
- **High cohesion / low coupling**: no direct supervisor imports in command runner internals.

---

## 1. Problem Statement and Failure Analysis

Observed failure signature in current runtime:

- provider worker subprocess invoked (CLI or API-backed: `claude`, `codex`, `copilot`, `gemini`, `kiro-cli`, etc.)
- process ends with `signal=SIGKILL`, `exit_code=1`, empty `stderr`
- orchestrator surfaces generic `provider_runtime_unavailable`

Current gap:

1. Timeout kill path exists, but root cause is not categorized (`timed_out` is present but not policy-driven).
2. No liveness/stall distinction:

- process can be alive but non-productive (hung waiting for input) and only fail at hard timeout.

3. No deterministic watchdog artifact for in-flight health:

- operators cannot distinguish `spawn failed` vs `stalled` vs `killed by timeout` vs `external kill`.

4. Role-level execution policy only covers malformed outputs and no-progress outputs:

- does not explicitly govern runtime timeout/stall failures.

5. No semantic progress verification between worker iterations:

- process-level liveness signals (stdout/stderr bytes) do not distinguish an agent actively
  building from one looping on context-fetch calls, emitting NOTEs, or producing patches that
  have no relation to the accepted plan. The orchestrator has no way to detect these conditions
  from process metrics alone.

Result: failures appear as generic provider errors, and operational RCA is slow. Long-running
build/QA waves may proceed for many minutes while the agent is healthy by every process metric
but is stuck in a reasoning loop or working on the wrong files, consuming compute budget and
blocking feature delivery.

### 1.1 Two Distinct Stall Classes

This spec explicitly recognizes two classes of stall that require independent detection mechanisms:

**Class A — Process-Level Stall**: the provider subprocess is alive but emits no stdout/stderr
bytes within an idle window. Indicates the process is hung waiting for input, deadlocked, or
otherwise not communicating. Detectable at the command runner layer via lifecycle callbacks.

**Class B — Semantic Stall**: the provider subprocess is healthy (returning from `runWorker`
successfully with valid JSON) but the agent is not making meaningful progress. This class has
two sub-types:

- **B1 — Output Loop**: the agent produces the same or structurally identical output across N
  consecutive iterations, indicating it is in a deterministic reasoning loop — seeing the same
  context, generating the same response, and the orchestrator's application of that response
  changes nothing about the situation.

- **B2 — Plan Conformance Failure**: the agent is producing output (PATCHes, NOTEs, REQUESTs)
  but none of the patches touch the files declared in the accepted plan's `files.create` or
  `files.modify` lists, and/or patches are landing on files outside `allowed_areas`. The agent
  is active but working on the wrong thing.

Class B is not detectable at the process layer. It requires post-hoc analysis of the content and
structure of what the agent produced, cross-referenced against the accepted plan. Whether a
provider adapter streams tokens in real time or buffers all output until subprocess exit is a
provider capability detail — Class B detection is post-hoc regardless, because it operates on
the semantic content of the completed output, not on process signals. All Class B detection
happens between iterations, after each `runWorker` call completes and returns its full output.

Both classes must be detected, classified, and governed by independent policy actions. Neither
subsumes the other. Class B detection operates entirely on data the supervisor already holds —
the plan (`plan.json`) and the worker output (`WorkerDecisionResult`) — with no additional
subprocess invocations or filesystem access required.

---

## 2. Revised Architecture Fit

### 2.1 Two-Tier Stall Detection Architecture

Stall detection is split across two layers that operate independently and complement each other:

**Tier 1 — Provider Layer (Class A: Process-Level Stall)**

Responsibility: `providers/` adapter layer.

Detects subprocess silence within a single `runWorker(...)` invocation. Each provider adapter
exposes a `WorkerHandle` that declares its capabilities (`supportsIdleDetection`,
`supportsSpawnDetection`) and emits provider-agnostic lifecycle events (`spawned`, `activity`,
`heartbeat`, `terminated`). The watchdog service tracks time since last activity event and
enforces spawn and idle deadlines — but only for capabilities the adapter declares it supports.
When a deadline fires, the subprocess is terminated and the error is classified as
`provider_spawn_timeout` or `provider_stall_timeout`. This tier has no visibility into what the
agent is producing — only that the process is communicating.

**Tier 2 — Supervisor Layer (Class B: Semantic Stall)**

Responsibility: `supervisor/` wave executors.

Detects semantic stall across multiple `runWorker(...)` invocations using two independent checks
that operate on data already available to the supervisor — no additional subprocesses or
filesystem access required:

- **B1 — `OutputLoopDetector`**: After each iteration, computes a normalized content fingerprint
  of the complete worker output (patch paths + content, request action + params, note text). Tracks
  a rolling window of fingerprints per feature. If the same fingerprint repeats for N consecutive
  iterations, the agent is in a deterministic loop: same input context → same output → no change
  in state → same input context again. Emits `provider_output_loop`.

- **B2 — `PlanConformanceScorer`**: After each iteration, cross-references the set of file paths
  touched by any PATCH outputs against the accepted plan's `files.create`, `files.modify`, and
  `allowed_areas` fields. Tracks the count of consecutive iterations where no patch touches any
  plan-declared file. If this count reaches threshold, the agent is working off-plan. Emits
  `provider_plan_conformance_stall`.

Both checks are post-hoc (run after `runWorker` returns), purely in-memory, and have no
visibility into process health.

Both tiers journal structured events and expose policy-governed actions. They do not share state
and can trigger independently.

### 2.2 Where Each Component Lives

| Component                                        | Layer      | File                                                                                    |
| ------------------------------------------------ | ---------- | --------------------------------------------------------------------------------------- |
| `ProviderCommandRunner` with lifecycle callbacks | Provider   | `providers/providers.ts`                                                                |
| `WorkerWatchdog`                                 | Provider   | `providers/worker-watchdog.ts` (new)                                                    |
| `CliWorkerProvider` integration                  | Provider   | `providers/cli-worker-provider.ts`                                                      |
| `ApiWorkerProvider` integration                  | Provider   | `providers/api-worker-provider.ts`                                                      |
| `OutputLoopDetector`                             | Supervisor | `supervisor/output-loop-detector.ts` (new)                                              |
| `PlanConformanceScorer`                          | Supervisor | `supervisor/plan-conformance-scorer.ts` (new)                                           |
| Wave executor integration                        | Supervisor | `supervisor/build-wave-executor.ts`, `planning-wave-executor.ts`, `qa-wave-executor.ts` |
| Worker event journaling                          | Supervisor | `supervisor/worker-decision-loop.ts`                                                    |

### 2.3 New Runtime Flow (CLI Providers)

1. `WorkerProvider.runWorker(...)` starts command execution and returns a `WorkerHandle` with
   declared `WorkerCapabilities`.
2. Provider adapter emits provider-agnostic lifecycle events via `WorkerHandle.onLifecycleEvent`:
   `spawned`, `activity` (any real-time signal that work is in progress), `heartbeat`, `terminated`.
3. **Tier 1 watchdog** evaluates:
   - spawn timeout: process did not reach ready state within `worker_spawn_timeout_ms`
   - idle/stall timeout: no lifecycle event received within `worker_idle_timeout_ms`
   - hard timeout: overall duration exceeded `worker_response_timeout_ms`

4. Runner terminates process deterministically (SIGTERM then SIGKILL after grace period).
5. Provider maps terminal result to explicit error code and details.
6. `WorkerDecisionLoop` journals structured event with watchdog metadata.
7. Wave executor applies patch outputs to worktree (or records failed applies).
8. **Tier 2A** (`OutputLoopDetector`) fingerprints complete normalized output, compares to rolling window; fires `provider_output_loop` if loop detected.
9. **Tier 2B** (`PlanConformanceScorer`) cross-references patch file paths against plan; fires `provider_plan_conformance_stall` if N consecutive iterations have no plan-aligned patches.
10. If either Tier 2 check fires, the configured policy action is applied for that check.

### 2.4 Cluster Agent Impact

- **Planner**: Tier 1 applies. Tier 2A (loop detection) applies — a planner that repeats the same REQUEST or PLAN_SUBMISSION across N iterations is looping. Tier 2B (conformance) does not apply — there is no accepted plan to conform to during planning.
- **Builder**: All tiers apply. Builder is the primary use case for both Tier 2 checks.
- **QA**: All tiers apply. QA emits PATCHes to fix test failures; loop detection catches repeated identical patch attempts; conformance scoring catches QA patches that don't touch test files in the plan.
- **Orchestrator**: continues processing other active features unless policy requires `fail_run`; Tier 2 stalls are feature-scoped and do not cascade.

---

## 3. Contract and Schema Deltas

## 3.1 Agents Runtime Contract (`agents.schema.json`)

Extend `runtime` with Tier 1 watchdog controls and Tier 2 semantic audit controls:

```yaml
runtime:
  # Tier 1 — process-level watchdog
  worker_response_timeout_ms: 600000 # hard timeout (10 min); accommodates large-context API latency
  worker_spawn_timeout_ms: 15000 # max time from spawn to first stdout/stderr byte
  worker_idle_timeout_ms: 120000 # max silence window after spawn (no stdout/stderr)
  worker_watchdog_poll_interval_ms: 1000 # watchdog evaluation cadence
  worker_kill_grace_period_ms: 1500 # SIGTERM -> SIGKILL escalation window

  # Tier 2A — output loop detection
  output_loop_detection_enabled: true # enable/disable B1 check entirely
  output_loop_stall_iterations: 3 # consecutive identical-output iterations = loop stall

  # Tier 2B — plan conformance scoring
  plan_conformance_check_enabled: true # enable/disable B2 check entirely
  plan_conformance_stall_iterations: 5 # consecutive iterations with zero plan-aligned patches = conformance stall
```

Rules for Tier 1:

- all values must be positive integers
- `worker_spawn_timeout_ms < worker_response_timeout_ms`
- `worker_idle_timeout_ms <= worker_response_timeout_ms`
- invalid combinations fail schema validation

Rules for Tier 2A (loop detection):

- `output_loop_stall_iterations` must be a positive integer ≥ 2
- active for planner, builder, and qa roles; fingerprint scope is the full normalized output struct
- counter resets whenever the output fingerprint changes between iterations
- counter resets when a feature is unblocked or resumed

Rules for Tier 2B (plan conformance):

- `plan_conformance_stall_iterations` must be a positive integer ≥ 2; defaults higher than 2A
  because legitimate early iterations may fetch context before producing plan-aligned patches
- active for builder and qa only; planner is exempt (no accepted plan exists yet during planning)
- "plan-aligned patch" means at least one path in a PATCH output appears in `files.create`,
  `files.modify`, or matches a prefix in `allowed_areas` from the accepted `plan.json`
- counter resets whenever a plan-aligned patch is produced
- counter resets when a feature is unblocked or resumed

## 3.2 Policy Contract (`policy.schema.json`)

Extend `execution` with Tier 1 and Tier 2 policy actions:

```yaml
execution:
  # Tier 1 — process-level watchdog actions
  provider_timeout_action: block_feature # block_feature | fail_run
  provider_stall_action: block_feature # block_feature | fail_run
  provider_spawn_failure_action: fail_run # block_feature | fail_run

  # Tier 2A — output loop action
  output_loop_stall_action: block_feature # block_feature | fail_run

  # Tier 2B — plan conformance action
  plan_conformance_stall_action: block_feature # block_feature | fail_run
```

Rules:

- `provider_spawn_failure_action` defaults to `fail_run`: a broken spawn indicates the runtime
  environment is broken for all features, not just one.
- `provider_timeout_action` and `provider_stall_action` default to `block_feature`: a single
  slow or hung invocation may recover on the next iteration.
- `output_loop_stall_action` defaults to `block_feature`: a looping agent may resolve when
  restarted with refreshed context after block; `fail_run` is appropriate for zero-tolerance
  environments where any loop is considered a hard failure.
- `plan_conformance_stall_action` defaults to `block_feature`: an off-plan agent may benefit from
  human review of the plan rather than a hard run failure; `fail_run` is appropriate when strict
  plan adherence is required.
- all action decisions must be logged in the worker event journal with the triggering metric.

## 3.3 Error Code Additions

Add to `apps/control-plane/src/core/error-codes.ts`:

**Tier 1 (process-level):**

- `provider_spawn_timeout` — process did not produce first output within `worker_spawn_timeout_ms`
- `provider_stall_timeout` — process produced no output for `worker_idle_timeout_ms` after spawning
- `provider_hard_timeout` — process did not exit within `worker_response_timeout_ms`
- `provider_process_killed` — process received an external signal before any timeout fired

**Tier 2A (output loop detection):**

- `provider_output_loop` — normalized output fingerprint identical for `output_loop_stall_iterations`
  consecutive iterations across any role

**Tier 2B (plan conformance scoring):**

- `provider_plan_conformance_stall` — no plan-aligned patch produced in `plan_conformance_stall_iterations`
  consecutive builder/QA iterations

Mapping rules (Tier 1):

- `timedOut=true` and watchdog reason is hard deadline → `provider_hard_timeout`
- watchdog idle deadline fires → `provider_stall_timeout`
- spawn deadline fires before first lifecycle event → `provider_spawn_timeout`
- signal present and `timedOut=false` → `provider_process_killed`

Mapping rules (Tier 2A):

- consecutive identical-fingerprint count ≥ `output_loop_stall_iterations` → `provider_output_loop`

Mapping rules (Tier 2B):

- consecutive no-conforming-patch count ≥ `plan_conformance_stall_iterations` → `provider_plan_conformance_stall`

## 3.4 `agents.schema.json` Normative Delta

The following fields must be added to the `runtime` object definition in
`agentic/orchestrator/schemas/agents.schema.json`:

```json
"worker_spawn_timeout_ms":              { "type": "integer", "minimum": 1 },
"worker_idle_timeout_ms":               { "type": "integer", "minimum": 1 },
"worker_watchdog_poll_interval_ms":     { "type": "integer", "minimum": 1 },
"worker_kill_grace_period_ms":          { "type": "integer", "minimum": 1 },
"output_loop_detection_enabled":        { "type": "boolean" },
"output_loop_stall_iterations":         { "type": "integer", "minimum": 2 },
"plan_conformance_check_enabled":       { "type": "boolean" },
"plan_conformance_stall_iterations":    { "type": "integer", "minimum": 2 }
```

Corresponding defaults must be added to
`agentic/orchestrator/defaults/policy.defaults.yaml` under the `runtime` key.

## 3.5 Worker Runtime Event Artifact Contract

Path:

```text
.aop/runtime/worker-events/<run_id>.jsonl
```

**Tier 1 (process-level timeout/stall) event shape:**

```json
{
  "ts": "2026-03-05T00:00:00.000Z",
  "run_id": "run:...",
  "feature_id": "feature_x",
  "role": "builder",
  "provider": "claude",
  "model": "local-default",
  "phase": "execution",
  "event_type": "worker_timeout",
  "watchdog_reason": "idle_timeout",
  "timeout_ms": 600000,
  "idle_for_ms": 120453,
  "exit_code": 1,
  "signal": "SIGKILL",
  "valid": false,
  "error_code": "provider_stall_timeout"
}
```

**Tier 2A (output loop) event shape:**

```json
{
  "ts": "2026-03-05T00:00:00.000Z",
  "run_id": "run:...",
  "feature_id": "feature_x",
  "role": "builder",
  "provider": "claude",
  "model": "local-default",
  "phase": "build",
  "event_type": "worker_output_loop",
  "output_fingerprint": "sha256:3f4a9c...",
  "consecutive_identical_iterations": 3,
  "loop_threshold": 3,
  "last_unique_iteration": 2,
  "valid": false,
  "error_code": "provider_output_loop",
  "policy_action": "block_feature"
}
```

**Tier 2B (plan conformance stall) event shape:**

```json
{
  "ts": "2026-03-05T00:00:00.000Z",
  "run_id": "run:...",
  "feature_id": "feature_x",
  "role": "builder",
  "provider": "claude",
  "model": "local-default",
  "phase": "build",
  "event_type": "worker_plan_conformance_stall",
  "consecutive_nonconforming_iterations": 5,
  "conformance_threshold": 5,
  "last_conforming_iteration": 1,
  "plan_declared_files": ["apps/control-plane/src/foo.ts"],
  "actual_patch_files": ["apps/control-plane/src/unrelated.ts"],
  "valid": false,
  "error_code": "provider_plan_conformance_stall",
  "policy_action": "block_feature"
}
```

Rules:

- append-only JSONL
- `event_type` must be one of:
  - `worker_started`
  - `worker_heartbeat`
  - `worker_completed`
  - `worker_failed`
  - `worker_timeout`
  - `worker_output_loop`
  - `worker_plan_conformance_stall`
- ordering per run is chronological append order
- Tier 2 events are emitted by the supervisor layer (wave executor) after each iteration, not
  by the provider adapter; they will never appear interleaved with process-level events mid-invocation

---

## 4. Detailed Design

## 4.1 WorkerHandle and WorkerCapabilities Contract (Tier 1)

Provider adapters expose a `WorkerHandle` interface to the watchdog and supervisor layers.
This is the sole contract between the supervisor and any worker provider — CLI-based, API-based,
or otherwise. No provider-specific details (process primitives, CLI flags, SDK types) leak through
this boundary.

```ts
/**
 * Declares which real-time liveness signals a provider adapter can emit.
 * Adapters that buffer all output until completion (e.g. stateless CLI invocations)
 * MUST set supportsIdleDetection: false. Adapters that stream intermediate tokens
 * or progress events MAY set supportsIdleDetection: true.
 *
 * The watchdog only activates deadlines for capabilities the adapter declares.
 * Declaring a capability the adapter cannot fulfill will produce false positive kills.
 */
interface WorkerCapabilities {
  supportsIdleDetection: boolean; // true if adapter emits `activity` events during execution
  supportsSpawnDetection: boolean; // true if adapter can signal when the worker is "ready"
}

/**
 * Provider-agnostic lifecycle events. Adapters emit these on the WorkerHandle's
 * onLifecycleEvent callback. The event vocabulary is intentionally minimal and
 * does not reference process internals (no stdout/stderr distinction).
 */
type WorkerLifecycleEventType =
  | 'spawned' // worker process/request has started and is accepting input
  | 'activity' // any real-time signal that work is in progress (token, chunk, progress line)
  | 'heartbeat' // periodic keepalive if the adapter supports explicit heartbeating
  | 'terminated'; // worker has exited or the API request has completed/failed

interface WorkerLifecycleEvent {
  type: WorkerLifecycleEventType;
  ts: number; // Date.now() at emission
  byteCount?: number; // optional: I/O volume for adapters that track it
}

/**
 * The handle returned by WorkerProvider.runWorker(...) before the worker completes.
 * The supervisor holds this handle for the duration of the invocation.
 */
interface WorkerHandle {
  capabilities: WorkerCapabilities;
  onLifecycleEvent?: (event: WorkerLifecycleEvent) => void;
  kill(signal: 'SIGTERM' | 'SIGKILL'): void;
}
```

Lifecycle events are emitted by the adapter as they occur. The supervisor must never throw
inside a lifecycle callback; the adapter guards with try/catch and drops callback errors silently.
The adapter remains responsible for returning a final `ProviderCommandResult` regardless of
whether a `WorkerHandle` consumer is attached.

## 4.2 Watchdog Service (New) — Tier 1

Create `apps/control-plane/src/providers/worker-watchdog.ts`.

**Responsibilities:**

- receive `WorkerCapabilities` from the provider adapter at construction time
- track time of last `activity` event (when `supportsIdleDetection: true`)
- evaluate deadlines conditionally based on declared capabilities:
  - spawn timeout: only when `supportsSpawnDetection: true`
  - idle/stall timeout: only when `supportsIdleDetection: true`
  - hard timeout: always, regardless of capabilities
- request termination escalation via injected `WorkerHandle.kill` (`SIGTERM` then `SIGKILL`)
- produce a structured `WatchdogOutcome` containing classification metadata

**Non-responsibilities:**

- parsing model output
- deciding feature state transitions
- writing kernel state
- knowing which provider CLI or SDK is in use

**Capability gating rules:**

- If `supportsSpawnDetection: false`, spawn timeout is disabled. The watchdog does not track
  whether a `spawned` event was received.
- If `supportsIdleDetection: false`, idle timeout is disabled. The watchdog does not track
  time since last `activity` event. Hard timeout is the only process-level deadline.
- These are not error conditions — buffering providers legitimately cannot provide real-time
  signals. The capability system makes this explicit rather than producing false-positive kills.

**Interface:**

```ts
interface WatchdogOptions {
  capabilities: WorkerCapabilities; // declared by the adapter
  spawnTimeoutMs: number; // used only if supportsSpawnDetection: true
  idleTimeoutMs: number; // used only if supportsIdleDetection: true
  hardTimeoutMs: number; // always enforced
  pollIntervalMs: number;
  killGracePeriodMs: number;
  handle: WorkerHandle; // adapter handle; kill() called on deadline
}

type WatchdogReason =
  | 'spawn_timeout'
  | 'idle_timeout'
  | 'hard_timeout'
  | 'external_kill'
  | 'clean_exit';

interface WatchdogOutcome {
  reason: WatchdogReason;
  idleForMs: number | null; // ms since last activity when watchdog fired; null if idle detection unsupported
  elapsedMs: number; // total wall time from start to outcome
  timedOut: boolean; // true for spawn/idle/hard timeout paths
}

class WorkerWatchdog {
  constructor(options: WatchdogOptions) {}
  start(): void; // begin polling loop
  recordActivity(type: WorkerLifecycleEventType): void; // called by adapter via WorkerHandle.onLifecycleEvent
  stop(): WatchdogOutcome; // called after worker completes normally
}
```

**Deadline evaluation order** (evaluated each poll interval):

1. If `supportsSpawnDetection: true` and spawn deadline exceeded and no `spawned` event recorded → fire `spawn_timeout`
2. If `supportsIdleDetection: true` and idle deadline exceeded since last `activity` event → fire `idle_timeout`
3. If hard deadline exceeded → fire `hard_timeout`

**Termination escalation** when any deadline fires:

1. Call `killFn('SIGTERM')`
2. Wait `killGracePeriodMs`
3. If process still running, call `killFn('SIGKILL')`

## 4.3 Provider Adapter Integration (Tier 1)

Every provider adapter (`CliWorkerProvider`, `ApiWorkerProvider`, or any future adapter) MUST:

1. Declare a `WorkerCapabilities` object that honestly reflects what real-time signals the
   underlying runtime provides:
   - CLI adapters using subprocess stdout/stderr streaming: `{ supportsIdleDetection: true, supportsSpawnDetection: true }`
   - CLI adapters that buffer all output until exit (stateless invocations): `{ supportsIdleDetection: false, supportsSpawnDetection: true }`
   - API adapters with SSE/streaming: `{ supportsIdleDetection: true, supportsSpawnDetection: false }`
   - API adapters with request-response only: `{ supportsIdleDetection: false, supportsSpawnDetection: false }`
2. Read `spawnTimeoutMs`, `idleTimeoutMs`, `hardTimeoutMs`, `pollIntervalMs`, and
   `killGracePeriodMs` from the `runtime` config loaded at construction time.
3. Construct a `WorkerHandle` and a `WorkerWatchdog` with the declared capabilities and
   a reference to the handle's `kill()` method.
4. Emit `WorkerLifecycleEvent` objects on the handle's `onLifecycleEvent` callback as the
   underlying runtime produces them (subprocess stdout/stderr bytes → `activity`; spawn → `spawned`).
5. Call `watchdog.start()` when execution begins; call `watchdog.stop()` on completion.
6. Merge `WatchdogOutcome` into the error details when the worker exits non-zero:

**Normative error detail keys (Tier 1):**

```ts
{
  timed_out: boolean;
  watchdog_reason: WatchdogReason;
  timeout_ms: number; // hard timeout
  idle_timeout_ms: number;
  spawn_timeout_ms: number;
  idle_for_ms: number | null;
  elapsed_ms: number;
  signal: string | null;
  exit_code: number | null;
  stderr: string;
}
```

**Error code mapping** (applied in `ensureRunSucceeded`):

| Condition                                  | Error Code                     |
| ------------------------------------------ | ------------------------------ |
| `watchdog_reason === 'spawn_timeout'`      | `provider_spawn_timeout`       |
| `watchdog_reason === 'idle_timeout'`       | `provider_stall_timeout`       |
| `watchdog_reason === 'hard_timeout'`       | `provider_hard_timeout`        |
| `signal` present and `timed_out === false` | `provider_process_killed`      |
| non-zero exit, no signal, no timeout       | `provider_runtime_unavailable` |

## 4.4 Capability Declaration by Provider Type

The table below lists normative capability declarations for the currently supported provider types.
New provider adapters MUST add a row here and declare capabilities accurately.

| Provider                      | Execution Model            | `supportsSpawnDetection` | `supportsIdleDetection` | Notes                                                     |
| ----------------------------- | -------------------------- | ------------------------ | ----------------------- | --------------------------------------------------------- |
| `claude` (stateless)          | Subprocess, buffers output | `true`                   | `false`                 | Subprocess spawns detectably; output arrives only at exit |
| `claude` (streaming)          | Subprocess, streams tokens | `true`                   | `true`                  | If invoked with a streaming flag                          |
| `codex`                       | Subprocess                 | `true`                   | adapter-defined         | Declare based on observed stdout behavior                 |
| `copilot`                     | Subprocess or API          | adapter-defined          | adapter-defined         | Declare per integration                                   |
| `gemini`                      | Subprocess or API          | adapter-defined          | adapter-defined         | Declare per integration                                   |
| `kiro-cli`                    | Subprocess                 | `true`                   | adapter-defined         | Declare based on observed stdout behavior                 |
| API-backed (SSE)              | Network streaming          | `false`                  | `true`                  | No subprocess spawn; SSE chunks → `activity` events       |
| API-backed (request-response) | Network, blocking          | `false`                  | `false`                 | Hard timeout only                                         |

Rules for all adapters:

- Hard timeout is always enforced regardless of capabilities.
- Idle timeout fires only when `supportsIdleDetection: true` AND the idle window elapses.
- Spawn timeout fires only when `supportsSpawnDetection: true` AND the spawn window elapses.
- Misrepresenting capabilities (declaring `supportsIdleDetection: true` when the adapter does not
  stream) WILL produce spurious `provider_stall_timeout` kills. Honest declaration is mandatory.

## 4.5 Semantic Progress Detectors (New) — Tier 2

### 4.5.1 Architectural Constraint: Post-Hoc by Design

Both Tier 2 detectors operate exclusively on data returned by `runWorker` after it completes.
This is a deliberate design choice, not a limitation of any specific provider. Class B detection
analyzes the semantic content and structure of the completed output — what the agent produced,
cross-referenced against what the plan declared. This is orthogonal to whether the underlying
provider streams tokens or buffers output: the analysis requires the full, parsed result, and
cannot be done incrementally on raw byte streams.

This design has a key consequence: a provider that declares `supportsIdleDetection: false` (i.e.
no real-time activity signals) still benefits from full Class B semantic stall detection after
each iteration. The two-tier architecture ensures that Tier 1 gaps (no idle detection) are
complemented by Tier 2 post-hoc analysis. Neither tier depends on the other.

No additional subprocesses, filesystem access, or network calls are required for either Tier 2
check — all inputs are already held in memory by the supervisor.

### 4.5.2 Output Loop Detector (Tier 2A)

Create `apps/control-plane/src/supervisor/output-loop-detector.ts`.

**Purpose**: Detect when an agent is producing structurally identical output across multiple
iterations, indicating it is caught in a deterministic reasoning loop. The agent receives the
same context (because its previous output didn't change anything), generates the same response,
and will continue indefinitely until externally interrupted.

**Common causes:**

- Agent keeps submitting the same `REQUEST` action (e.g. `request_context`) that the orchestrator
  cannot satisfy differently
- Agent submits a `PATCH` that fails to apply every time, falls back to the same `NOTE` about
  the failure, receives the same context in the next iteration
- Agent emits the same NOTE text each iteration, stuck analyzing the same problem without resolution

**Fingerprint mechanism:**

The output fingerprint is computed over a normalized representation of the complete
`WorkerDecisionResult`:

1. Extract all outputs from the result (patches, notes, requests)
2. For each PATCH output: normalize to sorted `(path, unified_diff_body)` pairs, stripping
   line number offsets that may shift between applications. Two patches touching the same file
   with the same content but different context line numbers must hash identically.
3. For each REQUEST output: normalize to `(action, sorted_param_keys_and_values)`
4. For each NOTE output: normalize to trimmed text content
5. Sort the normalized output list by type then content
6. SHA-256 hash the canonical JSON serialization of the sorted list

The resulting fingerprint is stable across cosmetically different outputs that carry the same
semantic content, while being sensitive to genuine changes in intent.

**Interface:**

```ts
interface LoopDetectorState {
  lastFingerprint: string | null;
  consecutiveIdentical: number;
  lastUniqueIteration: number;
  totalIterations: number;
}

interface LoopDetectionResult {
  fingerprint: string;
  isLoop: boolean; // consecutiveIdentical >= threshold
  consecutiveIdentical: number;
  lastUniqueIteration: number;
}

class OutputLoopDetector {
  constructor(options: {
    loopThreshold: number; // from output_loop_stall_iterations
  }) {}

  check(featureId: string, result: WorkerDecisionResult): LoopDetectionResult;
  reset(featureId: string): void;
  delete(featureId: string): void;
}
```

`check` is synchronous and pure — it takes the decision result, updates internal state, and
returns a result object. It performs no I/O.

### 4.5.3 Plan Conformance Scorer (Tier 2B)

Create `apps/control-plane/src/supervisor/plan-conformance-scorer.ts`.

**Purpose**: Detect when an agent is actively producing output but that output is not directed
at the files and areas declared in the accepted plan. This answers the question "is the agent
working on the right thing?" — the plan is the machine-readable specification of what the right
thing is. An agent producing patches on files entirely outside the plan's declared scope is
either misreading the spec, hallucinating file paths, or working on a problem it invented.

**What constitutes a "plan-aligned" patch:**

A PATCH output is plan-aligned if at least one of its target file paths satisfies ANY of:

1. The path appears exactly in `plan.files.create`
2. The path appears exactly in `plan.files.modify`
3. The path matches a prefix in `plan.allowed_areas` (i.e. `path.startsWith(allowedArea)`)

A PATCH output on a file that satisfies none of these conditions is non-conforming. Iterations
with no PATCH outputs at all (only NOTEs or REQUESTs) are treated as non-conforming for the
purposes of this counter, since forward progress on the plan requires patches.

**Why the threshold is higher than Tier 2A:**

Early build iterations legitimately fetch context (`REQUEST`) or summarize understanding (`NOTE`)
before producing code. A threshold of 5 (vs. 3 for loop detection) provides a grace window for
this setup phase before declaring conformance failure.

**Interface:**

```ts
interface ConformanceState {
  consecutiveNonConforming: number;
  lastConformingIteration: number;
  totalIterations: number;
}

interface ConformanceResult {
  conforming: boolean; // at least one plan-aligned patch this iteration
  isStalled: boolean; // consecutiveNonConforming >= threshold
  consecutiveNonConforming: number;
  lastConformingIteration: number;
  planDeclaredFiles: string[]; // files.create + files.modify from plan
  actualPatchFiles: string[]; // file paths from PATCH outputs this iteration
}

class PlanConformanceScorer {
  constructor(options: {
    stallThreshold: number; // from plan_conformance_stall_iterations
  }) {}

  check(
    featureId: string,
    result: WorkerDecisionResult,
    plan: AcceptedPlan, // plan.files + plan.allowed_areas
  ): ConformanceResult;
  reset(featureId: string): void;
  delete(featureId: string): void;
}
```

`check` is synchronous and pure. It reads `plan.files.create`, `plan.files.modify`, and
`plan.allowed_areas` from the injected plan object. The plan object is loaded once per feature
at build wave start from `.aop/features/<id>/plan.json` via the existing kernel tool call
(`feature.get_context`) and cached for the duration of the build wave.

### 4.5.4 Integration Point in Wave Executors

Both detectors are called by `BuildWaveExecutor` and `QaWaveExecutor` after each worker
iteration. The sequence within a single feature's iteration in `runBatch`:

```
1. workerDecisionRunner.execute(...)     → WorkerDecisionResult
2. patch outputs applied to worktree
3. loopDetector.check(featureId, result) → LoopDetectionResult
   if isLoop:
     → journal worker_output_loop event
     → apply output_loop_stall_action policy
     → if block_feature: mark blocked, skip gate run; return
     → if fail_run: throw provider_output_loop error
4. conformanceScorer.check(featureId, result, plan) → ConformanceResult
   if isStalled:
     → journal worker_plan_conformance_stall event
     → apply plan_conformance_stall_action policy
     → if block_feature: mark blocked, skip gate run; return
     → if fail_run: throw provider_plan_conformance_stall error
5. if neither stall fired: continue to gate run
```

Loop detection (step 3) runs before conformance scoring (step 4). A loop stall short-circuits
the iteration; conformance is only evaluated if no loop is detected. This ordering is deliberate:
a loop is a stronger signal (the agent is completely stuck) and should take precedence.

`PlanningWaveExecutor` runs `OutputLoopDetector` only (step 3). It does not run conformance
scoring — there is no accepted plan during planning.

### 4.5.5 Role Exemptions

- **Planner**: `OutputLoopDetector` applies. `PlanConformanceScorer` does NOT apply.
- **Builder**: Both detectors apply on every iteration (subject to enabled flags).
- **QA**: Both detectors apply. For conformance scoring, the plan's `files.modify` list is the
  primary reference since QA is expected to patch test and source files already declared.

### 4.5.6 State Lifecycle

Both detectors maintain per-feature state across iterations within a single run:

- **On feature start**: state is implicitly initialized to zero on first `check` call
- **On fingerprint/conformance change**: consecutive counter resets to 0
- **On feature block**: state is preserved so that a re-run after unblock starts fresh
  (wave executor calls `reset(featureId)` when unblocking)
- **On feature completion/removal**: `delete(featureId)` is called to free memory

## 4.6 Worker Decision Loop Integration (Both Tiers)

`WorkerDecisionLoop.execute(...)`:

- On Tier 1 provider runtime exception: append worker event with mapped error code, watchdog
  reason, and full watchdog metadata. Do not collapse specific timeout/stall codes to the generic
  `provider_runtime_unavailable` when a more specific code is available.
- Tier 2 events (`worker_output_loop`, `worker_plan_conformance_stall`) are NOT emitted by
  `WorkerDecisionLoop`; they are emitted directly by the wave executor after the decision loop
  returns, once the detector checks have run against the full result.

## 4.7 Wave Policy Integration (Both Tiers)

`PlanningWaveExecutor`:

- Apply Tier 1 watchdog policy only (no auditor):
  - spawn failure → `provider_spawn_failure_action`
  - stall timeout → `provider_stall_action`
  - hard timeout → `provider_timeout_action`

`BuildWaveExecutor` and `QaWaveExecutor`:

- Apply all Tier 1 policy actions (same as planner).
- After each iteration, run `OutputLoopDetector.check(featureId, result)`. If loop detected:
  - journal `worker_output_loop` event with fingerprint and consecutive count metadata
  - apply `output_loop_stall_action`:
    - `block_feature` → add feature to blocked set, skip gate run for this cycle; return
    - `fail_run` → throw `AppError` with code `provider_output_loop`
- If no loop: run `PlanConformanceScorer.check(featureId, result, plan)`. If stalled:
  - journal `worker_plan_conformance_stall` event with nonconforming count and file lists
  - apply `plan_conformance_stall_action`:
    - `block_feature` → add feature to blocked set, skip gate run for this cycle
    - `fail_run` → throw `AppError` with code `provider_plan_conformance_stall`
- Preserve existing malformed/no-progress policy logic (those counters are unaffected by Tier 2).
- Ensure blocked features skip remaining work in the current cycle, including gate runs.

`PlanningWaveExecutor`:

- Apply Tier 1 policy actions.
- Run `OutputLoopDetector.check(featureId, result)` after each iteration.
- Does NOT run `PlanConformanceScorer`.

`SupervisorRuntime`:

- Construct `OutputLoopDetector` and `PlanConformanceScorer` at run initialization when their
  respective enabled flags are true.
- Pass detector instances to wave executors via constructor injection.
- Call `detector.reset(featureId)` and `scorer.reset(featureId)` when a feature is resumed
  from blocked state so stall counters start clean.

---

## 5. Edge-Case Matrix (Normative)

**Tier 1 — Process-Level Edge Cases**

1. Process never spawns (ENOENT):
   - Classify as `provider_runtime_unavailable` with `error_code=ENOENT`; apply spawn-failure policy.
   - Auditor is not called (no iteration completed).

2. Process spawns but produces no output and never exits:
   - Idle timeout fires; classify `provider_stall_timeout`.
   - Auditor is not called (worker iteration did not return normally).

3. Process emits periodic stdout but never exits:
   - Each stdout chunk resets the idle timer; hard timeout fires at `worker_response_timeout_ms`.
   - Classify `provider_hard_timeout`.
   - Auditor is not called.

4. Process receives external SIGKILL before any timeout:
   - Classify `provider_process_killed` with `timed_out=false`.
   - Auditor is not called.

5. Process exits 0 with empty or invalid JSON payload:
   - Parser path; classify `provider_output_invalid`. Not a watchdog failure.
   - Auditor IS called (iteration completed); if no patches applied, unchanged fingerprint is recorded.

6. Very large stdout stream:
   - Each chunk refreshes the idle timer; memory growth is bounded by the command runner's stdout
     accumulation — the existing buffered approach is preserved. Watchdog heartbeat events are
     emitted by the poll interval independent of stdout rate.

7. Multi-feature run where one feature hits Tier 1 timeout:
   - Only that feature is blocked/failed per policy. Other features continue unaffected.

8. QA rotation with prior Tier 1 timeout failure:
   - No QA session rotation if the QA cycle failed pre-completion (existing invariant preserved).

9. Resume with previous watchdog-failed (blocked) state:
   - Resume honors existing blocked status until manually unblocked.
   - On unblock, `progressAuditor.reset(featureId)` is called so stall counter restarts cleanly.

10. Attach/send commands:
    - Unaffected by Tier 1 or Tier 2 watchdog. These commands use separate templates and
      timeout handling.

**Tier 2A — Output Loop Edge Cases**

11. Agent emits only NOTE outputs for N iterations (no patches):
    - Each NOTE iteration produces a distinct normalized fingerprint keyed on note text.
    - If the note text is identical each iteration (agent repeating the same observation),
      loop counter increments. At threshold, `output_loop_stall_action` fires.
    - If the note text varies (agent describing different blockers), loop counter resets. The
      agent is not looping; it is genuinely processing new information. No stall fires.

12. Agent emits the same REQUEST action with identical params N times:
    - Normalized fingerprint is identical each iteration; loop counter increments.
    - At threshold, `output_loop_stall_action` fires.
    - This is the prototypical loop case: the agent keeps asking for something the orchestrator
      cannot provide differently, expecting a different result.

13. Agent varies its REQUEST params slightly each iteration (e.g. different context_hint):
    - Normalized fingerprint differs each iteration; loop counter resets.
    - No loop stall fires. This is intentional: small variations indicate the agent is
      exploring the problem space, not stuck on a fixed response.

14. Agent emits a PATCH that fails to apply, then falls back to the same NOTE:
    - Iteration 1: PATCH (fails apply) + NOTE. Fingerprint recorded.
    - Iteration 2: PATCH (fails apply) + NOTE. Same content → same fingerprint. Counter = 1.
    - Iteration N: counter reaches threshold. `output_loop_stall_action` fires.
    - This is correct: a repeatedly rejected patch is a loop even if it looks like activity.

15. Feature is unblocked and resumes after a Tier 2A loop stall:
    - `loopDetector.reset(featureId)` is called; counter and last fingerprint cleared.
    - The agent gets a full N-iteration grace period before re-stall detection begins.

16. `output_loop_detection_enabled: false`:
    - `OutputLoopDetector` is not constructed. Wave executors skip the loop check.
    - Tier 1 and Tier 2B behavior is unaffected.

**Tier 2B — Plan Conformance Edge Cases**

17. Agent emits only NOTE and REQUEST outputs for the first several iterations (context-gathering):
    - No PATCH outputs → zero conforming patches → conformance counter increments.
    - With a threshold of 5, up to 4 such iterations are tolerated without penalty.
    - At iteration 5, if still no conforming patch, `plan_conformance_stall_action` fires.
    - This is intentional: the grace window accommodates legitimate setup, but an agent that
      never produces plan-aligned patches is not making progress toward the plan's definition of done.

18. Agent produces patches on files not in the plan (e.g. patching a README instead of source):
    - PATCH output is present, but the path does not appear in `files.create`, `files.modify`,
      or `allowed_areas` from the plan.
    - Conformance counter increments (non-conforming iteration).
    - At threshold, `plan_conformance_stall_action` fires.
    - This correctly catches the "working on the wrong thing" failure mode.

19. Agent produces a mix of conforming and non-conforming patches in one iteration:
    - If any single PATCH path in the output is plan-aligned, the iteration is conforming.
    - Conformance counter resets to 0.
    - The threshold is an OR condition, not an AND: one good patch redeems the iteration.

20. Agent produces a conforming patch every N-1 iterations (just below threshold):
    - Conformance counter resets on each conforming iteration. Stall never fires.
    - This is acceptable: any plan-aligned forward progress resets the clock. The threshold
      is a safeguard against complete disengagement, not a velocity requirement.

21. Plan has no `files.modify` and `files.create` is empty (plan only declares `allowed_areas`):
    - Conformance check falls back to `allowed_areas` prefix matching only.
    - A patch to any path with a matching prefix is plan-aligned.
    - If `allowed_areas` is also empty (malformed plan), conformance scoring is disabled for
      that feature with a warning logged; the check cannot make meaningful judgments.

22. `plan_conformance_check_enabled: false`:
    - `PlanConformanceScorer` is not constructed. Wave executors skip the conformance check.
    - Tier 1 and Tier 2A behavior is unaffected.

23. QA agent producing patches that fix tests but are outside the plan's declared test files:
    - If the patched test files are in `allowed_areas` but not in `files.modify`, the patch
      is still plan-aligned (conformance checks `allowed_areas` as a fallback).
    - Gate run then evaluates whether the fixes are correct; conformance scoring only asks
      "is the agent touching the right area?" not "is the output correct?"

---

## 6. Implementation Plan (Normative)

## M44.1 Contracts and Defaults

Files:

- `agentic/orchestrator/agents.yaml`
- `agentic/orchestrator/schemas/agents.schema.json`
- `agentic/orchestrator/defaults/policy.defaults.yaml`
- `agentic/orchestrator/schemas/policy.schema.json`
- `apps/control-plane/src/core/error-codes.ts`

Deliverables:

- new runtime watchdog fields (Tier 1: spawn/idle/hard timeouts, poll interval, kill grace)
- new runtime fields (Tier 2A: `output_loop_detection_enabled`, `output_loop_stall_iterations`;
  Tier 2B: `plan_conformance_check_enabled`, `plan_conformance_stall_iterations`)
- new execution policy action fields (Tier 1: spawn/stall/timeout; Tier 2A: `output_loop_stall_action`;
  Tier 2B: `plan_conformance_stall_action`)
- new error codes: `provider_spawn_timeout`, `provider_stall_timeout`, `provider_hard_timeout`,
  `provider_process_killed`, `provider_output_loop`, `provider_plan_conformance_stall`

## M44.2 Runner and Watchdog Core (Tier 1)

Files:

- `apps/control-plane/src/providers/providers.ts`
- `apps/control-plane/src/providers/worker-watchdog.ts` (new)

Deliverables:

- `WorkerCapabilities` interface: `supportsIdleDetection`, `supportsSpawnDetection` fields
- `WorkerLifecycleEvent` type with `type`, `ts`, `byteCount` fields; event types: `spawned`, `activity`, `heartbeat`, `terminated`
- `WorkerHandle` interface: `capabilities`, `onLifecycleEvent`, `kill()` — the sole boundary between supervisor and provider
- `NodeProviderCommandRunner` updated to emit `WorkerLifecycleEvent` objects via `WorkerHandle.onLifecycleEvent`
- `WorkerWatchdog` class: capability-gated three-deadline evaluation, SIGTERM→SIGKILL escalation via `WorkerHandle.kill()`, `WatchdogOutcome` output

## M44.3 Provider Adapter Integration (Tier 1)

Files:

- `apps/control-plane/src/providers/cli-worker-provider.ts`
- `apps/control-plane/src/providers/api-worker-provider.ts`
- `apps/control-plane/src/providers/worker-provider-factory.ts`

Deliverables:

- All provider adapters implement `WorkerHandle` and declare accurate `WorkerCapabilities` per §4.4
- `CliWorkerProvider` and `ApiWorkerProvider` construct `WorkerWatchdog` from runtime config and `WorkerHandle`
- Each adapter's `ensureRunSucceeded` (or equivalent) maps `WatchdogOutcome.reason` to specific error codes
- `WorkerProviderFactory` reads new runtime config fields and passes to provider constructors
- Capability declaration table (§4.4) populated for all supported providers: claude, codex, copilot, gemini, kiro-cli, API-backed

## M44.4A Output Loop Detector (Tier 2A)

Files:

- `apps/control-plane/src/supervisor/output-loop-detector.ts` (new)

Deliverables:

- `OutputLoopDetector` class with `check(featureId, result)`, `reset(featureId)`, `delete(featureId)`
- normalized output fingerprint computed in-memory from `WorkerDecisionResult`:
  PATCH paths + diff bodies (line-number-stripped), REQUEST action + params, NOTE text — all
  sorted and SHA-256 hashed
- per-feature rolling state: last fingerprint, consecutive identical count, last unique iteration
- `LoopDetectionResult` shape with `fingerprint`, `isLoop`, `consecutiveIdentical`, `lastUniqueIteration`
- synchronous, no I/O

## M44.4B Plan Conformance Scorer (Tier 2B)

Files:

- `apps/control-plane/src/supervisor/plan-conformance-scorer.ts` (new)

Deliverables:

- `PlanConformanceScorer` class with `check(featureId, result, plan)`, `reset(featureId)`, `delete(featureId)`
- conformance check: any PATCH path in `files.create | files.modify | allowed_areas` prefix match
- per-feature state: consecutive non-conforming count, last conforming iteration
- `ConformanceResult` shape with `conforming`, `isStalled`, `consecutiveNonConforming`,
  `planDeclaredFiles`, `actualPatchFiles`
- graceful degradation when plan has empty `files` and `allowed_areas`: log warning, skip check
- synchronous, no I/O

## M44.5 Supervisor and Policy Wiring (Both Tiers)

Files:

- `apps/control-plane/src/supervisor/worker-decision-loop.ts`
- `apps/control-plane/src/supervisor/planning-wave-executor.ts`
- `apps/control-plane/src/supervisor/build-wave-executor.ts`
- `apps/control-plane/src/supervisor/qa-wave-executor.ts`
- `apps/control-plane/src/supervisor/runtime.ts`

Deliverables:

- `WorkerDecisionLoop` journals Tier 1 error codes and watchdog metadata; does not collapse
  specific codes to `provider_runtime_unavailable`
- `BuildWaveExecutor` and `QaWaveExecutor` run `OutputLoopDetector` then `PlanConformanceScorer`
  after each iteration; apply respective policy actions; journal `worker_output_loop` and
  `worker_plan_conformance_stall` events as appropriate
- `PlanningWaveExecutor` runs `OutputLoopDetector` only; no conformance scoring
- `SupervisorRuntime` constructs `OutputLoopDetector` and `PlanConformanceScorer` when enabled;
  injects into wave executors; calls `reset(featureId)` on both when a feature is resumed

## M44.6 Verification and Regression Hardening

Files:

- `apps/control-plane/test/providers.spec.ts`
- `apps/control-plane/test/worker-provider-adapters.spec.ts`
- `apps/control-plane/test/worker-execution-policy.spec.ts`
- `apps/control-plane/test/output-loop-detector.spec.ts` (new)
- `apps/control-plane/test/plan-conformance-scorer.spec.ts` (new)
- `apps/control-plane/test/supervisor.unit.spec.ts`
- `apps/control-plane/test/supervisor.spec.ts`

Deliverables:

- Tier 1: unit tests for spawn timeout, idle stall, hard timeout, external kill paths
- Tier 1: policy action tests for planner/builder/qa on each error code
- Tier 1: regression for the previously observed SIGKILL/`provider_runtime_unavailable` path
- Tier 2A: unit tests for `OutputLoopDetector` covering identical/varied fingerprints, threshold,
  reset, planner vs builder role, and mixed output type scenarios
- Tier 2B: unit tests for `PlanConformanceScorer` covering conforming/non-conforming patches,
  threshold, reset, allowed_areas fallback, empty plan graceful degradation, and QA role
- Tier 2: integration tests for loop and conformance detection in full wave executor context
- Tier 2: policy action tests (`block_feature` vs `fail_run`) for both new error codes

---

## 7. Test Strategy (Must Be in `npm run verify` Path)

### 7.1 Unit — Runner and Watchdog (Tier 1)

- Spawn timeout: process mock never emits `spawned`; `provider_spawn_timeout` thrown within `worker_spawn_timeout_ms` + poll interval
- Idle timeout: process mock emits `spawned` then goes silent; `provider_stall_timeout` thrown within `worker_idle_timeout_ms` + poll interval; idle_for_ms in error details ≥ idle threshold
- Hard timeout: process mock emits continuous stdout chunks but never exits; `provider_hard_timeout` thrown at `worker_response_timeout_ms`; `timed_out=true` in error details
- External kill: process mock exits with non-zero code and signal, no timeout condition; `provider_process_killed` with `timed_out=false`
- SIGTERM→SIGKILL escalation: mock `killFn` records call sequence; verifies SIGTERM called first, then SIGKILL after grace period if process still alive
- Activity refresh: stdout chunk received just before idle deadline; deadline is reset; no timeout fires

### 7.2 Adapter Tests (Tier 1)

- Adapter with `supportsIdleDetection: false` never triggers idle timeout, even after long silence
- Adapter with `supportsIdleDetection: true` triggers `provider_stall_timeout` after idle window elapses with no `activity` events
- Adapter with `supportsSpawnDetection: false` never triggers spawn timeout
- Timeout metadata (`watchdog_reason`, `idle_for_ms`, `elapsed_ms`, etc.) is present in error details
- Parser errors (`provider_output_invalid`) are not reclassified as timeout errors when process exits 0
- `WorkerProviderFactory` reads new runtime config fields and passes them to provider constructors
- Capability declarations in existing adapters (claude, codex) match observed subprocess stdout behavior

### 7.3 Unit — Output Loop Detector (Tier 2A)

- First check: fingerprint recorded, `consecutiveIdentical = 0`, `isLoop = false`
- Second check, identical output: `consecutiveIdentical = 1`, `isLoop = false` (below threshold)
- Nth check at threshold: `isLoop = true`, `consecutiveIdentical === threshold`
- Output varies: `consecutiveIdentical` resets to 0, `isLoop = false`
- `reset(featureId)`: counter and fingerprint cleared; subsequent check starts fresh
- NOTE-only output: normalized fingerprint keyed on text content; identical NOTEs detect loop, varying NOTEs do not
- REQUEST-only output: fingerprint keyed on action + params; identical REQUESTs detect loop
- PATCH with failed apply same content each time: fingerprint stable → loop detected at threshold
- PATCH with line-number variance but identical content: normalization strips offsets → same fingerprint → loop detected

### 7.4 Unit — Plan Conformance Scorer (Tier 2B)

- Iteration with PATCH on `files.create` path: conforming → counter resets
- Iteration with PATCH on `files.modify` path: conforming → counter resets
- Iteration with PATCH on path matching `allowed_areas` prefix: conforming → counter resets
- Iteration with PATCH on unrelated path only: non-conforming → counter increments
- Iteration with NOTE only (no PATCH): non-conforming → counter increments
- Mixed iteration (one conforming PATCH + one non-conforming PATCH): conforming → counter resets
- Nth non-conforming iteration at threshold: `isStalled = true`
- `reset(featureId)`: counter cleared; subsequent check starts fresh
- Empty plan files + empty allowed_areas: scorer logs warning, skips check, returns `conforming = true`
- Planner role: scorer not called (verified at wave executor level)

### 7.5 Supervisor Policy Tests (Both Tiers)

**Tier 1:**

- Planner Tier 1 timeout with `block_feature` → feature status blocked, worker event includes `provider_stall_timeout`, no plan accepted
- Builder Tier 1 timeout with `fail_run` → run throws with deterministic code, no gate run
- QA Tier 1 stall with `block_feature` → no gate run, no QA session rotation for that cycle

**Tier 2A:**

- Builder N consecutive identical outputs → `worker_output_loop` event journaled, `block_feature` applied, gate skipped
- Builder loop stall with `fail_run` policy → run throws `provider_output_loop`, fingerprint + count in error details
- Planner N identical outputs → `worker_output_loop` fires (planner exempt from 2B only, not 2A)
- Feature unblocked after loop stall → detectors reset, next N iterations tolerated before re-detection

**Tier 2B:**

- Builder N consecutive non-conforming iterations → `worker_plan_conformance_stall` event journaled, `block_feature` applied, gate skipped
- Builder conformance stall with `fail_run` → run throws `provider_plan_conformance_stall`, file lists in error details
- QA N non-conforming iterations → same behavior as builder
- Planner N iterations → conformance check not called
- Feature unblocked after conformance stall → scorer reset, next N iterations tolerated

### 7.6 Integration Tests

- Simulated hung provider process triggers deterministic Tier 1 watchdog path in full run
- Simulated provider that always returns identical NOTE → Tier 2A loop stall after N iterations
- Simulated provider that always patches off-plan files → Tier 2B conformance stall after N iterations
- Multi-feature run: one feature hits loop stall (`block_feature`); other features continue unaffected
- Provider resumes normally after block: feature unblocked, detectors reset, next iteration counted from zero
- Loop detector and conformance scorer fire independently when conditions differ across iterations

---

## 8. Acceptance Criteria

**Tier 1 — Process-Level Watchdog:**

1. Live worker runtime failures are classified into explicit categories: `provider_spawn_timeout`,
   `provider_stall_timeout`, `provider_hard_timeout`, `provider_process_killed`.
2. `provider_runtime_unavailable` is no longer the terminal signal for timeout-kill scenarios;
   it is reserved for missing runtime configuration and ENOENT spawn failures.
3. Planner/builder/qa apply deterministic, policy-controlled actions on Tier 1 failures via
   `provider_spawn_failure_action`, `provider_stall_action`, and `provider_timeout_action`.
4. Worker event artifacts include watchdog metadata (`watchdog_reason`, `idle_for_ms`, `elapsed_ms`,
   `spawn_timeout_ms`, `idle_timeout_ms`) sufficient for RCA without reproducing the run.

**Tier 2A — Output Loop Detection:**

5. Planner, builder, and QA waves detect deterministic output loops (N consecutive iterations with
   identical normalized output fingerprint) and apply `output_loop_stall_action` deterministically.
6. `provider_output_loop` is journaled with `output_fingerprint`, `consecutive_identical_iterations`,
   and `loop_threshold` fields.
7. Output fingerprint normalization strips PATCH line-number offsets so cosmetically different but
   semantically identical patches hash identically.
8. Feature unblock resets loop detector state so the agent gets a clean N-iteration window.

**Tier 2B — Plan Conformance Scoring:**

9. Builder and QA waves detect plan-conformance stall (N consecutive iterations with no patch
   touching `files.create`, `files.modify`, or `allowed_areas`) and apply
   `plan_conformance_stall_action` deterministically.
10. `provider_plan_conformance_stall` is journaled with `plan_declared_files`, `actual_patch_files`,
    `consecutive_nonconforming_iterations`, and `conformance_threshold` fields.
11. Planner wave is exempt from plan conformance scoring.
12. Feature unblock resets conformance scorer state so the agent gets a clean N-iteration window.
13. Plans with empty `files` and `allowed_areas` gracefully skip the check (no false stalls).
14. `output_loop_detection_enabled: false` and `plan_conformance_check_enabled: false` each
    fully disable their respective checks independently with no behavioral change to other logic.

**Verification:**

15. `npm run verify` contains regression coverage for all Tier 1 failure modes (SIGKILL, idle stall,
    spawn timeout) and all Tier 2 stall scenarios (loop detection, conformance scoring, policy actions,
    reset behavior, role exemptions).

---

## 9. Risks and Mitigations

**Tier 1:**

- Risk: watchdog idle timeout fires on slow-but-healthy providers that buffer all output.
  - Mitigation: Provider adapters that buffer output until completion MUST declare
    `supportsIdleDetection: false`. The watchdog gates idle timeout on this capability flag —
    no idle timeout is applied when the adapter honestly declares it cannot provide real-time
    signals. Operators adding new adapters MUST verify and document the capability declaration
    in §4.4 before shipping. Streaming adapters that do emit `activity` events will reset the
    idle timer as expected.

- Risk: adapter capability declarations are wrong (e.g. declared `supportsIdleDetection: true`
  for a provider that actually buffers).
  - Mitigation: incorrect declarations produce false-positive `provider_stall_timeout` kills.
    Adapter tests (§7.2) MUST verify capability gating: an adapter with `supportsIdleDetection:
false` must never trigger idle timeout regardless of silence duration. CI catches regressions.

- Risk: lifecycle callbacks in the command runner increase coupling.
  - Mitigation: callbacks are optional (`onLifecycleEvent?`); existing call sites that do not
    pass callbacks are unaffected. Watchdog is isolated in a dedicated module.

- Risk: SIGTERM→SIGKILL grace period leaves zombie processes.
  - Mitigation: grace period is configurable (`worker_kill_grace_period_ms`); the runner's
    `close` event handler resolves the Promise regardless of which signal terminates the process.

- Risk: role behavior divergence due to partial Tier 1 policy wiring.
  - Mitigation: shared `applyWorkerPolicyAction` pattern in all three wave executors; role-specific
    tests verify each executor independently.

**Tier 2A — Output Loop Detection:**

- Risk: fingerprint normalization is incomplete, allowing cosmetically different but logically
  identical outputs to evade detection.
  - Mitigation: normalization is specified precisely (PATCH line-number stripping, sorted output
    list, canonical JSON serialization). Test suite covers the cosmetic variance cases explicitly.

- Risk: false positives where the agent legitimately produces similar outputs (e.g. short NOTEs
  about different sub-tasks that happen to hash identically after normalization).
  - Mitigation: `output_loop_stall_iterations` defaults to 3. Short identical outputs across 3
    consecutive iterations is a strong signal of a loop. Operators can raise the threshold for
    workflows with high legitimate output repetition.

- Risk: loop detector state leaks across features in long-running multi-feature runs.
  - Mitigation: `delete(featureId)` is called when a feature leaves the active set. State is
    scoped strictly per feature_id.

**Tier 2B — Plan Conformance Scoring:**

- Risk: false positives when the agent legitimately spends several iterations gathering context
  before producing plan-aligned patches.
  - Mitigation: `plan_conformance_stall_iterations` defaults to 5, providing a larger grace window
    than loop detection. Operators in context-heavy workflows can raise this further.

- Risk: plan file lists are incomplete or use relative paths that don't match patch paths.
  - Mitigation: conformance check uses both exact match (for `files.create`/`files.modify`) and
    prefix match (for `allowed_areas`). If the plan's allowed_areas covers the relevant directory,
    patches in subdirectories will still be plan-aligned. Operators with poorly-scoped plans should
    expand `allowed_areas` rather than disable the check.

- Risk: plan changes mid-build (plan amendment via `plan.update`) invalidate the cached plan.
  - Mitigation: the supervisor reloads the plan after any `plan.update` tool call and passes the
    refreshed plan object to `PlanConformanceScorer.check`. The cached plan must not be stale.

- Risk: event logging overhead.
  - Mitigation: Tier 2 events are written only when a stall fires, not on every iteration.
    Both detectors are synchronous in-memory operations; the overhead per iteration is a SHA-256
    hash and two map lookups.

---

## 10. Definition of Done

A completed implementation session MUST provide:

**Contracts:**

- Schema and default deltas for all Tier 1 watchdog fields and Tier 2 auditor fields
- Policy schema and defaults for `provider_timeout_action`, `provider_stall_action`,
  `provider_spawn_failure_action`, and `worktree_stall_action`
- All five new error codes registered in `error-codes.ts`

**Tier 1 — Process-Level Watchdog:**

- `WorkerHandle` interface and `WorkerCapabilities` type defined in `providers/providers.ts`
- All existing provider adapters (CLI and API) implement `WorkerHandle` with accurate capability declarations per §4.4
- `WorkerWatchdog` implements three-deadline evaluation (capability-gated), SIGTERM→SIGKILL escalation via `WorkerHandle.kill()`
- `WorkerWatchdog` disables spawn timeout when `supportsSpawnDetection: false`; disables idle timeout when `supportsIdleDetection: false`
- All provider adapters map `WatchdogOutcome.reason` to specific error codes
- `WorkerProviderFactory` reads and propagates new runtime config fields

**Tier 2A — Output Loop Detection:**

- `OutputLoopDetector` implemented with synchronous `check`, `reset`, `delete` methods
- Normalized output fingerprinting covers PATCH (line-number-stripped), REQUEST, and NOTE outputs
- `PlanningWaveExecutor`, `BuildWaveExecutor`, and `QaWaveExecutor` run loop check after each iteration
- `SupervisorRuntime` constructs and injects detector; calls `reset` on feature resume

**Tier 2B — Plan Conformance Scoring:**

- `PlanConformanceScorer` implemented with synchronous `check`, `reset`, `delete` methods
- Conformance check covers `files.create`, `files.modify`, and `allowed_areas` prefix matching
- `BuildWaveExecutor` and `QaWaveExecutor` run conformance check after loop check; `PlanningWaveExecutor` does not
- Graceful degradation for plans with empty file lists and allowed_areas
- `SupervisorRuntime` reloads plan after `plan.update` and passes refreshed plan to scorer

**Verification:**

- Worker event journaling for `worker_timeout` (Tier 1), `worker_output_loop` (Tier 2A), and
  `worker_plan_conformance_stall` (Tier 2B)
- Green `npm run verify` including all new Tier 1, Tier 2A, and Tier 2B regression tests
- `spec-files/progress.md` entry documenting delivery, test counts, and evidence
