# Watches

Persistent monitoring jobs that fire inbox notifications and follow-up actions when a target hits a condition. Watches survive engine restarts and are checked every 3 ticks (~3 minutes by default).

## What a Watch Is

A watch is a small JSON record persisted to `engine/watches.json`. It binds:

| Field        | Purpose                                                                 |
|--------------|-------------------------------------------------------------------------|
| `target`     | The thing to watch (PR number, work item id, plan filename, ...)         |
| `targetType` | Which registered target type (`pr`, `work-item`, `meeting`, ...)         |
| `condition`  | One of the conditions the target type accepts (e.g. `merged`, `failed`)  |
| `interval`   | Min ms between checks (default 5min, floor 60s)                          |
| `notify`     | `'inbox'` writes an inbox alert on trigger (set `'none'` to suppress)    |
| `owner`      | Inbox owner that receives notifications (`'human'` by default)           |
| `stopAfter`  | `0` = run forever for change conditions / fire-once for absolute; `N` = expire after N triggers |
| `onNotMet`   | `null` or `'notify'` — write a per-poll inbox note when the condition is not yet met |
| `action`     | Optional follow-up action (see "Follow-Up Actions" below)                |
| `requires`   | Optional guard: array of predicate objects evaluated against `state` / `entity` / `prevState`; trigger is suppressed when any guard fails (false-or-error). Used to gate a watch on "PR is mergeable AND build passing" etc. |
| `status`     | `WATCH_STATUS.ACTIVE` \| `PAUSED` \| `TRIGGERED` \| `EXPIRED`            |

`createWatch()` allocates a `watch-<uid>` id, defaults the fields above, and persists atomically via `mutateJsonFileLocked` *(source: `engine/watches.js:184-248`)*.

## Lifecycle (`WATCH_STATUS`)

Defined in `engine/shared.js` (`WATCH_STATUS`):

| Status      | Meaning                                                                 |
|-------------|-------------------------------------------------------------------------|
| `active`    | Eligible for evaluation each tick                                       |
| `paused`    | Skipped by `checkWatches`; persists indefinitely until resumed/deleted  |
| `triggered` | Reserved status (set on demand by callers; not auto-applied)            |
| `expired`   | Auto-set when `stopAfter` is reached, or on first trigger for absolute conditions when `stopAfter === 0`. The watch is left on disk for audit but no longer evaluated *(source: `engine/watches.js:507-508`)* |

Pause/resume flips the `status` field via `POST /api/watches/update` *(source: `engine/watches.js` `updateWatch`, `dashboard.js` `handleWatchesUpdate`)*.

## Conditions (`WATCH_CONDITION`)

Defined in `engine/shared.js` (`WATCH_CONDITION`). Conditions split into two families:

### Absolute conditions (`WATCH_ABSOLUTE_CONDITIONS`)
*(source: `engine/shared.js` `WATCH_ABSOLUTE_CONDITIONS`)*

`merged`, `build-fail`, `build-pass`, `completed`, `failed`, `concluded`, `approved`, `rejected`, `ready-for-merge`, `retry-limit-reached`, `all-items-done`, `item-failed-n-times`.

When `stopAfter === 0`, these are **fire-once** — the engine flips the watch to `expired` after the first trigger so a permanently-merged PR (or a permanently-true compound state assertion like `ready-for-merge`) doesn't keep notifying *(source: `engine/watches.js:507-508`)*.

> **Per-target override (W-mp7hg58e000b5212):** the global `WATCH_ABSOLUTE_CONDITIONS` set is the legacy fallback. Each target type now declares its own `absoluteConditions: [...]` array in its spec; `registerTargetType` normalizes that into a `Set` that takes precedence at evaluation time. The plugin contract (see below) uses this to keep absolute-vs-change semantics local to each target type. Plugins that omit `absoluteConditions` get an empty set (all change-based).

### Change-based conditions
`status-change`, `any`, `new-comments`, `vote-change`, `stage-complete`, `ran`, `enabled`, `disabled`, `activity-change`, plus the predicate conditions added under P-w4e2f6a1 / P-w5b8d2c9 for the `pr`, `work-item`, `plan`, and `pipeline` target types (`head-commit-change`, `mergeable-flipped`, `behind-master`, `draft-flipped`, `stalled`, `dependency-met`, `stage-advanced`, `stuck-in-stage`). See `engine/shared.js` `WATCH_CONDITION` for the canonical enum.

These compare the live entity against the watch's `_lastState` snapshot and run forever when `stopAfter === 0`. Baseline `_lastState` is captured on the first check so the very next change triggers the watch *(source: `engine/watches.js:434, 520`)*.

### Tick-counted conditions
`stalled`, `stuck-in-stage` — require N consecutive unchanged captures (default `WATCH_STALLED_DEFAULT_TICKS = 12`, `WATCH_STUCK_STAGE_DEFAULT_TICKS = 12`, both in `engine/shared.js`). Counters (`_unchangedTicks`, `_stuckStageTicks`) are recomputed inside `_captureState` by comparing the fresh snapshot against `prevState`.

### Predicate conditions

Several condition keys evaluate a derived predicate on the captured entity/state rather than comparing to `_lastState`. Built-in predicates per target type:

- **PR** — `head-commit-change` (headRefOid advanced), `mergeable-flipped` (mergeable true↔false), `ready-for-merge` (active+approved+passing+mergeable+!draft compound), `behind-master` (`pr.behindBy > 0`), `draft-flipped` (isDraft true↔false).
- **work-item** — `stalled` (no captured-state change for `WATCH_STALLED_DEFAULT_TICKS` checks, default 12 = ~60min), `retry-limit-reached` (`_retryCount >= ENGINE_DEFAULTS.maxRetries`), `dependency-met` (`_pendingReason` transitioned away from `dependency_unmet`).
- **plan** — `all-items-done` (`items_done === items_total > 0`), `item-failed-n-times` (any `missing_features[*]._retryCount >= ENGINE_DEFAULTS.maxRetries`).
- **pipeline** — `stage-advanced` (`current_stage_id` changed within the same `runId`), `stuck-in-stage` (current stage unchanged for `WATCH_STUCK_STAGE_DEFAULT_TICKS` checks, default 12).

Compound state-assertion predicates (`ready-for-merge`, `retry-limit-reached`, `all-items-done`, `item-failed-n-times`) live in `WATCH_ABSOLUTE_CONDITIONS` so they fire-once when `stopAfter === 0` — without that they would re-fire every tick while the assertion holds *(source: `engine/shared.js` `WATCH_ABSOLUTE_CONDITIONS`)*.

## Target Types — `TARGET_TYPES` Registry

Target-type behavior in `engine/watches.js` is **data-driven via a registry** *(source: `engine/watches.js:124-153`)*. Each spec must provide:

- `label` — human name shown in dashboard pickers
- `description` — short help text
- `conditions` — non-empty array of accepted condition keys (also acts as the per-type allowlist)
- `absoluteConditions` — *(optional)* subset of `conditions` that fire-once when `stopAfter === 0`; defaults to `[]` (all conditions treated as change-based)
- `fetchEntity(target, state)` — entity-or-null lookup
- `captureState(entity)` — snapshot used for change-detection diffs
- `evaluate(condition, entity, prevState, target)` — returns `{ triggered, message }`

The registry IS the allowlist for `createWatch` and `/api/watches/target-types`; the old hardcoded "pr or work-item" check is gone. Add a new target type at runtime with `registerTargetType(type, spec)` and look one up with `getTargetType(type)`. `listTargetTypes()` returns the serializable form used by the dashboard *(source: `engine/watches.js:124-174`)*.

### User-extensible via `watches.d/` (W-mp7hg58e000b5212)

At engine boot, every `*.js` file in `<MINIONS_DIR>/watches.d/` is auto-loaded **after** the built-in registrations *(source: `engine/watches.js:1319-1354`)*, so plugins can both add new target types and override built-ins. A plugin file exports either `{ name, spec }` or an array of such objects. Failures are logged-and-skipped — one bad plugin must not break boot or block other plugins. Reloads require an engine restart.

Canonical example: `watches.d/http.js` (W-mp7i22mu00191b07) — a generic HTTP poller covering the full plugin contract including `extractState` (custom snapshot fields not on the entity itself) and `extendTemplateVars` (custom action-template vars like `{{httpStatus}}`, `{{prevExtracted}}`).

### Built-in target types

The eight built-ins are registered at module load *(source: `engine/watches.js` — the long `registerTargetType(...)` block)*. Constants live in `WATCH_TARGET_TYPE` in `engine/shared.js`.

| `targetType`  | Target value                         | Conditions                                                                 | Notes |
|---------------|--------------------------------------|----------------------------------------------------------------------------|-------|
| `pr`          | PR number, canonical id, or display id | `merged`, `build-fail`, `build-pass`, `status-change`, `any`, `new-comments`, `vote-change`, `head-commit-change`, `mergeable-flipped`, `ready-for-merge`, `behind-master`, `draft-flipped` | Reads from `pull-requests.json` for any project; `new-comments` watches `humanFeedback.lastProcessedCommentDate`; `behind-master` requires `engine.watchesIncludeBehindBy: true` so the GH poller populates `pr.behindBy` |
| `work-item`   | Work item id                         | `completed`, `failed`, `status-change`, `any`, `stalled`, `retry-limit-reached`, `dependency-met` | `completed` matches `DONE_STATUSES`; `failed` matches `WI_STATUS.FAILED`; `stalled` fires after N unchanged captures (default 12 ≈ 60 min); `retry-limit-reached` fires when `_retryCount >= ENGINE_DEFAULTS.maxRetries` |
| `meeting`     | Meeting id                           | `concluded`, `status-change`, `any`                                        | `concluded` fires on terminal status (`completed`, `archived`) |
| `plan`        | PRD JSON filename or plan id         | `approved`, `rejected`, `completed`, `status-change`, `any`, `all-items-done`, `item-failed-n-times` | Looked up by `_source` (PRD file), `_sourcePlan` (.md), or `id`; uses `PLAN_STATUS` |
| `schedule`    | Schedule id                          | `ran`, `enabled`, `disabled`, `status-change`, `any`                       | `ran` fires when `lastRun` advances; `enabled`/`disabled` fire on the flip |
| `pipeline`    | Pipeline id (latest run is tracked)  | `completed`, `failed`, `stage-complete`, `status-change`, `any`, `stage-advanced`, `stuck-in-stage` | `failed` covers `PIPELINE_STATUS.FAILED` and `STOPPED`; `stage-complete`/`stage-advanced` only count within the same `runId`; `stuck-in-stage` fires after N unchanged captures (default 12) |
| `dispatch`    | Dispatch entry id                    | `completed`, `failed`, `status-change`, `any`                              | Looks across `pending` / `active` / `completed` lists |
| `agent`       | Agent id                             | `activity-change`, `status-change`, `any`                                  | `activity-change` fires only on transitions in/out of `'working'` |

`evaluateWatch` dispatches to `tt.evaluate(...)`; unknown target types return `"Unknown target type: ..."` and unknown conditions return `"Unknown condition: ..."` — both are non-triggering *(source: `engine/watches.js:318-371`)*.

### Plugin folder (`watches.d/`) — user-extensible target types

W-mp7hg58e000b5212 added a **plugin folder** so operators can register new target types without editing engine source. At engine boot, `engine/watches.js` scans `<MINIONS_DIR>/watches.d/*.js` *after* the eight built-ins are registered (so plugins can override a built-in by re-using its key — last-write-wins) and calls `registerTargetType()` for each export *(source: `engine/watches.js:1319-1354`)*.

Each `watches.d/<name>.js` file must export `{ name, spec }` (or an array of those):

```js
module.exports = {
  name: 'my-target',
  spec: {
    label: 'My Target',
    description: 'What it watches',
    conditions: ['status-change', 'value-equals'],
    absoluteConditions: ['value-equals'],          // optional, defaults to []
    fetchEntity(target, state) { /* sync entity lookup */ },
    captureState(entity) { /* snapshot for diffs */ },
    evaluate(condition, entity, prevState, target) {
      return { triggered: <bool>, message: '...' };
    },
  },
};
```

Resolution is `path.join(shared.MINIONS_DIR, 'watches.d')` so it works in both dev checkouts and installed `~/.minions/` layouts. Missing folder is silent; per-file failures are isolated (load errors log a `WARN` line, don't abort boot). Tests can re-scan via the exported `_loadPluginTargetTypes()`.

**Canonical plugin example:** [`watches.d/http.js`](../watches.d/http.js) (W-mp7i22mu00191b07) ships a generic `http` target type — point a watch at any URL or `{ url, method, headers, body, extract, expected, regex, timeoutMs }` config and the plugin exposes `status-change`, `value-change`, `value-equals`, `value-matches`, and `http-error` conditions. Auth headers support `{{env.GH_TOKEN}}`-style substitution at fetch time (secrets are never stored in `watches.json`). All conditions are change-based (`absoluteConditions: []`).

## Tick Integration

`engine.js` calls `checkWatches(config, state)` every `ENGINE_DEFAULTS.watchPollEvery` ticks (default 18 ⇒ ~3 min at the default 10s tick) inside its own `safe('checkWatches', ...)` block *(source: `engine.js:7439-7470`)*. The engine builds the state object from cached project files + module reads:

```
{
  pullRequests, workItems,    // flattened across all projects + central
  meetings, plans,            // optional — try/catch'd, missing modules give []
  scheduleRuns, pipelineRuns,
  dispatch, agents, config,
}
```

`checkWatches` walks every active watch and, inside a single `mutateJsonFileLocked` callback *(source: `engine/watches.js:410-561`)*:

1. Skips paused/expired watches and any watch checked within its `interval`.
2. Captures a baseline `_lastState` on first check (so change conditions have something to diff).
3. Calls `evaluateWatch(watch, state)`.
4. On trigger: increments `triggerCount`, sets `last_triggered`, queues an inbox notification (if `notify === 'inbox'`), and snapshots an action task for any configured `watch.action`.
5. Applies fire-once / `stopAfter` expiration.
6. On non-trigger: writes a per-poll inbox note when `onNotMet === 'notify'`.
7. Refreshes `_lastState` for the next check.

I/O happens **outside the lock**: notifications via `writeToInbox`, follow-up actions via `_runActionTask` (`Promise` per action, failures isolated). Each action's result is persisted back onto the watch as `_lastActionResult` in a follow-up locked write *(source: `engine/watches.js:562+` `_runActionTask`)*.

## Follow-Up Actions on Trigger

`watch.action` is an optional structured action that runs after the inbox notification fires. Action types live in a sibling registry in `engine/watch-actions.js` and are validated at create/update time *(source: `engine/watches.js:184-248` `createWatch`, `engine/watch-actions.js:56` `registerActionType`)*. `GET /api/watches/action-types` returns the live list for dashboard pickers.

### Built-in actions

| Action type            | What it does                                                                                  |
|------------------------|-----------------------------------------------------------------------------------------------|
| `notify`               | Explicit inbox write; lets you customize `owner`/`body` instead of the default trigger string |
| `dispatch-work-item`   | Append a new WI to the project (or central) `work-items.json` with `createdBy: "watch:<id>"`  |
| `run-skill`            | Wrapper around `dispatch-work-item` that asks the agent to run a specific `.claude` skill     |
| `webhook`              | `http`/`https` request to an arbitrary URL (10s safety timeout, JSON or string body)          |
| `minions-api`          | Loopback call to the in-process dashboard at `http://127.0.0.1:${MINIONS_PORT \|\| 7331}` — `endpoint` must start with `/api/`; sets `X-Minions-Internal: 1`; returns `{ok, status, summary, response}` for chain-step templating |
| `cancel-work-item`     | Flip a WI to `WI_STATUS.CANCELLED` across all known work-items files                          |
| `trigger-pipeline`     | Start a new pipeline run (skipped if the pipeline already has an active run)                  |
| `archive-plan`         | Set PRD `status="archived"` + `archivedAt`                                                    |
| `resume-plan`          | Set PRD `status=PLAN_STATUS.ACTIVE` and clear `planStale`                                     |
| `cc-triage`            | Invoke Command Center headlessly via the loopback `POST /api/command-center/triage` endpoint with the trigger context (and optional completion-report / live-output artifacts). Wraps the prompt in `<UNTRUSTED-INPUT>`, uses a default 10-min timeout (capped at 1 h), and is isolated from the user CC session |

Constants live in `WATCH_ACTION_TYPE` (`engine/shared.js`); handlers in `engine/watch-actions.js`.

### Templating

Action params support `{{var}}` substitution from the trigger context *(source: `engine/watch-actions.js:145-220`)*. Built-in vars from `buildTriggerContext`:

- Always: `target`, `targetType`, `condition`, `watchId`, `triggerCount`, `message`
- All scalar fields from `newState` (e.g. `status`, `buildStatus`, `reviewStatus`, `lastRun`)
- Type-specific aliases: `prNumber`, `branch`, `prId`, `prUrl`, `project` (pr); `workItemId`, `workItemTitle`, `project` (work-item); `meetingId`, `meetingTitle` (meeting); `planFile`, `planMd` (plan); `pipelineId`, `runId` (pipeline); `dispatchId` (dispatch); `agentId` (agent)

Unknown vars are left as `{{var}}` so callers can detect them downstream.

### Failure isolation

Action handlers must return `{ ok: false, summary }` rather than throw — `runWatchAction` wraps them in try/catch as a backstop *(source: `engine/watch-actions.js:223-270`)*. A bad action never blocks other watches in the same tick, and the failed result is persisted onto the watch as `_lastActionResult` for debugging. Multi-step action chains run via `runWatchActionChain` *(source: `engine/watch-actions.js:271+`)* — each step's `{ok, summary, result}` is templated into the next step's params as `{{steps.N.result.<field>}}`.

## Dashboard Surface

| Route                          | Method | Handler                       | Purpose                                                                 |
|--------------------------------|--------|-------------------------------|-------------------------------------------------------------------------|
| `/watches`                     | GET    | `dashboard/pages/watches.html` | Watches page (sidebar link in `dashboard/layout.html:75`)               |
| `/api/watches`                 | GET    | `handleWatchesList`           | List all watches                                                        |
| `/api/watches/target-types`    | GET    | `handleWatchesTargetTypes`    | Live registry from `listTargetTypes()` — used by the create-watch modal |
| `/api/watches/action-types`    | GET    | `handleWatchesActionTypes`    | Live registry from `listActionTypes()`                                  |
| `/api/watches`                 | POST   | `handleWatchesCreate`         | Create a watch (body: `target, targetType, condition, interval?, owner?, description?, project?, notify?, stopAfter?, onNotMet?, action?`) |
| `/api/watches/update`          | POST   | `handleWatchesUpdate`         | Pause/resume/modify (body: `id, status?, interval?, description?, notify?, stopAfter?, onNotMet?, condition?, action?`) |
| `/api/watches/delete`          | POST   | `handleWatchesDelete`         | Delete (body: `id`)                                                     |

*(source: route table in `dashboard.js`; handlers `handleWatchesList` / `handleWatchesTargetTypes` / `handleWatchesActionTypes` / `handleWatchesCreate` / `handleWatchesUpdate` / `handleWatchesDelete`)*

The watches page is rendered by `dashboard/js/render-watches.js`; the **+ New** button calls `openCreateWatchModal()` which fetches `/api/watches/target-types` and `/api/watches/action-types` to populate the picker dynamically *(source: `dashboard/pages/watches.html`, `dashboard/js/render-watches.js`)*.

The CC state preamble injects a `Watches: <count>` line via `getWatches()` so the chat brain knows how many watches exist *(source: `dashboard.js` `buildCCStatePreamble`)*.

## Command Center Integration

CC creates, pauses, resumes, and deletes watches by calling the REST API directly via its `Bash` tool — the older `===ACTIONS===` `create-watch`/`delete-watch`/`pause-watch`/`resume-watch` actions have been retired *(source: `test/unit.test.js:16976-17008`, `test/unit/watches-module.test.js:673`)*. To use a watch from CC chat, ask it something like:

> "Watch PR 1234 for build-pass and dispatch a verify work-item when it fires."

CC will discover the endpoints from `GET /api/routes` and `POST /api/watches` with the appropriate `targetType`, `condition`, and `action` payload.

## Expected Outputs

When a watch fires:

- `engine/watches.json` entry gets `triggerCount++`, `last_triggered=<ISO>`, `_lastTriggerMessage=<message>`, and a refreshed `_lastState`.
- An inbox file lands at `notes/inbox/<owner>/watch-<watchId>-<n>.md` (when `notify === 'inbox'`).
- If `action` is set, a follow-up runs (work item, webhook POST, plan archive, ...) and writes `_lastActionResult` (`{ type, ok, summary, dispatchedItemId?, at }`) back onto the watch.
- `engine.log.json` gets `Watch triggered: <id> — <message>` and `Watch <id> action <type>: <summary>`.

Absolute conditions firing under `stopAfter === 0` flip `status` to `expired`; `stopAfter > 0` flips it to `expired` once `triggerCount >= stopAfter`.

## Common Failure Modes

| Symptom                                                                 | Likely cause / debug                                                                 |
|-------------------------------------------------------------------------|----------------------------------------------------------------------------------------|
| Watch never fires                                                       | Check `status === 'active'`; check `last_checked` advancing each cycle; confirm engine tick is running and `interval` isn't longer than your test window |
| `evaluateWatch` returns `"<label> <target> not found"`                   | `fetchEntity` got nothing back — wrong `target` (e.g. PR display id vs canonical id), the target type isn't loaded, or the underlying file (PR cache, plan PRD) doesn't exist |
| `"Unknown target type"` / `"Unknown condition"`                         | The registry doesn't recognise the value. Check `GET /api/watches/target-types` to see what's registered server-side; condition must be in that target type's `conditions[]` |
| Change condition fires immediately on first tick                        | Won't happen — baseline `_lastState` is captured on the first check before `evaluate` runs *(source: `engine/watches.js:434`)*. If you see this, suspect manual edits to `watches.json` |
| Absolute watch fires forever instead of once                            | `stopAfter` is set to a non-zero value; only `stopAfter === 0` triggers fire-once expiration |
| Action runs but inbox notification doesn't                              | `notify` field isn't `'inbox'`, or `owner` is empty. `notify` and `action` are independent — both can fire, or only one |
| `_lastActionResult.ok === false` with `"unknown action type"`           | The `action.type` isn't registered. List with `listActionTypes()` / `GET /api/watches/action-types` |
| Webhook action returns `"only http/https allowed"`                      | URLs must use `http://` or `https://` schemes; other protocols are rejected by design *(source: `engine/watch-actions.js` `WEBHOOK` handler)* |
| Trigger fires but follow-up `dispatch-work-item` is missing              | Check the engine log for `Watch <id> action <type>: <summary>`. Common reasons: missing `title`, the project's `work-items.json` couldn't be written, or the WI landed in central `work-items.json` because no project was specified |
| Watch `_lastActionResult` shows `"timeout"` for webhook                 | Webhooks have a 10s safety timeout to keep the watches tick fast *(source: `engine/watch-actions.js:482-484`)* |
| `checkWatches` block crashes silently                                   | Wrapped in `safe('checkWatches', ...)` so one failure doesn't abort the tick *(source: `engine.js:7443`)*. Inspect `engine/log.json` for `Watch check error (<id>)` lines. Regression #1088: the block must use `getProjects(config)`, never the long-removed `PROJECTS` constant |

## See Also

- `engine/shared.js` — `WATCH_STATUS`, `WATCH_TARGET_TYPE`, `WATCH_CONDITION`, `WATCH_ABSOLUTE_CONDITIONS`, `WATCH_ACTION_TYPE` constants
- `engine/watches.js` — registry, lifecycle, tick integration, `watches.d/` plugin loader
- `engine/watch-actions.js` — action registry and built-in handlers (including `minions-api`)
- `watches.d/http.js` — canonical user-extensible target type plugin
- `dashboard/pages/watches.html`, `dashboard/js/render-watches.js` — dashboard UI
- `test/unit/watches-module.test.js`, `test/unit/watch-actions.test.js`, `test/unit/watches-plugin-loader.test.js` — module-level tests
- [`auto-discovery.md`](auto-discovery.md) — overall tick cycle context
