# QA Runbooks

> Plan item **W-mpeiwz6k0005bf34-a** — schema + persistence + CRUD endpoints.
> Run dispatch, run records, and UI live in follow-up items.

## Storage location

Runbooks are per-project test plans. Each runbook is a single JSON file at:

```
<MINIONS_DIR>/projects/<project-name>/runbooks/<runbook-id>.json
```

This mirrors the `projects/<name>/pull-requests.json` precedent — anything
scoped to a single project lives under its `projects/<name>/` state dir
rather than a root-level `runbooks/` directory. Two reasons:

1. **Lifecycle parity with the project.** When a project is removed via
   `engine/projects.js removeProject`, its `projects/<name>/` dir is
   archived as one unit. Co-locating runbooks under that dir means they
   travel with the project rather than dangling in a global `runbooks/`
   that has no relationship to the project being removed.
2. **No central collision with multi-project setups.** Two projects can
   pick the same human-readable runbook name without stepping on each
   other on disk. The runbook **id** is still globally unique (kebab-case,
   ≤ 64 chars) so single-id lookups don't need a project hint.

## Schema

```jsonc
{
  "id": "kebab-case-id",           // required, kebab-case, ≤ 64 chars, globally unique
  "name": "Human-readable name",   // required, ≤ 200 chars
  "project": "project-name",       // required, the owning project (matches projects/<name>/)
  "targetName": "string",          // required, the system under test (e.g. process name, URL, target)
  "steps": [                       // ≤ 20 steps
    {
      "description": "Step 1 description",   // required, ≤ 500 chars
      "command": "optional shell command"    // optional, ≤ 2000 chars
    }
  ],
  "expectedArtifacts": [           // ≤ 20 artifacts
    {
      "type": "screenshot",        // required, one of: screenshot | video | log | other
      "label": "Login page",       // required, ≤ 200 chars
      "path": "screenshots/login.png"  // optional hint, ≤ 500 chars
    }
  ],
  "createdAt": "2026-05-20T20:42:00.000Z",  // ISO-8601, set on first save
  "updatedAt": "2026-05-20T20:42:00.000Z"   // ISO-8601, set on every save
}
```

`id`, `createdAt`, and `updatedAt` are managed by `saveRunbook`. The id
must match `/^[a-z0-9]+(?:-[a-z0-9]+)*$/`.

## API

| Method | Path                          | Notes                                                     |
| ------ | ----------------------------- | --------------------------------------------------------- |
| GET    | `/api/qa/runbooks`            | List all. Optional `?project=<name>` filter.              |
| GET    | `/api/qa/runbooks/<id>`       | Fetch a single runbook by globally-unique id.             |
| POST   | `/api/qa/runbooks`            | Create or update. Body is the full runbook spec.          |
| DELETE | `/api/qa/runbooks/<id>`       | Remove a runbook. Returns 404 when not found.             |

Responses:

- `200 { items: [...] }` — list
- `200 { ...runbook }` — get/save
- `200 { ok: true, id }` — delete
- `400 { error, details? }` — validation failure (`details` is the
  `validateRunbook` error array)
- `404 { error }` — not found
- `409 { error }` — cross-project id collision; `deleteRunbook(id)` then
  retry with the new project

## Module

`engine/qa-runbooks.js` exports:

```js
{
  ARTIFACT_TYPES,       // ['screenshot','video','log','other']
  LIMITS,               // schema bounds (idMax, nameMax, stepsMax, ...)
  validateRunbook(spec) // → { ok: boolean, errors: string[] } — never throws
  listRunbooks(project?) // → array of parsed runbook records
  getRunbook(id)        // → record | null (scans all projects by id)
  saveRunbook(spec)     // upsert; throws on validation or cross-project collision
  deleteRunbook(id)     // → boolean; locks the runbook's file before unlink
}
```

All writes use `mutateJsonFileLocked` per the repo convention. Deletes use
`withFileLock` directly to coordinate with concurrent saves before the
unlink (so an in-progress `saveRunbook` rename can't race with the
unlink).

## Run records, artifacts, and UI

The deferred follow-up items (W-mpeiwz6k0005bf34-b/c/d) have since landed. Brief pointers — see [CLAUDE.md](../CLAUDE.md) → "QA validation runs" for the deep dive:

- **Run dispatch + persistence** (`engine/qa-runs.js`): `POST /api/qa/runbooks/run` creates a `qa-runs.json` record with `status ∈ pending|dispatched|running|passed|failed|error` and dispatches a `qa-validate` work item against the runbook's `target`. Read via `GET /api/qa/runs?limit=N&status=...` and `GET /api/qa/runs/<id>`.
- **Artifact contract**: the `qa-validate` agent writes `agents/<id>/qa-run.json` before exit; the engine copies listed files into `engine/qa-artifacts/<runId>/` and serves them via `GET /api/qa/artifacts/<runId>/<file>` (path-traversal-gated, 403 on escape). Per-file size cap: `engine.qaArtifactsMaxBytes`.
- **UI**: `/qa` dashboard page (`dashboard/pages/qa.html`, `dashboard/js/qa.js`) polls `GET /api/qa/runs` every 5s while active; auto-detects screenshots/videos/logs for inline preview.
- **Playbook**: `playbooks/qa-validate.md` (routed via the synthetic `qa-validate` task-type in `routing.md`).
