# Verification Protocol

The Phase 05 SSOT — how the orchestrator verifies a migration is complete and clean before committing. Phase 04 collected N cluster reports; this protocol turns those reports plus an independent global grep into a binary verdict (clean / yellow / red) and a final residual classification.

---

## The Three Verification Streams

Phase 05 runs three independent verification streams. ALL THREE must pass for a clean verdict:

| Stream | Source | What it proves |
|--------|--------|----------------|
| **Residual grep** | Orchestrator runs `Grep(pattern=FROM_PATTERN, …)` across the migration's `scope_dirs` | Every remaining `from_pattern` occurrence is intentional (one of 4 residual classifications). |
| **Dual-path sweep** | Orchestrator runs `diff -rq` across each dual-path subtree | Every edited file pair is byte-identical; no drift. |
| **FLAGGED resolution audit** | Orchestrator walks the consolidated FLAGGED list from Phase 04 | Every FLAGGED site has either been hand-resolved OR is deferred with a tracking ID. |

A pass on all three lets Phase 06 commit. A fail on any one HALTs the workflow.

---

## Residual Grep Classification

After Phase 04, the orchestrator re-greps the migration's `scope_dirs` for `from_pattern`. Every remaining hit MUST fall into exactly ONE of four classifications:

| Classification | Definition | Resolution |
|----------------|-----------|------------|
| `ALREADY-MIGRATED` | Prose that already explains the migration ("X retired per ADR-NNN"). | LEAVE (intentional). |
| `DOC-historical` | Sentence records a past event — changelog row, MIGRATION-NOTES entry, postmortem prose. | LEAVE (history must read true at the time it was written). |
| `EXCLUDED` | File path matches an `EXCLUSIONS` glob — out of scope by design (templates, archives, `/plans/`). | LEAVE (file was never edited). |
| `FLAGGED-type-sensitive` | The cluster worker deliberately FLAGGED this occurrence; the orchestrator either resolved it in Phase 05 hand-fix OR deferred it with tracking. | LEAVE the original `from_pattern` only if the fix was structural-around-the-noun; otherwise the noun itself has been transformed during hand-resolution. |

### What's NOT a valid residual

A residual classified as `READ-OP`, `LISTING`, `PATH-VALIDATION`, or `DOC-current-state` is a FAILURE — these are transformable occurrences a cluster worker missed. The protocol:

1. Identify which cluster owned the file. Phase 05 HALTs and the orchestrator re-invokes that cluster worker with a minimal scope of `{the missed file}`.
2. The re-invoked worker applies the same transform-rules and self-`diff`s.
3. The residual grep is re-run; this loop continues until no transformable residuals remain.

Five+ such missed occurrences across multiple files is a contract breach — the orchestrator HALTs and AskUserQuestions whether to abort the migration or attempt orchestrator-inline cleanup.

### Residual grep command

```bash
# Per scope_dir (the orchestrator iterates):
Grep(pattern="<FROM_PATTERN>", path="<scope_dir>", output_mode="content", "-n": true)
```

The `-n` flag (line-numbered output) lets the orchestrator classify each hit by file:line cross-referenced against `classification.md` + the consolidated cluster reports.

---

## Dual-Path Sweep

For dual-path migrations, the orchestrator runs:

```bash
diff -rq .claude/agents/   src/claude/agents/
diff -rq .claude/skills/   src/claude/skills/
diff -rq .claude/commands/ src/claude/commands/
diff -rq .claude/rules/    src/claude/rules/
diff -rq .claude/hooks/    src/claude/hooks/
```

Each command's expected output is EMPTY (subject to the documented pre-existing drift in `CLAUDE.md` ↔ `src/CLAUDE.md`, which is out-of-scope for a generic migration).

Any non-empty output lines after the migration's edits are DRIFT. The orchestrator hand-reconciles each drifted pair (using `src/` as the authoritative side per `dual-path-protocol.md`) and re-runs the sweep until empty.

For migrations declared single-path in `transform-rules.md` Section 2, this stream is skipped — the migration report documents which tree was edited.

---

## FLAGGED Resolution Audit

The orchestrator reads `tmp/${MIGRATION_ID}/phase-04-cluster-reports.md`, extracts every FLAGGED site (across all clusters), and verifies each has a disposition:

| Disposition | Required artifacts |
|-------------|---------------------|
| RESOLVED | An orchestrator-side `Edit` log (timestamp + before/after diff snippet) + a sanity-test result (smoke-check or test run) |
| DEFERRED | A tracking ID (STORY-NNN, GitHub issue, or follow-up plan entry) recorded in the migration report's "FLAGGED deferred" section |
| INVALIDATED | A documented rationale (e.g. "site was a false positive — re-reading confirmed it's a pure noun-swap, transformed inline") |

A FLAGGED site lacking ANY of those three dispositions HALTs Phase 05. The orchestrator MUST dispose of every flag before Phase 06 may commit.

---

## Revert-Test Pattern

When a migration touches tested code (`*.py`, `*.test.*`, integration tests under `tests/`) and tests fail after the cluster worker's edits, the orchestrator MUST run the **revert-test** before concluding the migration broke them.

Source: `[[feedback_dual_path_mechanical_migration]]` Step 6 — saved 2 false-positive test failures during the source-tree migration's Phase 7.4.

### Procedure

1. **Identify the failing tests** from the test run output.
2. **Identify the file(s) the affected cluster worker edited** that the failing tests depend on (Grep the test files for imports of the edited modules).
3. **Revert ONLY the cluster worker's edits to those files** to their pre-migration content (use `git checkout HEAD -- <file>`).
4. **Re-run the failing tests.**
5. **Interpret:**
   - Tests still fail → pre-existing red, NOT caused by the migration. Document in the report; revert the revert (restore the cluster worker's edits); continue.
   - Tests now pass → the migration broke them. Re-examine the cluster worker's edits; the worker likely changed something it shouldn't have (e.g. transformed a `LISTING` occurrence inside a test fixture). Hand-resolve, re-test.

### When NOT to revert-test

- Test failures that are clearly the migration's fault (a test asserts on the OLD value and the migration changed it — this is expected; the test needs updating in a follow-up). Document, don't revert.
- Tests that the migration's `scope_dirs` deliberately included (e.g. a test file the migration was supposed to edit). The revert-test is for collateral damage, not intentional changes.

---

## Interpreting Cluster Reports — Green / Yellow / Red

Each cluster's report from Phase 04 has a verdict:

| Verdict | Definition | Phase 05 action |
|---------|------------|-----------------|
| **Green** | Section `(b)` says all pairs byte-identical; section `(c)` residuals all classified; section `(d)` FLAGGED list empty; section `(e)` Uncertain list empty. | Spot-`diff` 5 random pairs (Phase 04 Step 4.4); no further action. |
| **Yellow** | Sections `(a)`–`(c)` clean; section `(d)` has FLAGGED entries OR section `(e)` has Uncertain entries. | Orchestrator hand-resolves FLAGGED in Phase 05; AskUserQuestion for Uncertain calls. |
| **Red** | Section `(b)` reports drift OR section `(c)` shows residuals classified as READ-OP/LISTING/PATH-VALIDATION/DOC-current-state OR the report is missing a section. | HALT; re-invoke the cluster worker OR hand-reconcile pair-by-pair. Phase 05 cannot complete until the report becomes Green or Yellow. |

A "yellow" verdict is normal for migrations with type changes — the FLAGGED list is a feature, not a defect. A "red" verdict is a contract breach and is rare on well-scoped migrations.

---

## Phase 05 Exit Conditions

Phase 05 exit gate fires only when ALL of these are true:

- [ ] Residual grep across all `scope_dirs` returns ONLY occurrences classified as ALREADY-MIGRATED / DOC-historical / EXCLUDED / FLAGGED-type-sensitive.
- [ ] `diff -rq` sweep across every dual-path subtree the migration touched returns empty (modulo the documented pre-existing CLAUDE.md drift).
- [ ] Every FLAGGED site from Phase 04 has RESOLVED / DEFERRED / INVALIDATED disposition.
- [ ] Every Uncertain call from Phase 04 has been resolved via AskUserQuestion (the user chose a disposition).
- [ ] Tests affected by the migration have been run; failures (if any) have either been resolved by the migration OR isolated as pre-existing red via the revert-test.

Failing any of these HALTs the workflow and the orchestrator AskUserQuestions for next-step guidance.

---

## References

- `phase-05-verification.md` — Phase 05's full step-by-step file.
- `dual-path-protocol.md` — the `diff` byte-identity rule.
- `type-change-bug-flagging.md` — FLAGGED site definitions + hand-resolution.
- `classification-bucket-catalog.md` — the bucket definitions that the residual classifier uses.
- `[[feedback_dual_path_mechanical_migration]]` — playbook step 6 (revert-test) + the orchestrator-trusts-but-verifies principle.
- `[[feedback_verify_truncated_subagent_reports]]` — why the orchestrator's independent global verification is non-negotiable.
