# Fix Application Telemetry

This doc describes the telemetry layer added to `/new` (Phases 2.55 / 3 / 3.5 / 3.7) that records, for every check finding, who applied the fix (orchestrator inline vs `coder` subagent) and how big the diff was. The layer is **observation only** — it does not change `/new`'s behaviour. It exists so we can decide, with real data, whether the per-phase delegation rules need to change.

## Why this exists

Empirical observation showed that fixes flagged by the simplify / doc-reviewer / qa-sentinel / codex agents often get applied **directly by the orchestrator** instead of being delegated to the `coder` agent. The cost is twofold: (1) the orchestrator does not inherit the coder's system prompt + project overlay (testing conventions, naming, project rules), and (2) inlining 5+ agent outputs saturates the orchestrator's context window mid-card.

Before changing the rules — which would mean inventing thresholds, domain overrides, and escape valves — we measure the current state. The schema below is the contract.

## Schema (SSOT)

Authoritative definition lives in `framework/.claude/skills/new/SKILL.md` → section **"Fix Application Log Schema (telemetry)"**. The row format:

```
<phase> | <domain> | est_lines=<n> | decision=<inline|coder|coder-batch|skipped> | applied_by=<orchestrator|coder|orchestrator-fallback|-> | <key=val>...
```

Each row is appended to the tracker's `## Fix Application Log` section. One row per finding, including findings the orchestrator skipped as false-positive. The trailing `key=val` pairs are phase-specific extras (`finding=<1-line>`, `severity=<...>`, `retry=<n>`, `reason=<...>`).

## How to read the data

Run the aggregator:

```bash
node framework/scripts/analyze-fix-application.js docs/references/trackers/*.md
```

(Substitute the actual tracker directory if your project uses a different `${paths.references_dir}`.)

Output is a tabular summary grouped by `phase × domain × decision × applied_by`, plus a violation-pattern footer.

### Patterns to look for

**Violation patterns** — these should be zero. If they're non-zero, the orchestrator is bypassing a delegation rule:

| Pattern | What it means |
|---|---|
| `3 \| doc \| ... \| applied_by=orchestrator` | Phase 3 doc-review fix applied inline (violates the `doc` Domain-Override rule — doc fixes are owned by `doc-reviewer`). |
| `3 \| doc \| ... \| applied_by=coder` | Phase 3 doc fix routed to `coder` (violates the `doc` Domain-Override rule since v3.40.0 — doc fixes are owned by `doc-reviewer`, which has the doc-invariant contract the coder lacks). |
| `3.7 \| codex-security \| ... \| applied_by=orchestrator` | Codex HIGH security finding fixed inline (violates `SKILL.md` Phase 3.7 "spawn `coder` agent"). |
| `* \| * \| ... \| applied_by=orchestrator-fallback` | Sub-agent failed and orchestrator fell back to inline. Expected to be rare; investigate if frequent. |

The analyzer script flags these in its footer automatically.

**Delegation candidates** — patterns suggesting the rule could be tightened:

| Pattern | What it suggests |
|---|---|
| `2.55 \| simplify-* \| est_lines≥6 \| decision=inline \| applied_by=orchestrator` (high count) | Large simplify refactors applied inline. Candidate for threshold-based delegation. |
| `2.55 \| simplify-* \| ... \| applied_by=orchestrator` total lines > 50/card avg | Cumulative inline diff bloating orchestrator context. Candidate for `coder-batch`. |
| `3.5 \| qa-blocker \| retry=2` (many rows) | Coder not converging on QA fixes in 2 retries. Candidate for revised qa-sentinel feedback loop. |

**Healthy patterns**:

| Pattern | What it means |
|---|---|
| `2.55 \| simplify-* \| est_lines∈[1-5] \| decision=inline \| applied_by=orchestrator` | Small inline fixes — efficient, low cost. Expected majority of Phase 2.55. |
| `3 \| doc \| ... \| decision=doc-reviewer \| applied_by=doc-reviewer` | Doc fixes applied by the doc-reviewer in write mode (since v3.40.0). Expected for every card with doc impact. |
| `D.4a \| doc \| ... \| decision=doc-reviewer \| applied_by=doc-reviewer` | Team-mode doc fixes applied by the doc-reviewer over the group. Expected for every group with doc impact. |
| `3.7 \| * \| decision=skipped \| applied_by=- \| severity=FALSE-POSITIVE` | Codex false positives correctly filtered. |

## When to act on the data

After roughly **10 cards** of telemetry, run the analyzer and look at:

1. **Violation count**. If non-zero, the rules are being bypassed — Release B (loophole fix + Phase 3 reinforcement) should already have made these zero.
2. **Phase 2.55 inline distribution**. If `est_lines≥6 | applied_by=orchestrator` exceeds 30% of Phase 2.55 rows, Release C (threshold rule) becomes worth considering.
3. **Mean inline diff per card**. Compute `(sum of est_lines for orchestrator rows) / (number of cards)`. > 50 indicates context bloat.

The full decision tree for Release C lives in the plan file (`/Users/antoniobaldassarre/.claude/plans/sto-notando-una-cosa-misty-lightning.md` → "Release C — Threshold Rule (CONDIZIONALE)").

## Cost

Per-card overhead: 5-15 rows × ~80-120 bytes = ~1-2 KB added to the tracker. Orchestrator context impact: negligible (tracker is appended to disk, not held in context after each phase). Analysis cost: a single Node script invocation. The aggregator has zero dependencies and runs in well under a second on hundreds of trackers.

## Pointers

- Schema definition (canonical): `framework/.claude/skills/new/SKILL.md` → "Fix Application Log Schema (telemetry)"
- Per-phase logging instructions: SKILL.md Phase 2.55 step 4, Phase 3 step 15, Phase 3.5 step 23, Phase 3.7 step 4
- Aggregator script: `framework/scripts/analyze-fix-application.js`
- Decision plan: `/Users/antoniobaldassarre/.claude/plans/sto-notando-una-cosa-misty-lightning.md`
