# Testing policy (Forge product principle)

**Lock date:** 2026-06-04. Replaces the earlier draft
`tdd-first.md` which proposed TDD-first universally.

Pablo's review: TDD-first as a universal product principle has
more downsides than upsides. The cost is real (tokens, latency,
arquitectural lock-in, frustrated UX, ambiguous specs) and only
pays off in a narrow slice (backends + middleware + finance +
regulatory + external integrations). For the majority of Forge
projects (UI / dashboards / CRUDs / demos / glue), TDD-first
slows down iteration without adding value.

This doctrine sets the new rule.

## The rule

1. **Tests are required POST-implementation for every change**
   that lands in a Forge-managed project. The default order is:
   spec -> code -> tests -> scaffold-with-gate.
2. **TDD-first is opt-in**, not default. It applies when the
   slice falls into the **TDD-first scope** (see below).
3. **The scaffold gate** requires the test suite to be green
   before writing to disk, regardless of order. The order
   determines WHEN tests are authored, not whether they exist.

## TDD-first scope (the narrow slice)

Only these domains warrant test-first:

- **APIs / backend endpoints with public contract**. The
  request/response shape is the most stable thing about the
  feature; the implementation will rewrite.
- **Data transformers / validators / parsers** where input
  -> output is the whole point.
- **Financial / regulatory / mathematical logic** (tax calc,
  retention, currency conversion, legal rules). The test
  is the spec.
- **External-service integrations** (Stripe webhook, Polar,
  OAuth, NAC bridges to MCP). Tests capture "this payload
  produces this effect" which is the only stable contract
  when the provider changes internals.

The Analyst marks the spec extraction with `tdd_first: true`
when:

- The plan contains `endpoints` AND no `components`.
- OR any extraction tag matches /financial|tax|regulatory|
  integration|webhook|validator|transformer/i.
- OR the user explicitly passes `--tdd` to `yf spec extract`.

When `tdd_first: false` (default), the Tester slot runs AFTER
the Coder. When true, the Tester slot runs BEFORE the Coder.

## TDD-first explicitly NOT for

- UI / frontend / drag-drop / animations / layout.
- Glue code / scripts / one-off transforms.
- Demos / prototypes / spikes (always `--mode=spike`).
- Stateful interactive surfaces (games, audio, sensor).
- Generated code from templates (Next.js scaffolds, etc.) --
  the template has its own tests; user-added features follow
  the default order.

Trying to TDD any of the above is fiction-writing. The tests
end up tautological (matching implementation) or vacuous
(asserting things that are trivially true). Forge will not
force users into that.

## Order of the 5 Director agents (canonical)

The Tester is NOT a single-shot slot -- it is re-entrant. It
runs at every checkpoint where the suite needs to be authored
or verified. In TDD-first, that's twice. In code-first, that's
once. The diagram below names the runs explicitly.

### TDD-first mode (Analyst tags `tdd_first: true`)

```
Director (orchestrator)
  |
  v
Analyst   -- extract spec + tag tdd_first=true
  |
  v
Tester #1 -- write the test suite (RED). Suite fails because
             no code exists yet.
  |
  v
Coder     -- implement until the suite turns green. Iterates
             with the brain until red->green or max iter cap.
  |
  v
Tester #2 -- verify green + add edge-case pass (3 boundary
             tests). If any test still red after edge pass,
             back to Coder (capped by max iter).
  |
  v
Scaffolder -- apply to disk ONLY when Tester #2 reports
              all-green (happy + edge cases).
```

### Code-first mode (Analyst tags `tdd_first: false`, the default)

```
Director (orchestrator)
  |
  v
Analyst   -- extract spec + tag tdd_first=false
  |
  v
Coder     -- implement straight from the plan.
  |
  v
Tester    -- write the test suite POST-implementation +
             verify green + edge-case pass.
  |
  v
Scaffolder -- apply to disk ONLY when Tester reports
              all-green.
```

In both modes, the **last Tester pass before Scaffolder is the
hard gate**. The flag `tdd_first` decides whether there is also
a FIRST Tester pass that writes red. The Scaffolder gate
(green-required) is identical in both modes.

The bridge ACL stays the same regardless of mode:
`Director.can_call = [Analyst, Tester, Coder, Scaffolder]`
and each worker accepts only from Director. Mode is encoded
in the messages, not in the topology.

## The scaffold gate (regardless of order)

`Scaffolder` refuses to write to disk when:

- The Tester report does not exist for the slice.
- Any test in the new suite is red.

The gate is enforced in `src/chat/spec_scaffold.ts`. Test
runner auto-detected from `package.json` / `pyproject.toml` /
etc.

## Loop protection (when iterating to green)

If the Coder iterates more than `agent.director.max_iters_to_green`
times (default 5) without turning the test suite green, the
agent stops and reports the failure to the Director, which
opens an R8 modal with three options for the user:

1. Mark suite as accepted-red (the user knows the tests are
   wrong; accept the implementation as-is). Audit-logged.
2. Re-extract spec (back to Analyst with clarification).
3. Cancel the slice.

This is the answer to "what does the agent do when it cannot
turn red into green". No silent application. No infinite
iteration.

## Mitigations against test pathologies

Even POST-implementation testing can produce bad suites if
left to the Tester LLM alone. Required mitigations:

- **Contract framing in Tester system prompt**: the prompt
  explicitly says "tests assert CONTRACT (input -> output,
  effect on world), NOT implementation. Internal refactors
  should not break a well-written test."
- **Edge-case pass**: after the happy-path tests are green,
  the Tester runs a second pass generating 3 edge cases
  (nulls, empty, boundary, error). Adds them to the suite.
  Coder iterates again.
- **No mocking-of-self**: tests cannot mock the function
  they are testing. Mocking external deps (network, fs) is
  allowed.
- **Tester report includes coverage line + branch %.**
  Below configurable threshold (default 80% line, 70%
  branch) is logged as a warning, not a block.

## What this doctrine covers + does NOT cover

This doctrine covers **functional correctness via tests**.
It does NOT cover:

- **UX quality** -- panel review, manual walkthroughs.
- **Accessibility** -- separate doctrine `accessibility.md`
  + `yujin.a11y.*` verbs.
- **Performance** -- separate (future doctrine, not blocking).
- **Security** -- separate.
- **Design / aesthetic** -- not testable by code.

"100% E2E tests passed" in the Director objective means
"functional contract passes". Other quality dimensions are
gated separately or via human review.

## How Sumi enforces this on Forge itself (dogfood)

Sumi-self ships every Forge slice with tests. The order
follows the same rule:

- Backend / middleware / integration slice -> TDD-first
  (test then code). The bridge.ts slice was this kind, and
  the bug found by the test is the evidence the order paid
  off there.
- UI / panel / cosmetic slice -> code-first then test. The
  legal modal opacity fix, the audit log preview, the
  Limites alignment -- all of these were code-first because
  the shape was visual and exploratory.

Sumi decides the order per slice. The graph rule #12 reflects
this.
