# Extraction-Quality Non-Circularity Protocol (QUAL-03)

This is the QUAL-03 protocol for the **producer-agnostic extraction-quality
instrument** (v1.8.1-B). The instrument scores a producer's emitted atoms/edges
(the OUTPUT) against a frozen gold set; it does not score authorship. For that
number to mean anything, the gold itself must be legitimate and the measurement
must not be circular. This doc documents the four disciplines that make it so,
the (DEFERRED) human-ceiling spike scaffold, the same-model circularity
limitation, and the PROVISIONAL-threshold contract.

> **AI-AUTHORED gold — restated here (the third honesty surface).**
> The gold atom/edge set in `tests/fixtures/extraction-quality/gold/` was
> **authored by an AI (Claude), NOT by a human.** This statement appears on three
> surfaces — the fixture README, every gold-file header, and here. A future
> reader must **never** assume human-curated gold. Same-model circularity is
> acknowledged below (§3), bounded but not engineered away this phase.

## 1. The non-circularity protocol (four disciplines)

### 1a. Label-and-freeze-before-build

The gold was authored **from the raw corpus prose** under the published ATOM-01
split-test (`references/vault-atoms.md`: one canonical claim per atom; multi-claim
passages MUST split), and committed **before** the 98-03 harness or any producer
run existed. Evidence:

- `tests/fixtures/extraction-quality/gold/adjudication-log.md` — the per-decision
  judgment audit trail (each contested span, the split-test verdict, the
  rationale). It is the proof the gold came from judgment on raw prose, not from
  copying machine output.
- Each gold file (`gold-atoms.tsv`, `gold-edges.tsv`, `negative-spans.tsv`) opens
  with an **AI-AUTHORED (not human)** provenance header.
- The 98-01 Task-2 commit lands the gold; no producer-run fixture exists at that
  commit. The freeze is verifiable from git history.

### 1b. Frozen-before-run

The producer scores against an **already-committed** gold. The producer never
sees the gold while authoring its atoms; it reads only the corpus prose. The
matcher (98-02) and the harnesses (98-03) consume the gold **read-only**. The
gold corpus carries a **DO-NOT-REGENERATE** notice precisely because re-deriving
gold from a producer run would collapse this guarantee into circularity.

### 1c. Adversarial negative spans

`tests/fixtures/extraction-quality/gold/negative-spans.tsv` enumerates spans that
must yield **no atom**: boilerplate (headers/footers/disclaimers),
two-claim-must-split run-ons (offered as one span — the producer must split, so a
single atom over the whole span is wrong), and hedged opinions (subjective hedges
that state no checkable claim). The **negative-span FP-rate** is a scored
dimension (98-02): a producer that emits an atom anchored to a negative span is
penalized. Without adversarial negatives, precision could be gamed by a producer
that emits an atom for every sentence.

### 1d. The adjudication log

The adjudication log (§1a) is itself a discipline: it forces each contested
decision to be written down with its split-test verdict, so the gold's judgment
is auditable and reproducible rather than implicit. A reviewer can re-walk every
boundary atom and every two-claim split and check the reasoning against the raw
prose.

## 2. Human-ceiling spike — SCAFFOLD ONLY (DEFERRED + PROVISIONAL)

**User ruling (2026-06-04):** the human-vs-human ceiling is **DEFERRED** and the
number is **PENDING**. This phase ships the *scaffold*, not the number.

```yaml
human_ceiling: PENDING (human input required)
status: DEFERRED (user ruling 2026-06-04)
thresholds: PROVISIONAL-not-ratified
```

### 2a. The scaffold (harness shape, no number asserted)

The spike, when a human runs it, is:

1. A **pinned token-Jaccard matcher** (the same matcher the instrument uses to
   score producer atoms against gold), with TAU pinned to a documented value.
2. A **two-independent-hand-atomizations comparison**: one messy paragraph is
   atomized independently by two humans; the matcher computes the **human-vs-human
   F1** between the two hand atomizations.
3. That F1 is the **human ceiling** — the empirical answer to "how well do humans
   even agree on this task," which bounds how good a machine can be expected to be.

This scaffold is documented here and wired as harness shape; the **actual
human-vs-human F1 is left `PENDING (human input required)`.** No human-validated
number is asserted in this doc, in the fixture README, in the instrument output,
or in the Phase-100 closure audit.

### 2b. Thresholds are PROVISIONAL-not-ratified

Because a human ceiling is the precondition to ratifying numeric pass-thresholds,
and that ceiling is PENDING, **every numeric pass-threshold ships
PROVISIONAL-not-ratified.** The machine/agent bar is expressed **relative to a
clearly-flagged PROVISIONAL ceiling**. The Phase-99 governance ruling (GOV-04)
must consult this clearly-provisional bar and **state in writing** that the
threshold is provisional pending the human-ceiling spike. Do not assert a ratified
threshold anywhere until the spike is done.

### 2c. AI self-consistency is NOT the human ceiling

An **AI-vs-AI baseline** (running the producer twice and comparing its two
atomizations) measures **model self-consistency**, not human task-difficulty.
It MAY be computed and reported separately, but ONLY under the explicit label:

> **AI self-consistency (NOT the human ceiling).**

It must **never** be placed under the "human ceiling" label or used as the
ceiling. Doing so would misrepresent a self-agreement number as a difficulty
ceiling and inflate the apparent quality of the result.

## 3. Same-model circularity limitation (honest, not hidden)

The gold author and the default atomization producer are the **same model
family** (Claude). This is a real limitation and is stated plainly rather than
hidden:

- **Cross-model (codex) is wired as a VERIFIER, not an atomization PRODUCER.** It
  checks/verifies; it does not author the competing atomization that would break
  the same-model loop.
- **No new cross-model producer wiring is added this phase** (explicit scope
  boundary). If a different gold-author model were trivially available it would be
  preferred, but the phase does not block on it.
- This circularity **bounds** the result — a same-model producer may agree with
  same-model gold partly because they share priors — but it **does not invalidate**
  it: the freeze-before-build discipline, the adversarial negatives, and the
  adjudication log still constrain what the gold can be, independent of the
  producer. A future reader must know this limitation when interpreting the
  number. Engineering it away (a genuinely independent gold-author model) is
  future work, not this phase.

## 4. How the number is used (reported-not-gated)

The extraction-quality number is **reported, not auto-acted-on**. It is consulted
by the Phase-99 GOV-04 ruling to bound (not certify) the auto-stage decision for
large unsupervised bulk runs; small human-reviewed batches proceed without it
(per D-V181-05). Trust derives from volume × reversibility × human-attention, not
authorship (D-V181-06) — which is exactly why an honestly-labeled, freeze-before-
build, AI-authored gold legitimately bounds the auto-stage threshold even though
no human ceiling has been established yet.

---

*Substrate: `tests/fixtures/extraction-quality/` (corpus + gold + negatives +
adjudication log + README). Matcher/dimensions: 98-02. Harnesses: 98-03.
Consumes the human ceiling (PENDING): Phase-99 GOV-04 (provisional bar) +
Phase-100 closure audit (QUAL legs). Split-test source:
`references/vault-atoms.md` (ATOM-01).*
