---
name: auto-research
description: Iteratively improve a local artifact (draft, document, page) by running a hill-climbing loop. The user names the artifact, the evaluator, and the budget. Claudia edits the artifact, scores it, keeps it if better or reverts if worse, repeats. Use when user says "iterate on this", "loop on this until it's better", "run experiments on this draft", "auto-research this", "make this better and don't stop until it's good". Workspace-scoped, never touches live files, no external actions during the loop.
effort-level: high
invocation: contextual
---

# Auto-Research

A hill-climber for any local artifact with a rubric. Inspired by [Karpathy's autoresearch pattern](https://github.com/karpathy/autoresearch), adapted to Claudia's safety principles.

## Mental model

Three primitives, one governance file:

1. **The artifact** — what gets iterated on (a draft email, a brief, a wiki page, a one-pager). Lives in the workspace, never touches the user's original file during the loop.
2. **The evaluator** — a scalar rubric. Plain English. An independent Checker (the `loop-checker` agent, Haiku) scores each iteration against it, not Claudia herself. The rubric must produce a number for the original artifact before the loop starts.
3. **The budget** — iteration count (default 20). Loop stops when budget is exhausted or score plateaus.
4. **The program** (governance file) — what the user wants, what's off-limits, what counts as "better." Claudia helps the user write this if they don't volunteer it.

Loop: read program + artifact + results history, Claudia (the Maker) proposes one specific change, implements it, then dispatches the `loop-checker` (the Checker) to score it independently. If the Checker's score beats the running best, ratchet (commit); if worse, revert (git reset). Write the status file. Report each iteration as a one-line update. Repeat until budget is hit.

## When to invoke this skill

User explicitly asks:
- "Iterate on this until it's [adjective]"
- "Loop on this draft"
- "Run experiments on this"
- "Auto-research this"
- "Hill-climb this against [criterion]"
- "Make this better, don't stop until it's good"

User implicitly invokes when:
- They've shared an artifact and a quality bar in the same message
- They've tried 2-3 manual revisions of the same draft in the same session and are still unsatisfied

## When NOT to invoke

Refuse to start the loop if:

1. **External-action artifact.** The artifact is, or directly drives, an external action (an email about to be sent, a Slack message in the compose window, a calendar invite). Auto-research on these is too easy to misuse. Hand-iterate with the user instead.
2. **No clear evaluator.** The user can't articulate what "better" means even after one pass of helping. Without an evaluator, the loop hill-climbs the wrong hill.
3. **Sensitive content.** Medical, deeply personal, legal-defensible artifacts. Iteration risks introducing fabricated detail that looks plausible. Decline and explain.
4. **The artifact is already good.** If you score the baseline and it's above the user's stated threshold, tell them and offer one quick polish instead of N iterations.
5. **Bold structural change is the actual need.** Karpathy's own limitation: RLHF-trained iteration is "cagy and scared." Iterations tend toward safe edits, not bold reframings. If the user wants a fundamentally different angle, iteration will not get them there. Suggest a fresh draft instead.

## Workspace layout

Each run gets its own workspace at `~/.claudia/auto-research/<task-id>/`:

```
~/.claudia/auto-research/<task-id>/
├── program.md           the brief (user-authored with Claudia's help)
├── artifact.md          (or .txt, the working copy that gets edited)
├── original.md          immutable copy of the input, for reference and diff
├── results.tsv          one row per iteration: timestamp, score, kept/reverted/contested, change-summary
├── research_status.md   loop control plane (see docs/loop-status-schema.md): iteration, verified, score, checker_verdict, next_action
├── best.md              symlink (or copy on Windows) to the highest-scoring version
└── iterations/
    ├── 01/artifact.md
    ├── 02/artifact.md
    └── ...
```

The task-id is the slugified user phrase plus a short timestamp: `iterate-board-update-20260515-1430`.

**Critical:** The loop edits `artifact.md` inside the workspace. The user's original file (wherever it lives in their file system) is **NEVER** modified during the loop. At the end, Claudia asks the user where to put `best.md`; the user decides.

## program.md template

```markdown
# Program for: <one-line description of the task>

## Goal

(1-3 sentences. What is the end state Claudia is iterating toward?)

## Evaluator (rubric)

Each iteration is scored 0-10 on each dimension. Total score = sum. Higher is better.

| Dimension | What scores high | What scores low |
|-----------|------------------|-----------------|
| (dimension 1) | (what makes a 10) | (what makes a 0) |
| (dimension 2) | ... | ... |
| (dimension 3) | ... | ... |

## Hard constraints (do NOT violate)

- Length cap: stay under N words.
- Must contain: specific phrase or fact.
- Must NOT contain: forbidden phrasings, names, claims.
- Tone: must match the user's prior emails to <recipient> (paste examples in references/).
- (etc.)

## Budget

- Max iterations: 20 (default)
- OR stop when score plateaus (no improvement for 5 iterations in a row)

## Out of scope

- (anything Claudia might be tempted to do that isn't the goal)
```

Claudia's first action when invoked: read the artifact, draft a program.md based on what the user said, present it for confirmation, then start the loop.

## The loop (Claudia's internal workflow)

For each iteration N:

1. **Read the state.** Read `program.md`, `artifact.md`, last 3 rows of `results.tsv`.
2. **Propose one change.** One specific edit, justified in one sentence. Not a rewrite. Not a refactor. One change.
3. **Implement.** Apply the edit to `artifact.md`. Save.
4. **Score (independent Checker).** Dispatch the `loop-checker` agent (Haiku) with the rubric from `program.md`, the new `artifact.md`, and the one change you just made plus your expected effect (your Maker self-score). The Checker returns a verdict JSON: `verified`, `score`, `max_score`, `issues`, `rationale`, `hard_constraint_violated`. You do NOT score it yourself; the Checker's `score` governs the decision. (See `.claude/skills/_loop/checker.md`.)
5. **Decide.** If the Checker's `score` > best score so far and `hard_constraint_violated` is false: ratchet. Copy `artifact.md` to `iterations/NN/`, update `best.md` to point at the new winner, append a row to `results.tsv` marked `kept`. Otherwise: revert. Copy `iterations/(best_iter)/artifact.md` back to `artifact.md`, append a row marked `reverted`.
6. **Flag divergence.** Compare the Checker's `score` against your Maker self-score. If they diverge by more than 1.5 points on a 10-point scale, mark the iteration `contested` in `results.tsv`. The Checker's verdict still governs; the flag just surfaces the disagreement in the end-of-run summary.
7. **Write the status file.** Atomically update `research_status.md` with `iteration`, `verified`, `score`, `budget_remaining`, `last_input`, `maker_proposal`, `checker_verdict` (the Checker's rationale), and `next_action`. Write to a temp sibling and rename; never edit it in place. (See `docs/loop-status-schema.md`.)
8. **Report.** One line to the user: `iter N: score 7.4 (kept), change: tightened lede to one sentence`. Append `[contested]` when flagged.
9. **Check stop conditions and self-repair.** If the Checker has returned `verified: false` 3 times in a row, or its score has regressed for 3 straight iterations, enter the self-repair sub-loop (see `.claude/skills/_loop/repair.md`) before giving up: it diagnoses whether the rubric or a brief is the real problem, proposes one fix, and proves it on the exact failing input. If budget exhausted: stop. If score plateaued (no improvement for 5 iterations): stop. Otherwise: next iteration.

## The independent Checker (Maker-Checker)

This loop separates the producer from the verifier (Proposal 11, E2). Claudia is
the **Maker**: she reads the state and proposes one change. The **Checker** is a
separate agent (`loop-checker`, Haiku) that scores the result independently. The
two never collapse into one model grading its own work, which is the bias the
split exists to remove.

- The Checker's `score`, not Claudia's, drives keep/revert. Claudia's expected
  effect is only a self-score, used to detect disagreement.
- The Checker is adversarial by brief: it hunts for faults and defaults to
  `verified: false` when uncertain. A passing verdict therefore means something.
- A `hard_constraint_violated: true` verdict forces a revert regardless of score.

**Contested iterations.** When the Checker's score and Claudia's self-score
diverge by more than 1.5 points (10-point scale), the iteration is marked
`contested`. The Checker still governs the decision; the flag makes the
disagreement visible. The end-of-run summary reports how many iterations were
contested, which is a useful signal that the rubric is ambiguous or that Claudia
was over-optimistic.

**Cost.** The Checker runs on Haiku, so the verification pass is cheap and fast.
With the default 20-iteration budget, that is at most 21 Checker dispatches
(including the baseline), each scoped to a single small artifact.

**Self-repair.** If the loop stalls (the Checker fails 3 times running, or scores
regress), `auto-research` enters a bounded self-repair sub-loop that treats the
harness, not the artifact, as the suspect: it diagnoses the rubric or a brief,
proposes one fix, and proves it on the exact failing input before adopting it.
See `.claude/skills/_loop/repair.md`. It is capped at 2 repair attempts and never
auto-edits shipped briefs (those changes go to a human approval gate).

## Safety rules (mandatory, follow without exception)

1. **Workspace-only edits.** The loop never writes to a file outside `~/.claudia/auto-research/<task-id>/`. The user's original file at `/Users/.../wherever.md` is untouched until the user explicitly approves the final hand-off.
2. **No external actions during the loop.** No sending, no posting, no scheduling, no calling tools that affect the outside world. The loop is a closed system.
3. **Baseline-score gate.** Before iteration 1, dispatch the `loop-checker` to score the original artifact and write it to `results.tsv` as `iter 0: score X (baseline)` and to `research_status.md` as iteration 0. If the Checker cannot produce a number for the original, the loop does not start; Claudia tells the user the rubric needs to be more specific.
4. **Bounded budget.** Default 20 iterations. User can set higher (50 max) or lower. Plateau detection (no improvement for 5 in a row) stops early. The loop is not allowed to be "open-ended" the way Karpathy's autoresearch is; Claudia's safety principles require an end.
5. **Per-iteration revertability.** Each iteration is a git commit inside the workspace. Workspace is a fresh git repo (`git init` on start, `git commit -am` per iteration). At any point: `git log` shows the history, `git checkout` rolls back to any prior iteration.
6. **User interrupt at any time.** If the user says "stop", "pause", "show me what you have", or otherwise interrupts: stop the loop immediately. The workspace persists. The best version so far is in `best.md`. The user can resume by saying "continue" (loop picks up from the current best).
7. **Honest about the conservatism ceiling.** Karpathy named this: RLHF-trained iteration tends toward safe, local-optimum edits. If the loop plateaus and the user clearly wanted bold reframing, say so: "I've plateaued at score 7.2. The iterations have been incremental polishing. If you wanted a fundamentally different angle, this loop won't get there; a fresh draft might."

## Hand-off at the end of the loop

When the loop stops (budget, plateau, or user interrupt):

1. Read `best.md`.
2. Read `results.tsv` and summarize: started at score X, ended at score Y, took N iterations, here are the three biggest jumps (which iterations and what they changed).
3. Show `best.md` content to the user.
4. Ask: "Want me to put this back at the original location (`<path>`), save it to a new location, or just leave it in the workspace?"
5. **Only after explicit user confirmation**, copy `best.md` to the destination they name. The user's original file is touched here for the first time and only here.
6. Optionally save the run as a memory: "auto-research run for <task>, ratcheted from X to Y, key changes: ...". Useful if the user wants to learn from past runs.

## What this skill is NOT

- **Not a draft generator.** It improves an artifact that already exists. If the user wants a draft from scratch, use `draft-reply`, `follow-up-draft`, `summarize-doc`, or just write it conversationally first.
- **Not for bold reframing.** It's a hill-climber. It finds local optima. For "completely rethink this from a different angle", iteration is the wrong tool.
- **Not autonomous in the Karpathy "NEVER STOP" sense.** Claudia's safety principles require bounded loops and user-in-loop hand-off. The loop runs without per-iteration approval, but it stops, and external actions stay with the user.

## Examples of good rubrics

For "iterate on this board update":

| Dimension | 10 | 0 |
|-----------|----|----|
| Lede strength | Opens with the one number or decision that matters | Opens with throat-clearing or backstory |
| Brevity | Reads in under 90 seconds | Goes over 5 minutes |
| Ask clarity | The ask, if any, is one sentence | Asks are buried or multi-step |
| Tone | Matches my prior board updates (samples in references/) | Sounds AI-generated |

For "iterate on this proposal":

| Dimension | 10 | 0 |
|-----------|----|----|
| Problem framing | Names the client's actual pain in their words | Generic problem framing |
| Differentiation | One sentence on why I'm the right fit | No clear differentiator |
| Scope clarity | Phases are explicit, no scope creep | Vague deliverables |
| Price anchoring | Price appears after value, not before | Price leads or is hidden |

Specific. Numerical. Tied to the user's actual standards.

## Examples of bad rubrics

"Make it better" — no signal.
"More professional" — directionally OK but no scoring criteria.
"Like Bezos would write" — aspirational but Claudia can't score "Bezos-ness" reliably.

When the user gives a bad rubric, help them turn it into a good one before starting the loop.

## See also

- `wiki` for iterating wiki pages specifically (compress, sharpen, restructure)
- `draft-reply`, `follow-up-draft` for one-shot draft generation
- `summarize-doc` for one-pass summarization
- `.claude/skills/_loop/` for the shared Maker and Checker role briefs
- `.claude/agents/loop-checker.md` for the Checker agent definition
- `docs/loop-status-schema.md` for the `research_status.md` control-plane format

## Open questions for future versions

- Shell-command evaluators (run `node test.js my-draft.md`, take the number it prints) are deferred. Today, the `loop-checker` agent scores against the rubric in `program.md`.
- Token-spend budget and wall-clock budget are deferred. Today, only iteration count is supported.
- Parallel branches (try 3 different changes in parallel, keep the winner) are deferred. Today, sequential only.
- Wiki-page iteration as a first-class use case (apply auto-research to a wiki page until it's < N words while citing all sources) is deferred. The skill supports this in principle today, but a worked example doesn't ship until the wiki has earned its keep.
