{"_id":"@dktelford/ai-orchestrator","name":"@dktelford/ai-orchestrator","dist-tags":{"latest":"0.21.0"},"versions":{"0.21.0":{"name":"@dktelford/ai-orchestrator","version":"0.21.0","description":"A minimal plan→act→verify coding orchestrator driven by a Claude Code sub-agent, with sealed adversarial evaluation, a git ratchet, and a Goodhart promote gate.","type":"module","main":"dist/index.js","types":"dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./package.json":"./package.json"},"bin":{"ai-orchestrator":"dist/cli.js","boundary-check":"standalone/boundary-check.mjs"},"scripts":{"build":"tsc","build:standalone":"npm run build && esbuild dist/boundary-standalone.js --bundle --platform=node --format=esm --outfile=standalone/boundary-check.mjs --banner:js='#!/usr/bin/env node'","test":"vitest run","test:watch":"vitest","dev":"tsc --watch","start":"node dist/cli.js","prepublishOnly":"npm run build && npm test && npm run build:standalone","check:standalone":"git ls-files --error-unmatch standalone/boundary-check.mjs > /dev/null && npm run build:standalone && git diff HEAD --exit-code -- standalone/","build:releases":"npm run build && node dist/releases-build.js","check:releases":"git ls-files --error-unmatch releases.json > /dev/null && npm run build:releases && git diff HEAD --exit-code -- releases.json"},"keywords":["ai","orchestration","claude","agent","automation","plan-act-verify","adversarial-verification"],"license":"MIT","author":{"name":"Dustin Telford"},"repository":{"type":"git","url":"git+https://github.com/dustin-telford/generic-ai-orchestrator.git"},"homepage":"https://github.com/dustin-telford/generic-ai-orchestrator#readme","bugs":{"url":"https://github.com/dustin-telford/generic-ai-orchestrator/issues"},"publishConfig":{"access":"public"},"engines":{"node":">=20"},"dependencies":{"zod":"^3.22.0"},"devDependencies":{"@types/node":"^20.0.0","esbuild":"0.21.5","typescript":"^5.0.0","vitest":"^2.0.0"},"gitHead":"b8826987dca2685500b82a519f664b076c8eab1c","_id":"@dktelford/ai-orchestrator@0.21.0","_nodeVersion":"24.18.0","_npmVersion":"11.16.0","dist":{"integrity":"sha512-e13F4qi+b79w7lpOyPtMWLY1PG/BXlitYuNF2MK+s8iQ2jmRVGczcPIUq/bIsCMJmHmMPTt5ubVU8s6s/pfRnA==","shasum":"3e24fb0c462f99a2c414010180f894b50ce5b42e","tarball":"https://registry.npmjs.org/@dktelford/ai-orchestrator/-/ai-orchestrator-0.21.0.tgz","fileCount":200,"unpackedSize":1183635,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQDr68+epsWJSr8kIVyG7IEB9B/wWtQmjz+KNK+cbzU45gIhAPBp08UbKV/bbs0tqH9ZduUpLuNue859CAMatkqMqQ3o"}]},"_npmUser":{"name":"dktelford","email":"dustin.telford@gmail.com"},"directories":{},"maintainers":[{"name":"dktelford","email":"dustin.telford@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/ai-orchestrator_0.21.0_1783734431532_0.43349807659236195"},"_hasShrinkwrap":false}},"time":{"created":"2026-07-11T01:47:11.437Z","0.21.0":"2026-07-11T01:47:11.732Z","modified":"2026-07-11T01:47:11.883Z"},"maintainers":[{"name":"dktelford","email":"dustin.telford@gmail.com"}],"description":"A minimal plan→act→verify coding orchestrator driven by a Claude Code sub-agent, with sealed adversarial evaluation, a git ratchet, and a Goodhart promote gate.","homepage":"https://github.com/dustin-telford/generic-ai-orchestrator#readme","keywords":["ai","orchestration","claude","agent","automation","plan-act-verify","adversarial-verification"],"repository":{"type":"git","url":"git+https://github.com/dustin-telford/generic-ai-orchestrator.git"},"author":{"name":"Dustin Telford"},"bugs":{"url":"https://github.com/dustin-telford/generic-ai-orchestrator/issues"},"license":"MIT","readme":"# ai-orchestrator\n\n> 📍 **Returning / picking up mid-stream?** [`docs/SESSION-STATE.md`](docs/SESSION-STATE.md) is the latest handoff — branch, what shipped, what's open, and how to resume.\n\nA minimal **plan → act → verify** coding orchestrator. You give it a goal; it asks a Claude Code\nsub-agent to break it into small steps, makes each change, and **only accepts a step when a real\nshell check passes**. State is persisted as JSON so runs are resumable.\n\nThe `plan→act→verify` core is deliberately small and honest about what it does; the governed platform\nlayer on top of it has since grown to ~54 source files. It was built by dogfooding on its own repo —\nincluding [implementing one of its own features](docs/dogfood-phase2.md).\n\n> Status: **v0.21 — the Sealed-Eval Ratchet core + a multi-agent platform layer.** The original\n> `plan→act→verify` tool still works end-to-end on real repos; on top of it sits a governed platform\n> in which **Claude orchestrates multiple AI adapters** (Claude, Codex, local Ollama) across a routed\n> task graph. See the **[Platform layer](ARCHITECTURE.md#platform-layer-multi-agent-orchestration)**\n> in [`ARCHITECTURE.md`](ARCHITECTURE.md), plus [`SPEC.md`](SPEC.md) and [`CHANGELOG.md`](CHANGELOG.md).\n> Every feature ships with tests (**576 across 58 files**, `benchmark` **19/19**) and the discipline is\n> unchanged: **no feature without a verification step.**\n>\n> **Platform commands:** `platform \"<goal>\" [--template <id>] [--adversarial N]`, `capabilities`,\n> `adapters`, `workflows`, `skills`, `reputation`, `calibration`, `cost`, `estimate`, `mcp`,\n> `security`, `stack-risk`, `visual`, `compete`, `lessons`, `platform-runs`/`platform-show`. The\n> platform routes each role to whichever AI is capable + has the best record, never lets the\n> implementer verify its own work, gates high-risk tasks on human approval, and records provenance for\n> every accepted change.\n>\n> **Honest scope of the cross-provider claim:** a live run with Codex installed\n> ([`docs/dogfood-codex-crossprovider.md`](docs/dogfood-codex-crossprovider.md)) proved the *mechanism*\n> — Codex really runs as a verifier on a different provider and catches real defects, failing closed on\n> crash/malformed/empty output. It did **not** demonstrate cross-provider *value*: on the battery so far,\n> Codex caught nothing Claude self-verification missed. And model judges *alone* fail open to a forged\n> verdict — so high-risk acceptance requires a **deterministic held-out check**, and adversarial judges\n> must be **cross-provider** (both now enforced + sealed-benchmarked). We claim the mechanism, not the value.\n>\n> **Independent field corroboration (2026-07-01):** two downstream repos reported on how they use this\n> project ([`docs/field-reports/`](docs/field-reports/)). Neither runs the engine or exercises\n> cross-provider verification — both adopted the *verification discipline* (ratchet / boundary guards /\n> fail-closed checks / adversarial roundtable) as prose + scripts and skipped the binary and the diversity\n> (one measured **0 of 33 loops** using a non-Claude verifier). So the honest headline is: **this is a\n> verification discipline with a cross-provider option whose value is measured by\n> [`valuebench`](docs/RELEASE-READINESS.md#tier-0--immense-decides-whether-the-core-claim-is-true), not\n> asserted.** To adopt the discipline directly, see **[`docs/ADOPTION.md`](docs/ADOPTION.md)**.\n>\n> **Near-term vs. long-term (2026-07-03):** the shippable identity *today* is the **verification\n> discipline** (adoptable credential-free). Measuring the cross-provider *value* live, bringing up a real\n> multi-provider ensemble, npm publish, and the live downstream connector are **credential-gated\n> ambitions** — deliberately parked as the longer-term roadmap in **[`docs/VISION.md`](docs/VISION.md)** so\n> they no longer gate day-to-day work. They execute the moment their one external input (a paid/quota'd\n> second provider, an npm org, downstream repo scope) arrives.\n\n> ⚠️ **Security:** this tool executes model-proposed commands and edits **on your machine with your\n> permissions** — run it only on repos and manifests you control. A planner gate blocks known-destructive/\n> exfiltrating checks (defence-in-depth, not containment), and an **opt-in** `Policy.sandbox:'unshare'` runs\n> the *check* step under a Linux user+network namespace that scrubs ambient credentials and cuts network —\n> a real but partial control (not a filesystem jail, and edits still run on the host). Read\n> **[`SECURITY.md`](SECURITY.md)** + the **[threat model](docs/THREAT-MODEL.md)** before pointing it at anything.\n\n## Quickstart (≈60 seconds)\n\n```bash\n# 1. Prerequisite: the `claude` CLI, installed and authenticated (used as the execution substrate).\nclaude --version\n\n# 2. Get the tool (from a checkout)\ngit clone https://github.com/dustin-telford/generic-ai-orchestrator && cd generic-ai-orchestrator\nnpm install && npm run build\n\n# 3. Run it on a goal — it plans, edits files via a Claude sub-agent, and verifies each step\nnode dist/cli.js orchestrate \"add a slugify(s) helper in src/ with a vitest test\"\n\n# 4. Inspect what happened\nnode dist/cli.js list                 # run ids\nnode dist/cli.js show <runId>         # plan + per-step status\n```\n\nSafe to try first: add `--dry-run` to only plan (no edits), or point at any repo with `--cwd <path>`.\nOnce published you'll be able to skip the clone with `npx @dktelford/ai-orchestrator orchestrate \"…\"`\n(see [`PUBLISHING.md`](PUBLISHING.md)).\n\nTo run the **multi-agent ensemble** (Claude conducting several AIs across a task graph), see\n[`examples/README.md` §0](examples/README.md) and the shipped manifests in [`examples/`](examples/).\n\n**Adopting this in your own repo?** [`docs/ADOPTION.md`](docs/ADOPTION.md) has a two-tier path: Tier 1 is\nthe discipline alone (the shipped `boundary-check` gate + a portable ratchet, no engine wiring); Tier 2 is\nthe full best-of-breed cross-provider ensemble with your own custom skills and agents.\n\n## Documentation\n\nNew here? The [**documentation map**](docs/README.md) routes you to the right docs by who you are —\n**user**, **developer**, **conductor** (Claude-as-lead), **agent/provider seat**, or\n**operator/security** — and organizes everything as tutorial · how-to · reference · explanation.\n\n## The Sealed-Eval Ratchet (v0.3)\n\nThe headline idea: **an agent that cannot lie to you about whether it succeeded.** Most agentic loops\noptimize a metric they can quietly game (Goodhart). SER composes five guards so gaming is structurally\nhard to hide:\n\n1. **Git-as-ratchet** — each accepted step is committed on a work branch; a rejected step is reset to\n   the last good checkpoint (`--ratchet`).\n2. **Sealed evaluation** — after the visible check passes, an optional **held-out check the act agent\n   never saw** must also pass.\n3. **Fail-closed adversarial ensemble** — independent judges inspect the actual diff; an unparseable\n   verdict retries once then counts as a refute. Acceptance needs a strict majority of clean upholds\n   (`--adversarial N`, or the `judge` command).\n4. **Goodhart guard** — promote to the base branch only if the held-out slice moved with the visible\n   metric; \"metric up, held-out flat\" is flagged as suspected gaming (`--promote` / `promote`).\n5. **Propose-then-promote** — auto-merge only when all signals pass, else surface a PR-style summary.\n\n```bash\n# the whole loop in one command\nai-orchestrator orchestrate \"<goal>\" --ratchet --adversarial 3 \\\n  --promote --metric \"<cmd>\" --held-out \"<cmd>\"\n```\nSee [`docs/dogfood-v0.3-ser-endtoend.md`](docs/dogfood-v0.3-ser-endtoend.md) for a real run.\n\n## Why\n\nMost \"AI did it\" output is unverified. The core idea here is boring and load-bearing: **every change\nis followed by a check, and a step isn't done until its check exits 0.** On failure the orchestrator\nfeeds the error back for one bounded retry. Optionally, independent judges try to *refute* a change\nbefore it's accepted (`--adversarial`).\n\n## Requirements\n\n- Node.js >= 20\n- The [`claude`](https://docs.claude.com/en/docs/claude-code) CLI installed and authenticated\n  (the orchestrator shells out to `claude -p` — no API key handling of its own).\n\n## Install / run\n\n```bash\n# from a checkout\nnpm install && npm run build\nnode dist/cli.js orchestrate \"<goal>\"\n\n# or, once published (the bare `ai-orchestrator` npm name is taken, so it's scoped)\nnpx @dktelford/ai-orchestrator orchestrate \"<goal>\"\n```\n\n## Usage\n\n```text\nai-orchestrator orchestrate \"<goal>\" [options]    # plan, act, verify (+ ratchet/promote)\nai-orchestrator explore \"<question>\" [options]     # parallel explorers -> one synthesis\nai-orchestrator judge \"<step>\" [--check] [--adversarial n]   # fail-closed ensemble on the diff\nai-orchestrator promote --work <branch> --metric \"<cmd>\" --held-out \"<cmd>\" [--base <branch>]\nai-orchestrator adapters                           # list model adapters + live health (claude, ollama)\nai-orchestrator sbom [--cwd <path>]                # write a CycloneDX sbom.json from the lockfile\nai-orchestrator provenance [--cwd <path>]          # show the accepted-step provenance trail\nai-orchestrator resume <runId>                     # continue an interrupted run\nai-orchestrator list / show <runId>                # inspect runs\n\nOptions:\n  --max-steps <n>         Max steps to execute (default 20)\n  --max-agent-calls <n>   Max Claude sub-agent calls (default 40)\n  --adversarial <n>       Fail-closed judges per step (majority clean-uphold required; default 0)\n  --ratchet               Commit each accepted step on a work branch; reset rejected steps\n  --promote               After a ratchet run, run the Goodhart gate (needs --metric & --held-out)\n  --metric / --held-out   Shell checks for the promote gate (on-metric vs sealed held-out slice)\n  --no-rollback           Don't auto-revert the git tree if a non-ratchet run fails (default: on)\n  --dry-run               Plan only; don't act or verify (safe, read-only)\n  --lenses <a,b,c>        explore: explorer perspectives (default: effort-routed)\n  --concurrency <n>       explore: max parallel explorers (default min(lenses,4))\n  --cache                 explore: reuse a stored result for a repeat question\n  --cwd <path>            Repo to operate on (default: current directory)\n```\n\nSee [`examples/`](examples/README.md) for worked examples.\n\n### Environment variables\n\n| Var | Default | Purpose |\n|---|---|---|\n| `ORCHESTRATOR_MODEL` | `opus` | Model passed to every `claude -p` sub-agent. |\n| `ORCHESTRATOR_AGENT_TIMEOUT_MS` | `600000` (10 min) | Per-sub-agent hard timeout. |\n| `ORCHESTRATOR_MAX_CONCURRENCY` | `4` | Global cap on parallel explorers when `--concurrency` isn't passed. Each explorer is a full `claude -p` process (~hundreds of MB), so this is the **memory amplifier** knob — on a small/WSL VM, set it to `2`. |\n| `ORCHESTRATOR_ADAPTER` | `claude` | Which model adapter the registry selects by default (`claude` \\| `ollama`). See `ai-orchestrator adapters`. |\n| `OLLAMA_HOST` | `http://127.0.0.1:11434` | Base URL (or `host:port`) of the local Ollama server for the `ollama` adapter. |\n| `OLLAMA_MODEL` | `llama3.2` | Model the `ollama` adapter requests. The `ollama` adapter is read-only (plan/review/refute/explore) — it cannot perform file-editing `act` steps. |\n\n## How it works\n\n```\norchestrate \"<goal>\"\n   │\n   ▼\n plan ──► for each step: act (Claude edits files) ──► verify (run the step's check)\n   │                              ▲                         │\n   │                              └──── 1 retry on fail ◄───┘\n   ▼\n persist Run to .ai-orchestrator/runs/<id>.json after every transition (resumable)\n```\n\n- **Task/Run model** — zod-validated (`src/types.ts`).\n- **Memory** — resumable JSON store under `.ai-orchestrator/` (`src/memory.ts`), auto-gitignored in any\n  target repo (and collision-safe with a target's own `.orchestrator/`).\n- **Agent connector** — wraps `claude -p --output-format json` (`src/agent.ts`).\n- **Loop** — plan→act→verify with budget + explicit stop criteria (`src/orchestrator.ts`).\n- **Sealed eval** — held-out check + fail-closed adversarial judges over the diff (`--adversarial`, `judge`).\n- **Fan-out** — parallel read-only explorers → one synthesizer (`explore`, `src/fanout.ts`).\n- **Ratchet + promote** — per-step commit/keep/reset and a Goodhart promote gate (`--ratchet`, `--promote`).\n- **Auto-rollback** — a failed run reverts its own git edits (`src/rollback.ts`, `--no-rollback`).\n- **Check-vs-edit** — if a check fails identically twice while the tree changed, the run flags the\n  *check* as suspect and preserves the rejected diff instead of discarding good work silently.\n\nThe loop is fully unit-tested with a **mocked agent** (no Claude calls needed) — see `npm test`.\n\n## Guardrails\n\n- Reversible: changes are git working-tree edits; `git checkout` is the undo.\n- Bounded: hard caps on steps and sub-agent calls; the loop can't run forever, and it logs what it\n  skipped.\n- No secret handling: it reuses your existing `claude` auth and never writes secrets.\n\n## Development\n\n```bash\nnpm run build   # tsc -> dist/\nnpm test        # vitest (576 tests / 58 files)\n```\n\nCI runs build + test + `benchmark` + a mutation gate (fixed core list, and PR-changed files)\non every push/PR (`.github/workflows/ci.yml`).\n\n## License\n\nMIT © Dustin Telford. See [`LICENSE`](LICENSE).\n\n---\n\n*This project grew out of orchestration work on the HTM CMMS project and was cleaned up into a\ngeneral-purpose tool. The earlier aspirational framing (multi-model platform, enterprise compliance,\nadoption targets) was removed in favor of a working slice; see `ROADMAP.md` for the honest plan.*\n","readmeFilename":"README.md","_rev":"1-6e6064cca05fa49249a5e6a76834e276"}