{"_id":"@cullet/ai-harness","_rev":"4-3e52117d9ecd913c9af2f738e48071ac","name":"@cullet/ai-harness","dist-tags":{"latest":"1.4.0"},"versions":{"1.1.0":{"name":"@cullet/ai-harness","version":"1.1.0","license":"MIT","_id":"@cullet/ai-harness@1.1.0","maintainers":[{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"}],"homepage":"https://github.com/fabiano-eduardo/cullet#readme","bugs":{"url":"https://github.com/fabiano-eduardo/cullet/issues"},"dist":{"shasum":"45f20cd87ef7987092185a32b2b121f729f6551e","tarball":"https://registry.npmjs.org/@cullet/ai-harness/-/ai-harness-1.1.0.tgz","fileCount":29,"integrity":"sha512-M69o1zdOy/KcsdpHgiJd+ffEBhk2zBRwweqSlKBd7/Pn3E+jqwHrGc1VDZBNqWnwTgX0lRTiU33l849bdQ1FmA==","signatures":[{"sig":"MEUCIC3SnYg43YzAIH1q23lfB+FP9hgbnP2DVldRVZGWbKNoAiEAhCyoLvYCezs//R9YbxY2x5lixfJ/Y4i1PF6VtJ1NASk=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@cullet%2fai-harness@1.1.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":130119},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","engines":{"node":">=18.17"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./node":{"types":"./dist/runtime/node.d.ts","import":"./dist/runtime/node.js"},"./package.json":"./package.json"},"gitHead":"129bade84b0098306c53bea15d191268d79eddcb","scripts":{"dev":"tsdown --config tsdown.config.ts --watch","build":"tsdown --config tsdown.config.ts"},"_npmUser":{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"},"repository":{"url":"git+https://github.com/fabiano-eduardo/cullet.git","type":"git","directory":"packages/ai-harness"},"_npmVersion":"10.8.2","description":"Provider-neutral AI agent harness: bring an API key (Anthropic, OpenAI, OpenRouter or Google), define tasks, and let the agent resolve them.","directories":{},"sideEffects":false,"_nodeVersion":"20.20.2","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/ai-harness_1.1.0_1781722706341_0.23561891616012032","host":"s3://npm-registry-packages-npm-production"}},"1.2.0":{"name":"@cullet/ai-harness","version":"1.2.0","license":"MIT","_id":"@cullet/ai-harness@1.2.0","maintainers":[{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"}],"homepage":"https://github.com/fabiano-eduardo/cullet#readme","bugs":{"url":"https://github.com/fabiano-eduardo/cullet/issues"},"dist":{"shasum":"b9a8eaade1e67240f661a86c68ca82c295b0eb8d","tarball":"https://registry.npmjs.org/@cullet/ai-harness/-/ai-harness-1.2.0.tgz","fileCount":29,"integrity":"sha512-c8gvKLKTyP8ua9nsgOfHjg4h996nK3q6jxirde5ahzSa0eWuaxF4uzoUglgTjJsyHHQiozEZ2uhtYD+2uLTaIg==","signatures":[{"sig":"MEUCIQCQ3q0Q8ivxXtU2xvXkytqsQGTDwzPDCPisZRyLbthnbgIgGsWz0PpmDaD6+UhLAOcDP+UjRQzcX3kiS/OpGdt5r9M=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@cullet%2fai-harness@1.2.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":139591},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","engines":{"node":">=18.17"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./node":{"types":"./dist/runtime/node.d.ts","import":"./dist/runtime/node.js"},"./package.json":"./package.json"},"gitHead":"7be222a4e560a5cb33075abf1cce47d6e6f3faf4","scripts":{"dev":"tsdown --config tsdown.config.ts --watch","build":"tsdown --config tsdown.config.ts"},"_npmUser":{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"},"repository":{"url":"git+https://github.com/fabiano-eduardo/cullet.git","type":"git","directory":"packages/ai-harness"},"_npmVersion":"10.8.2","description":"Provider-neutral AI agent harness: bring an API key (Anthropic, OpenAI, OpenRouter or Google), define tasks, and let the agent resolve them.","directories":{},"sideEffects":false,"_nodeVersion":"20.20.2","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/ai-harness_1.2.0_1781788671696_0.27663171245312346","host":"s3://npm-registry-packages-npm-production"}},"1.3.0":{"name":"@cullet/ai-harness","version":"1.3.0","license":"MIT","_id":"@cullet/ai-harness@1.3.0","maintainers":[{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"}],"homepage":"https://github.com/fabiano-eduardo/cullet#readme","bugs":{"url":"https://github.com/fabiano-eduardo/cullet/issues"},"dist":{"shasum":"4e264a2c4ac0f467afebfc33533feced6121e5a7","tarball":"https://registry.npmjs.org/@cullet/ai-harness/-/ai-harness-1.3.0.tgz","fileCount":29,"integrity":"sha512-OwjFDnrred7VVEdm4hUVUSTAXCK3BOTNSBXCwRle6Tz+NmyLu6nYDe6CsKE7cfo/c2kq7gwKE1GuYuXXhu1ccA==","signatures":[{"sig":"MEQCICLLwgkqkiHbhkaxYiqV+kaa2wSuZcKk00W0h1V+oR/lAiBQ+Y3YP6HmWzr1hrL5oxih8ibBZNzTTFZhnDaJd1Dhtw==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@cullet%2fai-harness@1.3.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":144086},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","engines":{"node":">=18.17"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./node":{"types":"./dist/runtime/node.d.ts","import":"./dist/runtime/node.js"},"./package.json":"./package.json"},"gitHead":"6623ffa3215c718052180cd47a80909ebf933137","scripts":{"dev":"tsdown --config tsdown.config.ts --watch","build":"tsdown --config tsdown.config.ts"},"_npmUser":{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"},"repository":{"url":"git+https://github.com/fabiano-eduardo/cullet.git","type":"git","directory":"packages/ai-harness"},"_npmVersion":"10.8.2","description":"Provider-neutral AI agent harness: bring an API key (Anthropic, OpenAI, OpenRouter or Google), define tasks, and let the agent resolve them.","directories":{},"sideEffects":false,"_nodeVersion":"20.20.2","publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/ai-harness_1.3.0_1781794760342_0.5155951684353017","host":"s3://npm-registry-packages-npm-production"}},"1.4.0":{"name":"@cullet/ai-harness","version":"1.4.0","description":"Provider-neutral AI agent harness: bring an API key (Anthropic, OpenAI, OpenRouter or Google), define tasks, and let the agent resolve them.","type":"module","repository":{"type":"git","url":"git+https://github.com/fabiano-eduardo/cullet.git","directory":"packages/ai-harness"},"main":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./node":{"types":"./dist/runtime/node.d.ts","import":"./dist/runtime/node.js"},"./package.json":"./package.json"},"scripts":{"build":"tsdown --config tsdown.config.ts","dev":"tsdown --config tsdown.config.ts --watch"},"sideEffects":false,"engines":{"node":">=18.17"},"publishConfig":{"access":"public"},"license":"MIT","_id":"@cullet/ai-harness@1.4.0","gitHead":"03e6fd8979349e41f2aa4ed428dbbc2fe74cf986","bugs":{"url":"https://github.com/fabiano-eduardo/cullet/issues"},"homepage":"https://github.com/fabiano-eduardo/cullet#readme","_nodeVersion":"20.20.2","_npmVersion":"10.8.2","dist":{"integrity":"sha512-8+J4LJDPn3dfc5gd29fm2K/JUKMEJzosgYMyjizLlvryNZcE7fdV1j1WwNJmtbqcJMXTaW5QOInXZYg6AD+KfA==","shasum":"92814b5a128cbbc9ffec8a5199ae09df389d39eb","tarball":"https://registry.npmjs.org/@cullet/ai-harness/-/ai-harness-1.4.0.tgz","fileCount":31,"unpackedSize":172974,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@cullet%2fai-harness@1.4.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIAuZ39iYB21VEQWpRsPrsE1o1JO4/a71D5K2LxWqCh17AiB0kSyhdk6Qf7r/tmDOTK9d/Q8ZkJWERqIZBJU1HGPnag=="}]},"_npmUser":{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"},"directories":{},"maintainers":[{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/ai-harness_1.4.0_1781872600737_0.9605878296330483"},"_hasShrinkwrap":false}},"time":{"created":"2026-06-17T18:58:26.210Z","modified":"2026-06-19T12:36:41.219Z","1.1.0":"2026-06-17T18:58:26.484Z","1.2.0":"2026-06-18T13:17:51.874Z","1.3.0":"2026-06-18T14:59:20.500Z","1.4.0":"2026-06-19T12:36:40.885Z"},"bugs":{"url":"https://github.com/fabiano-eduardo/cullet/issues"},"license":"MIT","homepage":"https://github.com/fabiano-eduardo/cullet#readme","repository":{"type":"git","url":"git+https://github.com/fabiano-eduardo/cullet.git","directory":"packages/ai-harness"},"description":"Provider-neutral AI agent harness: bring an API key (Anthropic, OpenAI, OpenRouter or Google), define tasks, and let the agent resolve them.","maintainers":[{"name":"fabiano-santos","email":"fabianoess.programador@gmail.com"}],"readme":"# @cullet/ai-harness\n\n**Provider-neutral AI agent harness.** Bring an API key, define a list of tasks, and let an AI agent work through them — retrying with feedback until each task is done or its limits are hit.\n\nFor the prompt-friendly summary see [`KIT_CONTEXT.md`](./KIT_CONTEXT.md). For the contracts common to every kit see the repository [`PHILOSOPHY.md`](../../PHILOSOPHY.md).\n\n---\n\n## What it delivers\n\n- **Provider adapters** (`createProvider`) for **Anthropic**, **OpenAI**, **OpenRouter** and **Google Gemini** — all over `fetch`, no vendor SDK dependency. The API key is always passed explicitly.\n- **`runHarness`** — an importable orchestration loop that selects the next runnable task, prompts the model, applies the result, optionally verifies it, and retries with feedback.\n- **Per-task provider & model.** A task can name its own `provider` + `model`; `createProviderResolver` maps those to a concrete provider (with your keys, memoized). A single `provider` still works as a default/fallback.\n- **Skills** — named, reusable instruction blocks. A task lists `skills` by name; the harness resolves them against a registry and the default prompt injects them under a `# Skills` section.\n- **Architecture-neutral by design.** The harness makes no assumption about TDD, file layout, or toolchain. You inject `apply` (what to do with the output) and, optionally, `verify` (how to check success) and `buildPrompt`.\n- **Opt-in Node helpers** under `@cullet/ai-harness/node`: write `FILE:` blocks to disk with guardrails, prompt for that format with `fileBlockPrompt`, and run shell commands (lint/typecheck/tests/build — your call) as verification sensors.\n\n## Quick start\n\n```ts\nimport { createProvider, runHarness, type Task } from \"@cullet/ai-harness\";\nimport {\n    fileBlockPrompt,\n    nodeFileWriter,\n    shellVerifier,\n} from \"@cullet/ai-harness/node\";\n\nconst provider = createProvider({\n    provider: \"openrouter\", // or \"anthropic\" | \"openai\" | \"google\"\n    apiKey: process.env.OPENROUTER_API_KEY!,\n    model: \"anthropic/claude-opus-4-8\",\n});\n\nconst tasks: Task[] = [\n    {\n        id: \"task-1\",\n        description: \"Implement the add(a, b) function in src/math.ts.\",\n    },\n];\n\nconst summary = await runHarness({\n    provider,\n    tasks,\n    // `fileBlockPrompt` asks the model for the `FILE:` format that\n    // `nodeFileWriter` applies — pair them, or the writer writes nothing.\n    buildPrompt: fileBlockPrompt,\n    apply: nodeFileWriter({ projectRoot: process.cwd() }),\n    verify: shellVerifier([\"npm run typecheck\", \"npm test\"]),\n    limits: { maxAttempts: 3, maxCostUSD: 5 },\n    onEvent: (e) => console.log(e.type),\n});\n\nconsole.log(summary); // { done, failed, pending, totalCostUSD, stoppedBy }\n```\n\n### Output contract: `FILE:` blocks (prompt ↔ writer)\n\n`nodeFileWriter` only applies output shaped as `FILE:` fenced blocks:\n\n````text\nFILE: src/math.ts\n```ts\nexport const add = (a: number, b: number) => a + b;\n```\n````\n\nThe architecture-neutral `defaultBuildPrompt` deliberately says nothing about\nthis format, so the two halves must be paired explicitly. Use **`fileBlockPrompt`**\n(from `@cullet/ai-harness/node`) as your `buildPrompt`: it wraps the default\nprompt with the `FILE:` contract that the writer consumes. Pairing one without\nthe other is the classic footgun — if the model never emits `FILE:` blocks, the\nwriter has nothing to apply. To make that failure observable rather than silent,\n`nodeFileWriter` reports a no-op via `onWrite` (`reason: \"no FILE: blocks in\noutput\"`) whenever the model returns non-empty text with no blocks. Supplying\nyour own `buildPrompt` is fine too — just keep emitting `FILE:` blocks, or swap\nin an `apply` that matches your own format.\n\nFull-control mode (copy the source into your project) is also available:\n\n```bash\nnpx cullet fc ai-harness@1.0.0\n```\n\n## Picking a provider\n\n| `provider`   | SDK          | Default base URL                           | Auth                    |\n| ------------ | ------------ | ------------------------------------------ | ----------------------- |\n| `anthropic`  | none (fetch) | `api.anthropic.com/v1`                     | `x-api-key`             |\n| `openai`     | none (fetch) | `api.openai.com/v1`                        | `Authorization: Bearer` |\n| `openrouter` | none (fetch) | `openrouter.ai/api/v1`                     | `Authorization: Bearer` |\n| `google`     | none (fetch) | `generativelanguage.googleapis.com/v1beta` | `?key=`                 |\n\n`model` is always required — the kit ships no hard-coded model id so it cannot rot. Override `baseURL`, `headers` or `fetchImpl` for gateways, proxies, or tests.\n\n## Per-task provider & model\n\nEach task can declare which vendor and model it runs on, instead of one global provider. The core stays provider-neutral and **never reads API keys** — so the string→provider bridge is a `resolveProvider` hook you supply. `createProviderResolver` is the batteries-included one: you hand it the keys once, it reads each task's `provider`/`model` and returns a memoized `AgentProvider`.\n\n```ts\nimport {\n    createProviderResolver,\n    runHarness,\n    type Task,\n} from \"@cullet/ai-harness\";\n\nconst resolveProvider = createProviderResolver({\n    anthropic: {\n        apiKey: process.env.ANTHROPIC_API_KEY!,\n        defaultModel: \"claude-opus-4-8\",\n    },\n    openai: { apiKey: process.env.OPENAI_API_KEY! },\n    defaultProvider: \"anthropic\", // for tasks that name no provider\n});\n\nconst tasks: Task[] = [\n    {\n        id: \"a\",\n        description: \"…\",\n        provider: \"anthropic\",\n        model: \"claude-opus-4-8\",\n    },\n    { id: \"b\", description: \"…\", provider: \"openai\", model: \"gpt-4o\" },\n    { id: \"c\", description: \"…\" }, // uses defaultProvider + defaultModel\n];\n\nawait runHarness({\n    resolveProvider,\n    // `provider` is still accepted as a plain fallback when you don't need a resolver.\n    tasks,\n    apply: /* … */ () => {},\n});\n```\n\nResolution precedence: vendor is `task.provider` → `defaultProvider`; model is `task.model` → `<vendor>.defaultModel` → `defaultModel`. Providers are cached by `provider|model|baseURL`. A task with no resolvable provider (no resolver and no default) fails with a clear error. The resolved provider also flows into `estimateCost(usage, provider)` and the `model-result` event (`event.provider`), so per-model pricing and logging just work.\n\n## Skills\n\nSkills are named, reusable instruction blocks. Register them on `skills` and reference them by name from each task; the default prompt renders the resolved skills under a `# Skills` section (and `fileBlockPrompt` inherits it).\n\n```ts\nawait runHarness({\n    provider,\n    skills: {\n        tdd: \"Write a failing test before the implementation.\",\n        \"sql-safe\": {\n            name: \"SQL safety\",\n            instructions:\n                \"Never build SQL by string concatenation; use parameters.\",\n        },\n    },\n    tasks: [\n        {\n            id: \"a\",\n            description: \"Implement the repository.\",\n            skills: [\"tdd\", \"sql-safe\"],\n        },\n    ],\n    apply: /* … */ () => {},\n});\n```\n\nA registry value can be a plain string (its key becomes the skill name) or a `Skill` object (`{ name?, instructions, description? }`). Referencing an unknown skill name fails fast, naming the missing skill. A custom `buildPrompt` receives the already-resolved skills via `args.skills` and can render them however it likes.\n\n## Extended thinking (reasoning)\n\nRequest reasoning/extended thinking by setting `thinking.budgetTokens` on `CompletionRequest`. The model's reasoning is returned separately in `CompletionResult.reasoning` — `text` stays the final answer only.\n\n```ts\nconst request: CompletionRequest = {\n    messages: [{ role: \"user\", content: \"Solve step by step\" }],\n    maxTokens: 16_000,\n    thinking: { budgetTokens: 10_000 },\n};\n\nconst result = await provider.complete(request);\nconsole.log(result.reasoning); // model's chain of thought\nconsole.log(result.text); // final answer\n```\n\n**Provider support:** only Anthropic implements thinking today. OpenAI, OpenRouter and Google accept the option without error but ignore it (`reasoning` is always `undefined`). When thinking is enabled on Anthropic, `temperature` is omitted (API constraint) and `maxTokens` must exceed `budgetTokens`.\n\nIn the example script, set `AI_THINKING_BUDGET=10000` to enable thinking.\n\n## Extending it\n\n- **Custom prompts**: pass `buildPrompt({ task, tasks })` to inject project rules, a system prompt, or output conventions. With `nodeFileWriter`, use `fileBlockPrompt` (or keep the `FILE:` contract in your own builder) so the model emits blocks the writer can apply.\n- **Custom apply/verify**: any function works — write files, open PRs, call your own tooling. The bundled `nodeFileWriter`/`shellVerifier` are just one convenient default.\n- **File-writer guardrails**: `nodeFileWriter` confines writes to `projectRoot` and refuses `protectedPatterns` (a denylist) out of the box. Tighten the blast radius further with `allowedPatterns` (deny-by-default — only matching paths are written, e.g. `[/^src\\//]`), and preview a run with `dryRun: true` (reports each intended write via `onWrite` without touching disk).\n- **Cost cap**: pass `estimateCost(usage, provider)` to translate token usage into USD and stop at `limits.maxCostUSD`. Setting `maxCostUSD` **without** `estimateCost` throws at entry — an unestimated cap prices every call at 0 and would never trip, so the harness refuses to run unbounded.\n- **Cancellation**: pass an `AbortSignal` via `signal`.\n\n## Git checkpointing (opt-in)\n\n`createGitCheckpoint` (from the `./node` subpath) commits each passing task and rolls back failed attempts, so a run produces clean, bisectable history and a failed attempt never leaks into the next one. Wire it through `verify`:\n\n```ts\nimport {\n    createGitCheckpoint,\n    nodeFileWriter,\n    shellVerifier,\n} from \"@cullet/ai-harness/node\";\n\nconst checkpoint = createGitCheckpoint({ cwd: projectRoot });\n\nawait runHarness({\n    provider,\n    tasks,\n    apply: nodeFileWriter({ projectRoot }),\n    verify: checkpoint.wrapVerify(\n        shellVerifier([\"npm run typecheck\", \"npm test\"]),\n    ),\n});\n```\n\n> Run this on a clean, dedicated branch: rollback is `git reset --hard` + `git clean -fd`, which discards **all** uncommitted changes in `cwd`. As a safety net, `createGitCheckpoint` refuses to start if the tree is already dirty (it would otherwise reset over your work); pass `requireCleanStart: false` to opt into running dirty on purpose.\n\n## Live smoke tests (opt-in)\n\nThe unit suite stubs `fetch`, so it never exercises a provider's real wire format. [`src/providers/live.spec.ts`](./src/providers/live.spec.ts) closes that gap with a smoke test per provider — but it is **skipped** unless `AI_HARNESS_LIVE=1` and that provider's key + model env vars are set, so CI never runs it and never spends a token. Run it by hand before a release, against each provider you actually ship:\n\n```bash\nAI_HARNESS_LIVE=1 \\\n  ANTHROPIC_API_KEY=sk-... ANTHROPIC_MODEL=claude-opus-4-8 \\\n  npm run test:live\n```\n\nSet the matching `<PROVIDER>_API_KEY` + `<PROVIDER>_MODEL` pair (`ANTHROPIC`, `OPENAI`, `OPENROUTER`, `GOOGLE`) for each provider you want covered; any pair you omit is skipped.\n\n## Examples\n\nA complete, runnable script lives in [`examples/run.ts`](./examples/run.ts) (loads [`examples/tasks.json`](./examples/tasks.json)):\n\n```bash\nnpm run build -w @cullet/ai-harness\nAI_PROVIDER=anthropic AI_MODEL=claude-opus-4-8 AI_API_KEY=sk-... \\\n  npx tsx packages/ai-harness/examples/run.ts\n```\n","readmeFilename":"README.md"}