![\"SpecPi](\"https://tannermidd.github.io/SpecPi/logo.svg\"){kind=logo}
\n\n \n
A Pi harness setup, built to specification.
\n\n\n ![\"npm](\"https://img.shields.io/npm/v/specpi?style=flat-square&color=084bdb\")
![\"Build](\"https://github.com/TannerMidd/SpecPi/actions/workflows/ci.yml/badge.svg?branch=main\"){kind=icon}
\n ![\"MIT](\"https://img.shields.io/badge/license-MIT-084bdb?style=flat-square\"){kind=icon}
\n Website · Documentation · Evaluations · Releases\n
\n\n\n \n ![\"SpecPi](\"https://tannermidd.github.io/SpecPi/media/specpi-chat-showcase-dark.png\")
\n
SpecPi Chat · Example workspace
\n\n---\n\nSpecPi 0.36.4 is a small starting point for the [Pi coding agent](https://pi.dev/). It is one opinionated setup for how the agent should work, not a marketplace of plugins.\n\nAt the center are two built-in extensions. **Scope control** keeps each task to the files it said it would touch. The **improvement loop** turns repeated friction into small, tested changes to the setup, instead of letting prompts and workarounds pile up. Around those are eight hand-picked packages, each locked to an exact version and checked before anything installs, plus **SpecPi Chat**, a VS Code panel for working alongside the agent.\n\nIt focuses on five things:\n\n- **Control:** clear scope, tool permissions, and lifecycle commands that ask before they change anything\n- **Accuracy:** exact version pins, checksums on state, rollback on failure, and proof over promises\n- **Improvement:** local notes become small, checked changes through `/harness-improvement`\n- **Efficiency:** delegation, persistent goals, and browser QA handled by the right tool for the job\n- **Keep talking:** long evals, builds and servers can run as background jobs, so the conversation is not stuck until they finish. The agent hears back when each one ends, and `/jobs` lists or stops them\n- **Lean default:** web access, browser QA, and delegation stay off until you need them. Turn them on for a session with `/webaccess on`, `/browser on`, and `/delegate on` — or let the agent ask when it hits the need, and answer the prompt\n\nEverything it touches is written down, versioned, and easy to undo.\n\n## A local command guard\n\nBefore the agent runs a shell command, [LANCET Nano](https://github.com/TannerMidd/LANCET-model), a 111M-parameter classifier, can check it on your own CPU. It reads Bash, PowerShell and cmd, takes about 10 ms for a typical command, needs no API key, sends nothing over the network, and costs nothing per check. Fixed rules settle the obvious cases first, blocking `rm -rf /` and passing `git status`, and the model scores the rest. A risky or uncertain verdict asks you before anything runs; `/lancet-guard mode block` blocks risky ones outright instead.\n\n| | LANCET Nano v0.4.3 |\n| --- | ---: |\n| Triage Score | 68.3 |\n| Safe commands wrongly stopped | 9.0% |\n| Cost per check | $0 |\n| Network | none |\n\nBoth figures are taken across three benchmarks, 7,911 commands in total, weighted by size: lancet-bench-2-next (1,602 risky commands and their safe look-alikes), the ShellRisk-Bench test split, and a 513-command neutral set. The [Triage Score](https://github.com/TannerMidd/LANCET-model) gives a guard a point for each risky command it asks about or blocks, and shrinks in proportion once it stops more than 10% of safe commands. The guard ships off. Run `/lancet-guard setup` in Pi once to download the model (about 109 MB, verified by SHA-256) and switch it on. [How it works, what it costs, and its limits](https://tannermidd.github.io/SpecPi/lancet/).\n\n## Measured context\n\nThis chart shows first-call context from a clean install: all eight pinned packages, the working agreement, and the skills Pi finds, in an interactive session with wishlist collection undecided. A headless session, or one with collection off, also leaves out the gap report and capability request tools. \"Enabled\" means browser QA, delegation, and web access are switched on, with no goal, scope, or improvement selection active.\n\nThe solid rows are measured by us, from the request each setup actually sends through one local test provider. That includes OpenCode, the DeepSeek Harness, and Oh My Pi, all measured as installed. The faded Codex CLI and Claude Code rows come from HarnessTax's published numbers, measured under their own setup. Treat those as a rough reference, not a head-to-head test. These are character counts. They say nothing about tokens, cost, or how well each tool does the job. The research page [breaks down the enabled setup by feature](https://tannermidd.github.io/SpecPi/research/#specpi), so you can see what each switch costs on its own.\n\n\n \n ![\"Bar](\"https://tannermidd.github.io/SpecPi/media/context-chart-dark.svg\")
\n
Measured tool schemas + system/developer instructions · node scripts/measure-context.mjs --chart --omp=<path to Oh My Pi's cli.js> --oc=<path to OpenCode's binary> --dsh=<path to the DeepSeek Harness bin> · Recorded measurements and package pins · Method and caveats
\n \n ![\"Bar](\"https://tannermidd.github.io/SpecPi/media/capability-chart-dark.svg\")
\n
Every tool in the measured request belongs to exactly one group · Leaving all three opt-in groups hidden keeps 23,797 characters of tool schema out of every request
\n\n## Harness evaluations\n\nThe chart above counts characters. It says nothing about what a harness costs to\nactually use, or whether it finishes the job. That is what the evals are for:\n[Terminal-Bench 2.0](https://www.tbench.ai/), the same model and the same frozen\nprice list, with only the harness changing.\n\n\n\n**947 scored attempts on two benchmarks and 6 harnesses**,\nall on `deepseek-v4.1-flash`. SpecPi is the published 0.33.0 release, with the experimental Jev layer off.\n\nSWE-bench Verified, 12 tasks × 3: SpecPi solved 34/36 against Pi's 34/36\n(p = 1.00), with 2% more prompt tokens and 1% less cost per attempt.\n\nTerminal-Bench 2.0, 731 attempts across 20 tasks:\n\n| Harness | Solved | Rate | Cost/attempt | Prompt tokens | Cache hit |\n| --- | --- | --- | --- | --- | --- |\n| OpenCode | 29/39 | 0.744 | $0.0124 | 494,173 | 96.5% |\n| SpecPi | 62/77 | 0.805 | $0.0144 | 407,416 | 92.4% |\n| Pi (base) | 164/230 | 0.713 | $0.0155 | 436,214 | 92.9% |\n| DeepSeek Harness | 21/38 | 0.553 | $0.0226 | 1,126,586 | 95.4% |\n| Oh My Pi | 115/151 | 0.762 | $0.0237 | 1,108,076 | 96.5% |\n| Claude Code | 81/112 | 0.723 | $0.0273 | 670,828 | 95.4% |\n\nSpecPi and Pi ran side by side in the 24 Sep · a sitting. SpecPi solved 30/38\nagainst Pi's 25/39 (Fisher p = 0.21), sending 16% fewer prompt tokens and costing\n22% less per attempt. On `sanitize-git-repo`, with that sitting's extra attempts, SpecPi solved\n10/10 against 3/10 (p = 0.003); `fix-git` was 10/10 for both.\n\nOverall solve rate is a different matter: one sitting cannot rank harnesses here. Bare Pi, on\nunchanged software and the same thirteen tasks, spans 56-77% across 5 sittings, a wider gap than any\nmeasured between two harnesses. Pooled across sittings, SpecPi leads DeepSeek Harness (p = 0.007) and Oh My Pi leads DeepSeek Harness (p = 0.015), but pooling sets one harness's sittings against another's. Cost is recomputed from recorded tokens against a dated\nprice file, never taken from a harness's self-report.\n\n\n\nThis run is still in progress. See the results and brief method on the\n[evaluations page](https://tannermidd.github.io/SpecPi/evaluations/). The table above is regenerated\nfrom the run data by `node scripts/tb2-site.mjs`, so it cannot drift from the\npublished figures.\n\n## Install\n\nRequires Node.js 22.19+, Git, npm, and an existing Pi installation on PATH.\n\n```sh\nnpm install --global specpi@latest\nspecpi plan\nspecpi install\nspecpi doctor\n```\n\n`plan` shows what will change without modifying anything. Restart Pi after install.\n\nFull setup options, package details, and requirements: [website](https://tannermidd.github.io/SpecPi/#install).\n\n## Where things live\n\n| | |\n| --- | --- |\n| [Packages](https://tannermidd.github.io/SpecPi/#packages) | The eight pinned packages and what each provides |\n| [Scope control](https://tannermidd.github.io/SpecPi/wiki/#scope) | `/scope` commands and drift monitoring |\n| [Command guard](https://tannermidd.github.io/SpecPi/lancet/) | LANCET, the local classifier that checks shell commands before they run |\n| [Improvement loop](https://tannermidd.github.io/SpecPi/#loop) | Local wishlist, `/harness-improvement`, and retirement with evidence |\n| [SpecPi Chat](https://tannermidd.github.io/SpecPi/#vscode-chat) | VS Code frontend and VSIX install · [Chat guide](https://github.com/TannerMidd/SpecPi/blob/main/vscode/README.md) |\n| [Updating](https://tannermidd.github.io/SpecPi/#updating) | Update, uninstall, and migration notes |\n\n## Development\n\n```sh\nnpm install --ignore-scripts --omit=peer --no-package-lock\nnode --test tests/workflow-controls.test.mjs tests/workflow-controls-extension.test.mjs\nnpm run check\n```\n\nInstaller tests use disposable Pi directories — never test against a live Pi installation. Publication follows the [release procedure](NPM_RELEASE.md).\n\n[Security model](SECURITY_MODEL.md) · [Third-party components](THIRD_PARTY.md) · [Release notes](CHANGELOG.md) · [MIT License](LICENSE)\n","readmeFilename":"README.md"}