{"_id":"@arhen/pi-core-vision","_rev":"7-ec6df9c73178ef30f7cdfa7b3ac41393","name":"@arhen/pi-core-vision","dist-tags":{"latest":"1.0.7"},"versions":{"1.0.0":{"name":"@arhen/pi-core-vision","version":"1.0.0","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.0","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-vision#readme","bugs":{"url":"https://github.com/arhen/pi-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"d9fd971c0f2756c485a9e4a7edebc66464beeb4c","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.0.tgz","fileCount":6,"integrity":"sha512-IgTq1HAZ5+ADwD6JcGGG71T3hUhHtcLhO+sjKysyUTzjhQuqpVqPCPba5DX6WsQ8yzROrSRoXU7JXmsyU7fjFw==","signatures":[{"sig":"MEQCIBOOicg4UeBNYJ2daMP4f8s0tqx9aKBBhNC6+eGyg++DAiBb7AWu2IydNrdJ0CbI5FokN8Yl38Qz0Hq4YcBz44Vf5g==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":40001},"type":"module","gitHead":"41329d74cf8907e94a1f759a0e2e7ac0b3b5090a","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-vision.git","type":"git"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.0_1786858690684_0.6363719112462338","host":"s3://npm-registry-packages-npm-production"}},"1.0.1":{"name":"@arhen/pi-core-vision","version":"1.0.1","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.1","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"0f1cc6f5ee9b5654f400c29781b5c5d8a5188bd9","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.1.tgz","fileCount":6,"integrity":"sha512-ixiFJRftnEVetxBw9B4NFPU5k+BW+vYBRrefRWgQQhveGbzQ0R2ff6SuuY/SYx3JFe0IzYvKx2vQLvEDL24y8A==","signatures":[{"sig":"MEQCICoY4CofOQgd3vRq/c5qsfsN0+k9m37B++m0iv2lJf6bAiBgwqEw5K/wqs/uu0BymEDNIxonEq/z7uvKbXM8Phchag==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":40147},"type":"module","gitHead":"2ac5f960f65a4ad618eb83c251f4a1ace8b248a0","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-core-vision.git","type":"git"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.1_1786867004999_0.667945007469912","host":"s3://npm-registry-packages-npm-production"}},"1.0.3":{"name":"@arhen/pi-core-vision","version":"1.0.3","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.3","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"cbf455123fdc0e17a9b44a2e967feb16e6308945","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.3.tgz","fileCount":6,"integrity":"sha512-xG2JEi+/+RmVYvy9Qcsi8qE0h9MWxcwrj33ZWk/lX93q3M49hLJ7Pg3/NpeIhZ1niFwT3rx5RLAaLfKH782pyg==","signatures":[{"sig":"MEYCIQDKikS+YtKuh031J/mIvKbzPYeklYOXlCRZ7/m9HT9oygIhAJMrqsGrWDauNOuQOhLLdSMe5V+vKfudofGXZm1+Ra+X","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":40690},"type":"module","gitHead":"87fd93d1b91785e93493cef8dafb86226d7f8be1","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-core-vision.git","type":"git"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.3_1786867624814_0.8326900687525536","host":"s3://npm-registry-packages-npm-production"}},"1.0.4":{"name":"@arhen/pi-core-vision","version":"1.0.4","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.4","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"a22ac4ae538eee4981c138cfd424118fde9bed43","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.4.tgz","fileCount":6,"integrity":"sha512-VzLSMY8O90rxIVNsU5f5WAF2G0EHJtI6yVAT21wPMinD9ZmFii9BCAfRKWtcma7vQ3fArVntqo3bg2Z7Bxv4sw==","signatures":[{"sig":"MEUCIQDlwZ0xarWSSRaHPCULZuM7tK0CqDnjfp07XFGJwB9dVgIgOILqstp7zHpofUpsYt/Rdw1xbnz8YLKt7Xw9MC4AmKk=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":41101},"type":"module","gitHead":"2af7c007e27e8e0d24fda6660db326802ae0966b","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-core-vision.git","type":"git"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.4_1786868975980_0.1579548954039125","host":"s3://npm-registry-packages-npm-production"}},"1.0.5":{"name":"@arhen/pi-core-vision","version":"1.0.5","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.5","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"2dd1bf7d38d12f467a57ae1aa728dc6e85c0eff4","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.5.tgz","fileCount":6,"integrity":"sha512-BisEEkq9iP7xIL0vUBt1g9dDzkTLbVvxIofHgZpMcykusHFaXSm/veiNMRncOXEkyisjcR0DgLMOtzTVjEgrUw==","signatures":[{"sig":"MEUCIQDex7uW/tkWK2CuYQQ8x1Qt3+wuObHq7qY0gMysRLDeGgIga8RIqPnWdDXTNKhTQMsePlegYf66S7gkyJceokHLNJ0=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":43992},"type":"module","gitHead":"416198e2d68a62f9a51c77fec208502bf8aeae06","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-core-vision.git","type":"git"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.5_1786890196656_0.5674582119363174","host":"s3://npm-registry-packages-npm-production"}},"1.0.6":{"name":"@arhen/pi-core-vision","version":"1.0.6","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"author":{"name":"arhen"},"license":"MIT","_id":"@arhen/pi-core-vision@1.0.6","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"03f819026e7fdbfe4fd4645439beb9ef121fb833","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.6.tgz","fileCount":6,"integrity":"sha512-kq5bt0Ydz72mt3wQ4Csv8gLYPrt4UUw2KeVxOof+zUnVZsPtp7qnchUYj8w1j2DweK7kfEcmBkpnUtLf803d8Q==","signatures":[{"sig":"MEYCIQD49qWiaaTENqlTXzHim2GvYB+MXC+dvh56CYW/IKN8/QIhANLfLrPM2qbj3+oUItZRUGcaDPt7WcSMOxoPmBU/aYXP","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":44170},"type":"module","gitHead":"5cad6a5943bb5ca20a7774f4036ab5028822cc4d","scripts":{"prepublishOnly":"bun src/self-check.ts"},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"repository":{"url":"git+https://github.com/arhen/pi-extensions.git","type":"git","directory":"packages/core/pi-core-vision"},"_npmVersion":"10.9.2","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","directories":{},"_nodeVersion":"23.8.0","publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-core-vision_1.0.6_1786950656454_0.17205113082377776","host":"s3://npm-registry-packages-npm-production"}},"1.0.7":{"name":"@arhen/pi-core-vision","version":"1.0.7","description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","license":"MIT","author":{"name":"arhen"},"type":"module","engines":{"node":">=20"},"repository":{"type":"git","url":"git+https://github.com/arhen/pi-extensions.git","directory":"packages/core/pi-core-vision"},"homepage":"https://github.com/arhen/pi-core-vision#readme","bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"peerDependencies":{"@earendil-works/pi-coding-agent":"^0.84.2","typebox":"^1.3.14"},"pi":{"extensions":["./index.ts"]},"scripts":{"prepublishOnly":"bun src/self-check.ts"},"publishConfig":{"access":"public","registry":"https://registry.npmjs.org/"},"_id":"@arhen/pi-core-vision@1.0.7","gitHead":"ac007b990447e17a3e10b797ac4dee0a7c71386f","_nodeVersion":"23.8.0","_npmVersion":"10.9.2","dist":{"integrity":"sha512-5EZUCarfxHtqMmTOPU7YJh41tu/1s3UGRzbrfJZ0BAkC4qiRRyJTyWFM6JywWvhTLaEVApeAi9W77xG6knSaMw==","shasum":"1b2afe24f6ec4592f45895a411a5ea7fa693a322","tarball":"https://registry.npmjs.org/@arhen/pi-core-vision/-/pi-core-vision-1.0.7.tgz","fileCount":6,"unpackedSize":44702,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIHremeGYIXPBzR800wWpbpBhOy+t58dxJSjs5HUnVd41AiEArR4EO9ZuYarokTFlgAl7ipUOFnyijm6xq2mdNDbxRng="}]},"_npmUser":{"name":"arhen","email":"itsme.arhen@gmail.com"},"directories":{},"maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/pi-core-vision_1.0.7_1787383407670_0.21210730077103768"},"_hasShrinkwrap":false}},"time":{"created":"2026-08-16T05:38:10.544Z","modified":"2026-08-22T07:23:27.973Z","1.0.0":"2026-08-16T05:38:10.841Z","1.0.1":"2026-08-16T07:56:45.165Z","1.0.3":"2026-08-16T08:07:04.949Z","1.0.4":"2026-08-16T08:29:36.126Z","1.0.5":"2026-08-16T14:23:16.818Z","1.0.6":"2026-08-17T07:10:56.606Z","1.0.7":"2026-08-22T07:23:27.824Z"},"bugs":{"url":"https://github.com/arhen/pi-core-vision/issues"},"author":{"name":"arhen"},"license":"MIT","homepage":"https://github.com/arhen/pi-core-vision#readme","keywords":["pi-package","pi","pi-extension","vision","multimodal","image","fallback","read"],"repository":{"type":"git","url":"git+https://github.com/arhen/pi-extensions.git","directory":"packages/core/pi-core-vision"},"description":"Transparent vision fallback for text-only pi models — overrides read so image files are described by a vision model when the active model cannot see images. Supports any OpenAI-compatible endpoint.","maintainers":[{"name":"arhen","email":"itsme.arhen@gmail.com"}],"readme":"# @arhen/pi-core-vision\n\n[![npm version](https://img.shields.io/npm/v/@arhen%2Fpi-core-vision?color=cb3837&logo=npm&label=npm)](https://www.npmjs.com/package/@arhen/pi-core-vision)\n[![npm downloads](https://img.shields.io/npm/dm/@arhen%2Fpi-core-vision)](https://www.npmjs.com/package/@arhen/pi-core-vision)\n[![pi extension](https://img.shields.io/badge/pi-extension-blueviolet)](https://github.com/earendil-works/pi-coding-agent)\n[![license](https://img.shields.io/github/license/arhen/pi-core-vision)](LICENSE)\n\n## Install\n\nRequires the [pi coding agent](https://github.com/earendil-works/pi) — install it first: `npm install -g @earendil-works/pi-coding-agent`.\n\n```sh\npi install git:github.com/arhen/pi-core-vision\n```\n\nor try without installing:\n\n```sh\npi -e git:github.com/arhen/pi-core-vision\n```\n\nTransparent vision fallback for text-only models in [pi](https://pi.dev).\n\nOverrides the built-in `read` tool:\n\n- **text file** → built-in behavior, untouched\n- **image + active model sees images** → built-in behavior, untouched (pi's own resize + native attach)\n- **image + text-only model** → pi resizes the image (Photon WASM), then the extension sends pi's resized output to a vision model and returns a compact text description\n\nThe model sees one result either way — no double reading, no new tool to learn. Text-only models (e.g. DeepSeek) can finally read screenshots, diagrams, and error messages.\n\n## Configure\n\nTwo modes — **raw** (any OpenAI-compatible endpoint) or **registry** (models from pi's own registry, auth via `auth.json`/`/login`/env).\n\nSet the vision model via `/pi-vision` command, env vars, or `~/.pi/pi-vision.json` (JSON wins over env).\n\n```bash\n/pi-vision set baseUrl=https://api.openai.com/v1 apiKey=sk-... model=gpt-4o-mini\n/pi-vision show          # current config (apiKey masked)\n/pi-vision reset         # clear config file\n```\n\nEnv vars:\n\n```bash\nexport PI_VISION_BASE_URL=\"https://api.openai.com/v1\"\nexport PI_VISION_API_KEY=\"sk-...\"\nexport PI_VISION_MODEL=\"gpt-4o-mini\"\n```\n\n`~/.pi/pi-vision.json` (extra options: `prompt`, `maxTokens`):\n\n```json\n{\n  \"baseUrl\": \"https://api.openai.com/v1\",\n  \"apiKey\": \"sk-...\",\n  \"model\": \"gpt-4o-mini\",\n  \"maxTokens\": 1500\n}\n```\n\n### Registry mode\n\nUse any model pi already knows — no duplicated credentials. The vision model must declare `\"input\": [\"text\", \"image\"]` in `models.json`, and auth resolves through pi's normal channels (stored credential in `auth.json`, `/login`, or provider `apiKey`).\n\n```json\n{\n  \"provider\": \"anthropic\",\n  \"model\": \"claude-sonnet-4-5\",\n  \"maxTokens\": 1500\n}\n```\n\n```bash\n/pi-vision set provider=anthropic model=claude-sonnet-4-5\n/pi-vision set provider=kitchen model=gemma-4-26b-a4b-it\n```\n\nOpenAI-compatible providers (openai-completions) are called through the extension's own transport (retry, SSE-safe, cache); other APIs (anthropic-messages, google-generative-ai, custom) go through pi's provider machinery.\n\nAny OpenAI-compatible endpoint works: OpenAI `/v1`, Google Gemini `/v1beta/openai`, Alibaba DashScope `/compatible-mode/v1`, Ollama `/v1`, LM Studio, vLLM. If your gateway streams SSE by default, the extension forces `stream: false`.\n\n## How it works\n\n- Delegates every read to pi's own `createReadToolDefinition` — byte-identical built-in behavior (Photon resize to 2000px / 4.5MB, magic-byte mime detection, truncation).\n- Checks `ctx.model.input.includes(\"image\")` at call time. Vision-capable model → built-in result untouched. Text-only model + image → vision model describes pi's already-resized base64.\n- Nested vision usage is reported back, so pi session stats stay accurate.\n- Raw-file fallback (with 20MB guard) covers the case where pi's image processing is unavailable (e.g. BMP).\n\n## Benchmark\n\nSingle-run comparison vs community alternatives, text-only parent model (deepseek-v4-flash), same screenshot through the same kitchen gateway. See also [pi-vision-handoff](https://github.com/monotykamary/pi-vision-handoff) and [pi-sense](https://github.com/ssdiwu/pi-sense).\n\n### Healthy vision model (gemma-4-26b via kitchen)\n\n| tool | e2e | flow | precision |\n|---|---|---|---|\n| **pi-vision (this)** | **32s** | clean 1 read → description in result | **6/6 facts** (title, theme, tabs, chat, layout) |\n| pi-vision-handoff | 44s | clean 1 read → context swap | 5/6 |\n| pi-sense | 122s | description in result | ~5/6 |\n\n### Vision API down (kitchen haiku 429 for the whole window)\n\n| tool | result |\n|---|---|\n| **pi-vision (pre-fix)** | 200s+ hang — threw on 429 → parent retry-looped the read |\n| **pi-vision (post-fix)** | **19s** — graceful `[image: description unavailable]` → model moves on |\n| pi-vision-handoff | 42s — graceful placeholder → OCR fallback |\n| pi-sense | 27s — graceful placeholder → OCR fallback |\n\nFindings:\n- Fastest end-to-end when vision is healthy, and the only tool that returns the full description inside the read result.\n- Vision failures degrade gracefully (placeholder text, no hang) — shipped after the 429 incident above.\n- Descriptions are framed as UNTRUSTED DATA (prompt-injection mitigation).\n\nCaveats: single run per cell; gateway routing flakiness affects variance.\n\n\n## Development\n\n```bash\nbun src/self-check.ts    # logic self-checks (no pi needed, no API calls)\n```\n\n## License\n\nMIT\n","readmeFilename":"README.md"}