{"_id":"@boole/boole","_rev":"5-84eae405eadef1153152ce1e7334a43b","name":"@boole/boole","dist-tags":{"latest":"0.2.0"},"versions":{"0.0.0":{"name":"@boole/boole","version":"0.0.0","keywords":["llm","inference","local-first","llama.cpp","gguf","ai","machine-learning"],"license":"Apache-2.0","_id":"@boole/boole@0.0.0","maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"homepage":"https://github.com/yourusername/boole#readme","bugs":{"url":"https://github.com/yourusername/boole/issues"},"dist":{"shasum":"1904cf0e6fbfece70a43c4f9a4a6321efdcdf0ee","tarball":"https://registry.npmjs.org/@boole/boole/-/boole-0.0.0.tgz","fileCount":9,"integrity":"sha512-YXvlsZSZi1iHxMNYPHAxeN4fyKxO1HhRAtWcYG5icEcjB0YgEXJIisQqqR8NI5tbXjaqlq75vf7jGuZqncw1pg==","signatures":[{"sig":"MEUCIB+heyS/LMtICr6wujNolnsmFMsNEa4sQ7BhS9QYIDKuAiEAj1KdufiMA9xdPGxmu9Il9JxisQhQuyZ1MavM8RFyaJ8=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":124489},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.mjs","engines":{"node":">=20"},"exports":{".":{"import":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"gitHead":"fe9c88b82c69adf3b271cb0737210d26fb79c4c8","scripts":{"lint":"eslint src","test":"vitest run","build":"tsup","format":"prettier --write \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","release":"pnpm build && changeset publish","version":"changeset version","changeset":"changeset","typecheck":"tsc --noEmit","test:watch":"vitest","format:check":"prettier --check \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\""},"_npmUser":{"name":"jordan.plows","email":"plowstjordan@gmail.com"},"repository":{"url":"git+https://github.com/yourusername/boole.git","type":"git"},"_npmVersion":"10.8.2","description":"Local-first LLM inference SDK with optional remote burst scaling","directories":{},"_nodeVersion":"20.17.0","dependencies":{"node-llama-cpp":"^3.1.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsup":"^8.3.5","eslint":"^9.15.0","vitest":"^2.1.4","prettier":"^3.3.3","@eslint/js":"^9.39.5","typescript":"^5.6.3","@types/node":"^20.17.6","@changesets/cli":"^2.27.9","@typescript-eslint/parser":"^8.13.0","@typescript-eslint/eslint-plugin":"^8.13.0"},"_npmOperationalInternal":{"tmp":"tmp/boole_0.0.0_1786427646678_0.08557189871842419","host":"s3://npm-registry-packages-npm-production"}},"0.0.1":{"name":"@boole/boole","version":"0.0.1","keywords":["llm","inference","local-first","llama.cpp","gguf","ai","machine-learning"],"license":"Apache-2.0","_id":"@boole/boole@0.0.1","maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"homepage":"https://github.com/boole-ai/boole-npm#readme","bugs":{"url":"https://github.com/boole-ai/boole-npm/issues"},"dist":{"shasum":"573936ed0f6cdf57ad360c04d7a183ae13bd00cd","tarball":"https://registry.npmjs.org/@boole/boole/-/boole-0.0.1.tgz","fileCount":9,"integrity":"sha512-dWUFEwo/JcCQMaAKc/8E7Z+Q0li5pY/EtcYku7HLzohY0kjeqIWWj+EWjSGxpilwWKivVnj3nwBHvBXVo+JvVQ==","signatures":[{"sig":"MEUCIEFAJY18H/dCKwiFemZwrioXx1l6LOwPnUHY85CPztDsAiEA/bEhVfNp2sBVxVjGqvwduME0zCdWaFJnGFYmUEIKZOQ=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":124357},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.mjs","engines":{"node":">=20"},"exports":{".":{"import":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"gitHead":"1cc4c4b38ca40fa736cdc7d68cf047d7e5134ced","scripts":{"lint":"eslint src","test":"vitest run","build":"tsup","format":"prettier --write \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","release":"pnpm build && changeset publish","version":"changeset version","changeset":"changeset","typecheck":"tsc --noEmit","test:watch":"vitest","format:check":"prettier --check \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\""},"_npmUser":{"name":"jordan.plows","email":"plowstjordan@gmail.com"},"repository":{"url":"git+https://github.com/boole-ai/boole-npm.git","type":"git"},"_npmVersion":"10.8.2","description":"Local-first LLM inference SDK with optional remote burst scaling","directories":{},"_nodeVersion":"20.17.0","dependencies":{"node-llama-cpp":"^3.1.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsup":"^8.3.5","eslint":"^9.15.0","vitest":"^2.1.4","prettier":"^3.3.3","@eslint/js":"^9.39.5","typescript":"^5.6.3","@types/node":"^20.17.6","@changesets/cli":"^2.27.9","@typescript-eslint/parser":"^8.13.0","@typescript-eslint/eslint-plugin":"^8.13.0"},"_npmOperationalInternal":{"tmp":"tmp/boole_0.0.1_1786427932057_0.6114904798252363","host":"s3://npm-registry-packages-npm-production"}},"0.0.2":{"name":"@boole/boole","version":"0.0.2","keywords":["llm","inference","local-first","llama.cpp","gguf","ai","machine-learning"],"license":"Apache-2.0","_id":"@boole/boole@0.0.2","maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"homepage":"https://github.com/boole-ai/boole-npm#readme","bugs":{"url":"https://github.com/boole-ai/boole-npm/issues"},"dist":{"shasum":"fd301ef94ec7779a4ffd93caa7bc4b9334ce1768","tarball":"https://registry.npmjs.org/@boole/boole/-/boole-0.0.2.tgz","fileCount":9,"integrity":"sha512-GZuGZVKGZ6dnF6DI5vgr2ZycTXWX9mpABuQq8TjWwFcYksoBAH+dFZ4nXREETrpbuV+TIkFibTofuUCuCYoYzg==","signatures":[{"sig":"MEUCIDSMLMqFB3CdYPV5LoUt4PykNUxX32k7rvbRGYvGEX8oAiEA/zbU2nbD3DhFNbjJDjOcZbTMNNRW1Wfjy6A3libbi2w=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":124268},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.mjs","engines":{"node":">=20"},"exports":{".":{"import":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"gitHead":"5928d847d146c5d6835ec1529961e6133acc6faa","scripts":{"lint":"eslint src","test":"vitest run","build":"tsup","format":"prettier --write \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","release":"pnpm build && changeset publish","version":"changeset version","changeset":"changeset","typecheck":"tsc --noEmit","test:watch":"vitest","format:check":"prettier --check \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\""},"_npmUser":{"name":"jordan.plows","email":"plowstjordan@gmail.com"},"repository":{"url":"git+https://github.com/boole-ai/boole-npm.git","type":"git"},"_npmVersion":"10.8.2","description":"Local-first LLM inference SDK with optional remote burst scaling","directories":{},"_nodeVersion":"20.17.0","dependencies":{"node-llama-cpp":"^3.1.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsup":"^8.3.5","eslint":"^9.15.0","vitest":"^2.1.4","prettier":"^3.3.3","@eslint/js":"^9.39.5","typescript":"^5.6.3","@types/node":"^20.17.6","@changesets/cli":"^2.27.9","@typescript-eslint/parser":"^8.13.0","@typescript-eslint/eslint-plugin":"^8.13.0"},"_npmOperationalInternal":{"tmp":"tmp/boole_0.0.2_1786428170404_0.7963733187386417","host":"s3://npm-registry-packages-npm-production"}},"0.0.3":{"name":"@boole/boole","version":"0.0.3","keywords":["llm","inference","local-first","llama.cpp","gguf","ai","machine-learning"],"license":"Apache-2.0","_id":"@boole/boole@0.0.3","maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"homepage":"https://github.com/boole-ai/boole-npm#readme","bugs":{"url":"https://github.com/boole-ai/boole-npm/issues"},"dist":{"shasum":"a577a8ff2e880e22741fe63b9c31fd229b235771","tarball":"https://registry.npmjs.org/@boole/boole/-/boole-0.0.3.tgz","fileCount":9,"integrity":"sha512-v5YthHjd84744dvGH28RA/WN5pZ15BNCY5nenFTfiLQX08e8k6HnqkjcBktXYGx28mciZcQiOLCNQHVgNne5hg==","signatures":[{"sig":"MEUCIQCWVs4zGjOsDb8upIO1sb0pZ1RAn7glR3x9O7Fygg4MPwIgCtAt+RUNEqtVdlUYNfg/JlLhKEIQCVuqHtWetzfOKBk=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":124136},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.mjs","engines":{"node":">=20"},"exports":{".":{"import":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"gitHead":"16e98ef6d6ddfa247e33ca5fe493be03c4c42be2","scripts":{"lint":"eslint src","test":"vitest run","build":"tsup","format":"prettier --write \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","release":"pnpm build && changeset publish","version":"changeset version","changeset":"changeset","typecheck":"tsc --noEmit","test:watch":"vitest","format:check":"prettier --check \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\""},"_npmUser":{"name":"jordan.plows","email":"plowstjordan@gmail.com"},"repository":{"url":"git+https://github.com/boole-ai/boole-npm.git","type":"git"},"_npmVersion":"10.8.2","description":"Local-first LLM inference SDK with optional remote burst scaling","directories":{},"_nodeVersion":"20.17.0","dependencies":{"node-llama-cpp":"^3.1.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsup":"^8.3.5","eslint":"^9.15.0","vitest":"^2.1.4","prettier":"^3.3.3","@eslint/js":"^9.39.5","typescript":"^5.6.3","@types/node":"^20.17.6","@changesets/cli":"^2.27.9","@typescript-eslint/parser":"^8.13.0","@typescript-eslint/eslint-plugin":"^8.13.0"},"_npmOperationalInternal":{"tmp":"tmp/boole_0.0.3_1786428480877_0.7014310063243574","host":"s3://npm-registry-packages-npm-production"}},"0.2.0":{"name":"@boole/boole","version":"0.2.0","description":"Local-first LLM inference SDK for JavaScript & TypeScript. Run GGUF models on your own hardware via llama.cpp, with an App/Function/Sandbox API and a 10x cost cut over always-remote inference.","license":"Apache-2.0","type":"module","engines":{"node":">=20"},"exports":{".":{"import":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"main":"./dist/index.cjs","module":"./dist/index.mjs","types":"./dist/index.d.ts","publishConfig":{"access":"public"},"scripts":{"build":"tsup","test":"vitest run","test:watch":"vitest","typecheck":"tsc --noEmit","lint":"eslint src","format":"prettier --write \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","format:check":"prettier --check \"src/**/*.ts\" \"test/**/*.ts\" \"examples/**/*.ts\"","changeset":"changeset","version":"changeset version","release":"pnpm build && changeset publish"},"dependencies":{"node-llama-cpp":"^3.1.1"},"devDependencies":{"@changesets/cli":"^2.27.9","@eslint/js":"^9.39.5","@types/node":"^20.17.6","@typescript-eslint/eslint-plugin":"^8.13.0","@typescript-eslint/parser":"^8.13.0","eslint":"^9.15.0","prettier":"^3.3.3","tsup":"^8.3.5","typescript":"^5.6.3","vitest":"^2.1.4"},"keywords":["llm","inference","local-first","on-device","edge-ai","llama.cpp","gguf","local-llm","typescript","nodejs","sdk","ai","machine-learning","text-generation","self-hosted","offline-ai","sandbox","serverless-alternative"],"repository":{"type":"git","url":"git+https://github.com/boole-ai/boole-npm.git"},"_id":"@boole/boole@0.2.0","gitHead":"34917b319b012115063f821e6a61ed29ca0d9b89","bugs":{"url":"https://github.com/boole-ai/boole-npm/issues"},"homepage":"https://github.com/boole-ai/boole-npm#readme","_nodeVersion":"20.17.0","_npmVersion":"10.8.2","dist":{"integrity":"sha512-pIou0EmTxZ06CaFaN3CTk1MUysy7dwwCLsYMPjO/JJBmRztGNvJhh/uiIOxb/gySUp0SZKbHTEIQrojAzcNBBg==","shasum":"01210d82fe36de3446ea7d3149bd3ad2df944811","tarball":"https://registry.npmjs.org/@boole/boole/-/boole-0.2.0.tgz","fileCount":9,"unpackedSize":124465,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQCZ7c1WK11E6pSFzLGdTZhLY7b5qAnirB/kBNwLFCHAywIhAKlxjW8uoTt+ZEZOQe0Re27abnWXSHScRXoA8/zF7nZ4"}]},"_npmUser":{"name":"jordan.plows","email":"plowstjordan@gmail.com"},"directories":{},"maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/boole_0.2.0_1786491809235_0.37953952261631985"},"_hasShrinkwrap":false}},"time":{"created":"2026-08-11T05:54:06.500Z","modified":"2026-08-11T23:43:29.539Z","0.0.0":"2026-08-11T05:54:06.832Z","0.0.1":"2026-08-11T05:58:52.206Z","0.0.2":"2026-08-11T06:02:50.584Z","0.0.3":"2026-08-11T06:08:01.079Z","0.2.0":"2026-08-11T23:43:29.364Z"},"bugs":{"url":"https://github.com/boole-ai/boole-npm/issues"},"license":"Apache-2.0","homepage":"https://github.com/boole-ai/boole-npm#readme","keywords":["llm","inference","local-first","on-device","edge-ai","llama.cpp","gguf","local-llm","typescript","nodejs","sdk","ai","machine-learning","text-generation","self-hosted","offline-ai","sandbox","serverless-alternative"],"repository":{"type":"git","url":"git+https://github.com/boole-ai/boole-npm.git"},"description":"Local-first LLM inference SDK for JavaScript & TypeScript. Run GGUF models on your own hardware via llama.cpp, with an App/Function/Sandbox API and a 10x cost cut over always-remote inference.","maintainers":[{"name":"jordan.plows","email":"plowstjordan@gmail.com"}],"readme":"# Boole\n\n**Local-first LLM inference for JavaScript & TypeScript.**\nRun GGUF models on your own hardware via llama.cpp — get cloud-SDK ergonomics without the cloud bill.\n\n[![npm version](https://img.shields.io/npm/v/@boole/boole.svg)](https://www.npmjs.com/package/@boole/boole)\n[![license](https://img.shields.io/npm/l/@boole/boole.svg)](./LICENSE)\n[![node](https://img.shields.io/node/v/@boole/boole.svg)](package.json)\n\n```bash\nnpm install @boole/boole\n```\n\n---\n\n## Why Boole\n\nMost inference SDKs assume every call leaves your machine. You pay per token, per second of\nGPU time, per cold start — even for workloads your own laptop or workstation could handle in\nmilliseconds. Boole flips the default: **inference runs locally unless you tell it not\nto.**\n\n- **~10x cheaper by default** — no metered API calls for work your hardware can already do.\n- **No cold starts** — models load once into a long-lived local process, not a fresh\n  container on every request.\n- **No data leaves your machine** — prompts, context, and outputs stay local unless you\n  explicitly opt into remote burst.\n- **Familiar shape** — `App`, `Function`, and `Sandbox` primitives will feel immediately\n  natural if you've used a serverless inference SDK before.\n- **Burst when you need to** — for models too large for local hardware, or workloads that\n  need to scale past one machine, the same function can transparently hand off to remote\n  compute (opt-in, v1).\n\n## Quickstart\n\n```ts\nimport { App } from \"@boole/boole\";\n\nconst app = new App({ name: \"my-app\" });\n\nconst generate = app.function(\n  { model: \"TheBloke/Mistral-7B-Instruct-v0.2-GGUF\", quant: \"Q4_K_M\" },\n  async (ctx, prompt: string) => ctx.llm.generate(prompt),\n);\n\nconst result = await generate.call(\"Write a haiku about GPUs\");\nconsole.log(result);\n```\n\nThe first call downloads and caches the GGUF weights to `~/.boole/models`; every call\nafter that loads from disk and runs entirely on your machine.\n\n## Core concepts\n\n| Primitive | What it does |\n|---|---|\n| `App` | Top-level container that groups functions and shared config. |\n| `Function` | A typed, callable unit of inference work, bound to a specific model. |\n| `Sandbox` | An isolated local execution context for running arbitrary code with resource limits (timeout, memory cap). |\n| `Client` | SDK entry point — model cache directory, default backend, auth for future remote mode. |\n| `RemoteBurst` *(opt-in)* | Routes a `Function` call to remote compute when local hardware can't handle it. |\n\n### Streaming generation\n\n```ts\nfor await (const token of ctx.llm.stream(prompt)) {\n  process.stdout.write(token);\n}\n```\n\n### Running untrusted code in a Sandbox\n\n```ts\nconst sandbox = app.sandbox({ timeoutMs: 5000, memoryLimitMb: 512 });\nconst { stdout } = await sandbox.exec(\"node\", [\"-e\", \"console.log(1 + 1)\"]);\n```\n\n## Platform support\n\nBoole uses native bindings (via `node-llama-cpp`) to talk to llama.cpp directly, with\nGPU offload where available.\n\n| Platform | CPU | GPU acceleration |\n|---|---|---|\n| macOS (Apple Silicon) | ✅ | ✅ Metal |\n| macOS (Intel) | ✅ | — |\n| Linux (x64/arm64) | ✅ | ✅ CUDA / Vulkan |\n| Windows (x64) | ✅ | ✅ CUDA / Vulkan |\n\nPrebuilt binaries are used where available; unsupported platform/architecture combinations\nfall back to compiling from source on install.\n\n## Configuration\n\n```ts\nimport { Client } from \"@boole/boole\";\n\nconst client = new Client({\n  modelCacheDir: \"~/.boole/models\", // where GGUF files are stored\n  defaultBackend: \"llama-cpp\",      // inference backend\n});\n```\n\n## Roadmap\n\n- [x] Local inference via llama.cpp / GGUF\n- [x] `App` / `Function` / `Sandbox` primitives\n- [ ] `RemoteBurst` — opt-in remote fallback for oversized models / scaled workloads\n- [ ] Structured output / grammar-constrained generation helpers\n- [ ] Bun runtime support\n\n## Contributing\n\nIssues and PRs welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md) for local dev setup\n(`pnpm install`, `pnpm test`, `pnpm build`).\n","readmeFilename":"README.md"}