{"_id":"@agentkit-js/model-local","_rev":"3-4910a695c6bd7c6b89835880f0d1610c","name":"@agentkit-js/model-local","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@agentkit-js/model-local","version":"0.1.0","keywords":["agentkit","agent","ai","llm","local","offline","llama.cpp","gguf","qwen","gemma"],"author":{"name":"agentkit-js contributors"},"license":"Apache-2.0","_id":"@agentkit-js/model-local@0.1.0","maintainers":[{"name":"tellerlin","email":"lintao_mailbox@163.com"}],"homepage":"https://github.com/telleroutlook/agentkit-js/tree/main/packages/model-local#readme","bugs":{"url":"https://github.com/telleroutlook/agentkit-js/issues"},"dist":{"shasum":"3e11c04a54fa53a31dbac5c773ddfc1e04d340fb","tarball":"https://registry.npmjs.org/@agentkit-js/model-local/-/model-local-0.1.0.tgz","fileCount":31,"integrity":"sha512-w24dBzLFIRE/34kQkMttYSpAxn7xLAQkRQWr8IVysmwD67/MGGYWUOQi9US4kl2ZORwsAQEyiJXqItxw4uif3w==","signatures":[{"sig":"MEUCICqCZNrGMjAP0K11NbMzD8ta819DYrdHA0qRKRT3i4dKAiEAzxv/E1rl3YcT72jmHEIAgZ1TuCwih0b8/xueiO7MHA8=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@agentkit-js%2fmodel-local@0.1.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":127946},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","engines":{"node":">=20.0.0"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./registry":{"types":"./dist/registry.d.ts","import":"./dist/registry.js"},"./downloader":{"types":"./dist/downloader.d.ts","import":"./dist/downloader.js"}},"gitHead":"c629f75b91a8855e107cb0130c858b10e8ea6e79","scripts":{"test":"vitest run","build":"tsc -p tsconfig.json","clean":"rm -rf dist .turbo","typecheck":"tsc -p tsconfig.json --noEmit"},"_npmUser":{"name":"tellerlin","email":"lintao_mailbox@163.com"},"deprecated":"This package has moved to @wasmagent/model-local. Please update your dependency.","repository":{"url":"git+https://github.com/telleroutlook/agentkit-js.git","type":"git","directory":"packages/model-local"},"_npmVersion":"10.9.8","description":"Embedded local LLM provider for agentkit-js — node-llama-cpp adapter with grammar-constrained tool calling, multi-mirror download (HF/hf-mirror/ModelScope), and cert pipeline","directories":{},"_nodeVersion":"22.22.3","dependencies":{"@agentkit-js/core":"workspace:*"},"publishConfig":{"access":"public","provenance":true},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.24.0","vitest":"^2.1.0","typescript":"^5.7.0"},"peerDependencies":{"node-llama-cpp":">=3.18.0"},"peerDependenciesMeta":{"node-llama-cpp":{"optional":true}},"_npmOperationalInternal":{"tmp":"tmp/model-local_0.1.0_1781508321192_0.8332401454337142","host":"s3://npm-registry-packages-npm-production"}}},"time":{"created":"2026-06-15T07:25:20.994Z","modified":"2026-07-04T01:07:21.716Z","0.1.0":"2026-06-15T07:25:21.342Z"},"bugs":{"url":"https://github.com/telleroutlook/agentkit-js/issues"},"author":{"name":"agentkit-js contributors"},"license":"Apache-2.0","homepage":"https://github.com/telleroutlook/agentkit-js/tree/main/packages/model-local#readme","keywords":["agentkit","agent","ai","llm","local","offline","llama.cpp","gguf","qwen","gemma"],"repository":{"url":"git+https://github.com/telleroutlook/agentkit-js.git","type":"git","directory":"packages/model-local"},"description":"Embedded local LLM provider for agentkit-js — node-llama-cpp adapter with grammar-constrained tool calling, multi-mirror download (HF/hf-mirror/ModelScope), and cert pipeline","maintainers":[{"email":"lintao_mailbox@163.com","name":"tellerlin"},{"email":"hyin.sybase@gmail.com","name":"haining-yin"}],"readme":"# @agentkit-js/model-local\n\n> Embedded local-LLM provider for [agentkit-js](https://github.com/telleroutlook/agentkit-js) — `node-llama-cpp` adapter with grammar-constrained tool calling, multi-mirror downloads (HuggingFace / hf-mirror / ModelScope), and a certification harness for picking which models actually work in agent workflows.\n\nThe whole agent stack — model, code execution, state — runs on the user's machine. No cloud LLM, no API key, no telemetry.\n\n## Install\n\n```bash\n# Provider (small package, no native deps).\nnpm install @agentkit-js/model-local\n\n# Optional native peer — pre-built binaries for macOS/Linux/Windows + ARM/x64.\nnpm install node-llama-cpp\n```\n\nThe native peer is **optional**: if you only want the registry/downloader/types (e.g. to ship a server that proxies models), you can skip it. `LocalModel.generate()` will throw a typed `LocalModelDependencyError` with an actionable install hint if it's missing.\n\n## Quick start\n\n```ts\nimport { LocalModel, localFirst } from \"@agentkit-js/model-local\";\nimport { AnthropicModel, CodeAgent } from \"@agentkit-js/core\";\n\n// Pick one of three sources:\nconst local = new LocalModel({ source: { model: \"qwen2.5-1.5b\" } });        // alias\n// or:        new LocalModel({ source: { path: \"./my-model.gguf\" } });       // user GGUF\n// or:        new LocalModel({ source: { url: \"https://...\" } });            // direct URL\n\n// Use it directly:\nconst agent = new CodeAgent({ model: local, tools: [] });\n\n// Or compose with a cloud fallback for prod:\nconst model = localFirst(\n  local,\n  new AnthropicModel(\"claude-haiku-4-5-20251001\", process.env.ANTHROPIC_API_KEY),\n);\n```\n\n## Three model sources\n\n| Source | Use when | Verification |\n|---|---|---|\n| `{ model: \"alias\" }` | You want a maintained, vetted model | sha256 (registry-pinned) |\n| `{ path: \"./x.gguf\" }` | You have a self-trained or hand-downloaded GGUF | none (your file, your trust) |\n| `{ url: \"https://...\" }` | One-off pull from any URL | sha256 only if you supply `expectedSha256` |\n\n## Mirror selection (大陆友好)\n\nThree resolution layers, high → low precedence:\n\n1. **Programmatic** — `new LocalModel({ source: { model: \"qwen2.5-1.5b\" }, mirror: \"modelscope\" })`\n2. **Environment** — `AGENTKIT_MODEL_MIRROR=hf-mirror` (or `modelscope`, or any URL prefix)\n3. **Registry default** — HuggingFace first, then mirrors\n\nBuilt-in presets:\n- `huggingface` — origin (sha256 anchor)\n- `hf-mirror` — `hf-mirror.com`, community-run, URL-compatible with HF\n- `modelscope` — `modelscope.cn`, ModelScope魔搭 国内 CDN\n\nCustom CDN: pass any URL prefix as `mirror`, and the downloader will append the canonical filename and hit your CDN first, falling back to the registry chain if it fails.\n\n```bash\n# One-line CLI override:\nAGENTKIT_MODEL_MIRROR=modelscope npx agentkit model pull qwen2.5-1.5b\n```\n\n⚠️ **Mirror trust model**: every download is sha256-verified against the registry value (which is anchored to the HuggingFace original). Mirrors are *transport channels*, not trust roots.\n\n## Grammar-constrained tool calling\n\nSub-1B models routinely emit malformed JSON when asked to call tools. `LocalModel` enables JSON-schema grammar in the sampler by default, so `tool_use` output is **structurally legal 100% of the time**. Semantic correctness still depends on the model.\n\n```ts\nconst model = new LocalModel({\n  source: { model: \"qwen2.5-1.5b\" },\n  enableGrammar: true,  // default\n});\n```\n\nSet `enableGrammar: false` to compare A/B against free-form sampling — useful for diffing on the cert harness.\n\n## CLI\n\n```bash\n# Browse the registry.\nagentkit model list\n\n# Pull (resumable, sha256-verified, multi-mirror).\nagentkit model pull qwen2.5-1.5b\n\n# Force a mirror.\nagentkit model pull qwen2.5-1.5b --mirror modelscope\n\n# Verify a cached file's sha256.\nagentkit model verify qwen2.5-1.5b\n\n# Free up disk.\nagentkit model rm qwen2.5-1.5b\n```\n\n`agentkit-js/cli` declares `@agentkit-js/model-local` as an **optional peer** — if you don't install this package, the CLI falls back to a clean error message rather than crashing.\n\n## Routing presets\n\n```ts\nimport { localFirst, offlineOnly, devLocalOr } from \"@agentkit-js/model-local\";\n\n// Try local; fall through to cloud on any error.\nconst a = localFirst(localModel, cloudModel);\n\n// Loud \"no cloud, ever\" envelope (passthrough today; reserves a hook for\n// future enforcement).\nconst b = offlineOnly(localModel);\n\n// Dev convenience: AGENTKIT_DEV_LOCAL=1 → local; otherwise → cloud.\nconst c = devLocalOr(localModel, cloudModel);\n```\n\nThese are documented combinations of the existing `FallbackModel` from `@agentkit-js/core` — *not* a parallel routing mechanism. You get the same retry/fallover semantics as everywhere else in the framework.\n\n## Recommended models — current registry\n\n> All entries are <1.5 GB at q4_k_m or smaller quantisation. The `recommended` flag flips on once the cert harness publishes a passing score (see L4). Until then you can still `agentkit model pull <alias>` and self-evaluate.\n\n| Alias | Best for | License | Size |\n|---|---|---|---|\n| `qwen2.5-0.5b` | Tool calling on tiny footprint — **3/3 form/picked/semantic on cert** (real-machine, 2026-06-12) | Apache-2.0 | ~409 MB (q4_0, sha256 pinned 2026-06-13) |\n| `qwen3-0.6b` | English/code, only Q8_0 quant published | Apache-2.0 | ~610 MB (q8_0, sha256 pinned 2026-06-13) |\n| `qwen2.5-1.5b` | Chinese + English, 32K context (Stage-0 ≤2GB winner per evomerge GSM8K 70.5%) | Apache-2.0 | ~1.07 GB (q4_k_m, sha256 pinned 2026-06-13) |\n| `gemma-3-1b` | English tasks, ggml-org mirror | Gemma ToU | ~769 MB (q4_k_m, sha256 pinned 2026-06-13) |\n| `llama-3.2-1b` | English/code, 128K context, lmstudio-community mirror | Llama 3.2 Community | ~770 MB (q4_k_m, sha256 pinned 2026-06-13) |\n\nSee `docs/reports/local-model-cert-2026-06-12.md` in the agentkit-js repo for the full real-machine baseline.\n\nRun the cert harness on any of them (or your own GGUF):\n\n```bash\nnode examples/benchmarks/local-model-cert.mjs --model qwen2.5-1.5b --kernel quickjs\nnode examples/benchmarks/local-model-cert.mjs --path ./my-model.gguf --out report.md\n```\n\n## Honest caveats\n\n- **Sub-1B models are not Claude/GPT-class.** Complex tool routing, multi-step reasoning, and long-form synthesis are still cloud-class jobs. The local model is for high-frequency, lower-difficulty work — drafts, intent classification, summarisation, dev/CI runs.\n- **Grammar guarantees form, not semantics.** A grammar-clean output can still pick the wrong tool or wrong arguments. The cert harness's *form rate* and *semantic rate* are reported separately.\n- **Native binding.** `node-llama-cpp` brings prebuilt binaries but requires Node.js 20+ on a desktop/server platform. **Cloudflare Workers cannot run this.** Use `localFirst` with a cloud model if you deploy to edge runtimes.\n\n## License\n\nApache-2.0 — see [LICENSE](./LICENSE).\n\nModel files have their own licenses; they are downloaded from the publisher's host on demand and never re-distributed by this package. See `MODEL_REGISTRY` (in `src/registry.ts`) for the license attribute on each entry.\n","readmeFilename":"README.md"}