{"_id":"@cyberlangke/tokkit-zyphra","name":"@cyberlangke/tokkit-zyphra","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-zyphra","version":"1.11.0","description":"Zyphra tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"1605738513582c2a4a1e527379e6414662045386","_id":"@cyberlangke/tokkit-zyphra@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-cwfRtoKl3U3WXeKiAhwxc92dOGh66NxX6IBYPFSLtGicuowugoj/0LvIJYyPaFxqh/L9EtSUmyL1N4qXRF8fCg==","shasum":"cb98baa34d6d6dd673869673e258881e6549ff0e","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-zyphra/-/tokkit-zyphra-1.11.0.tgz","fileCount":22,"unpackedSize":15335149,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQCSMc1iJ+IkAUNb0/JqiFY/8H4RXgTviVI/Ft6jJk+cOAIhAI0fMQIoi1FGKiFrc1NUj7ml1v9i38jXQo6pP9h7Dryc"}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-zyphra_1.11.0_1775031098078_0.1629641904481307"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-01T08:11:37.933Z","1.11.0":"2026-04-01T08:11:38.530Z","modified":"2026-04-01T08:11:38.806Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"Zyphra tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-zyphra\n\nZyphra 官方文本 tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `zamba-7b-v1`\n  - 覆盖 `Zyphra/Zamba-7B-v1`\n- `zamba2-1.2b`\n  - 覆盖 `Zyphra/Zamba2-1.2B`\n- `zamba2-2.7b`\n  - 覆盖 `Zyphra/Zamba2-2.7B`\n- `zamba2-instruct`\n  - 覆盖 `Zyphra/Zamba2-1.2B-instruct`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `Zyphra/Zamba2-2.7B-instruct`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `Zyphra/Zamba2-7B-Instruct`\n- `zamba2-instruct-v2`\n  - 覆盖 `Zyphra/Zamba2-1.2B-Instruct-v2`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `Zyphra/Zamba2-2.7B-Instruct-v2`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `Zyphra/Zamba2-7B-Instruct-v2`\n- `zr1-1.5b`\n  - 覆盖 `Zyphra/ZR1-1.5B`\n- `zaya1`\n  - 覆盖 `Zyphra/ZAYA1-base`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `Zyphra/ZAYA1-reasoning-base`\n\n当前不纳入：\n\n- `Zyphra/Zamba-7B-v1-phase1`\n- `Zyphra/Zamba2-7B`\n- `Zyphra/BlackMamba-*`\n- `Zyphra/Mamba-370M`\n- `Zyphra/ZUNA`\n\n说明：\n\n- `Zamba2` 当前存在多组彼此不同的 tokenizer，不能先验假设整个系列只复用一个 family。\n- `Zyphra/Zamba2-7B` 当前是 gated 模型，匿名环境下无法稳定取得公开 tokenizer 快照，因此暂不并入当前公开子包。\n- `Zyphra/ZR1-1.5B` 虽然 `model_type` 是 `qwen2`，但 `tokenizer.json` hash 与当前仓库内已有 `qwen2` family 不同，因此单独保留 `zr1-1.5b` family。\n- `Zyphra/ZAYA1-base` 与 `Zyphra/ZAYA1-reasoning-base` 的 `tokenizer.json` SHA256 一致，因此收口到同一个 `zaya1` family。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-zyphra\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-zyphra\"\n\nconst zamba = await getTokenizer(\"zamba-7b-v1\")\nconst zaya = await getTokenizer(\"Zyphra/ZAYA1-reasoning-base\")\n\nconsole.log(zamba.encode(\"Hello, Zyphra!\"))\nconsole.log(zaya.encode(\"Hello, Zyphra!\"))\n```\n","readmeFilename":"README.md","_rev":"1-e508689dd78e968b06e033f9dbafd5a7"}