{"_id":"@cyberlangke/tokkit-nanbeige","name":"@cyberlangke/tokkit-nanbeige","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-nanbeige","version":"1.11.0","description":"Nanbeige tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"fd24cf9f8a97d11756ce56f13fb0198cfac28162","_id":"@cyberlangke/tokkit-nanbeige@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-gSrDmSvqdNJfw0CGHsTew1xOOOaVd5vypK7GU+3c+tgOAZfG2d5/PgGeE/Ymupkx+O4S8LRgai4w2zMBBlv+FQ==","shasum":"0d40f6ce89d23a029163d71df31c56d6188504ef","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-nanbeige/-/tokkit-nanbeige-1.11.0.tgz","fileCount":12,"unpackedSize":9801180,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQDDDEILx0nKfHBfQCNldZwBU7k4CbNwNhTwiB+kbzBhMAIgSW8L4J9kuCOhlgms56uvtoTM5diCoodsqIlTkDxQJtE="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-nanbeige_1.11.0_1774969567439_0.2089699478964926"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-31T15:06:07.350Z","1.11.0":"2026-03-31T15:06:07.863Z","modified":"2026-03-31T15:06:08.151Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"Nanbeige tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-nanbeige\n\nNanbeige 官方文本 tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `nanbeige4`\n  - 覆盖 `Nanbeige/Nanbeige4.1-3B`\n  - 也聚合当前确认完全复用同一 tokenizer 的：\n    - `Nanbeige/Nanbeige4-3B-Thinking-2510`\n    - `Nanbeige/Nanbeige4-3B-Thinking-2511`\n    - `Nanbeige/ToolMind-Web-3B`\n- `nanbeige4-base`\n  - 覆盖 `Nanbeige/Nanbeige4-3B-Base`\n\n说明：\n\n- `Nanbeige4.1-3B`、`Nanbeige4-3B-Thinking-2510`、`Nanbeige4-3B-Thinking-2511` 与 `ToolMind-Web-3B` 的 `tokenizer.json` SHA256 一致，因此收口到同一个 `nanbeige4` family。\n- `Nanbeige4-3B-Base` 的 `tokenizer.json` SHA256 与上面一组不同，因此单独保留 `nanbeige4-base` family。\n- 当前官方 `text-generation` 主线里可直接纳入的 BPE tokenizer 已在这个子包内收口。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-nanbeige\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-nanbeige\"\n\nconst base = await getTokenizer(\"nanbeige4-base\")\nconst chat = await getTokenizer(\"Nanbeige/ToolMind-Web-3B\")\n\nconsole.log(base.encode(\"Hello, Nanbeige!\"))\nconsole.log(chat.encode(\"Hello, Nanbeige!\"))\n```\n","readmeFilename":"README.md","_rev":"1-82cdf219477517fcf88f8e20c4f22856"}