{"_id":"@cyberlangke/tokkit-cyberagent","name":"@cyberlangke/tokkit-cyberagent","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-cyberagent","version":"1.11.0","description":"cyberagent tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"SEE LICENSE IN LICENSE","gitHead":"3bd35aeb879f80b548455b4d6c6447b9c0daa6e4","_id":"@cyberlangke/tokkit-cyberagent@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-uFEwQF4bR3qXMvrmqw6uqXuE92puwJs7VVwSoqiugT5hXQbv0ASJ3TfEqjdNDYiwmb5V0O7Q4lQwtAPc/kibIg==","shasum":"c3ce5245113790406ec16ccd13bbfb89b7aa05f5","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-cyberagent/-/tokkit-cyberagent-1.11.0.tgz","fileCount":14,"unpackedSize":3708294,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQDnsjtR1LLmFWaTj0NneTOi+/raPUc8iwdP+d4I+ao1AQIhAIU2JrD+Y7KBHYahXTFonkKCC0mfVpEZQnAMFdtHkfdq"}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-cyberagent_1.11.0_1775117669592_0.24160815941876979"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-02T08:14:29.514Z","1.11.0":"2026-04-02T08:14:29.899Z","modified":"2026-04-02T08:14:30.128Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"cyberagent tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"SEE LICENSE IN LICENSE","readme":"# @cyberlangke/tokkit-cyberagent\n\ncyberagent 官方文本 tokenizer 的 tokkit 子包。\n\n这是一个独立特殊协议子包，不包含在 `@cyberlangke/tokkit` 总包里。\n\n当前内置 family：\n\n- `open-calm`\n  - 覆盖 `cyberagent/open-calm-1b`\n  - 覆盖 `cyberagent/open-calm-small`\n  - 覆盖 `cyberagent/open-calm-medium`\n  - 覆盖 `cyberagent/open-calm-3b`\n  - 覆盖 `cyberagent/open-calm-large`\n  - 覆盖 `cyberagent/open-calm-7b`\n- `calm2`\n  - 覆盖 `cyberagent/calm2-7b`\n  - 覆盖 `cyberagent/calm2-7b-chat`\n- `calm3`\n  - 覆盖 `cyberagent/calm3-22b-chat`\n\n当前不纳入：\n\n- `cyberagent/Mistral-Nemo-Japanese-Instruct-2408`\n- `cyberagent/DeepSeek-R1-Distill-Qwen-*`\n- `cyberagent/calm2-7b-chat-dpo-experimental`\n- `cyberagent/markupdm`\n\n说明：\n\n- `open-calm-*` 当前共享同一套 tokenizer。\n- `calm2-7b` 与 `calm2-7b-chat` 当前共享同一套 tokenizer。\n- `calm3-22b-chat` 当前使用独立 tokenizer。\n- 由于该子包同时分发 `Apache-2.0` 与 `CC-BY-SA-4.0` 的 tokenizer 资产，因此不进入 `@cyberlangke/tokkit` 总包。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-cyberagent\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-cyberagent\"\n\nconst openCalm = await getTokenizer(\"open-calm\")\nconst calm2 = await getTokenizer(\"cyberagent/calm2-7b-chat\")\n\nconsole.log(openCalm === calm2)\n```\n","readmeFilename":"README.md","_rev":"1-00364c30f7178e367f679923fb6482b2"}