{"_id":"@cyberlangke/tokkit-abeja","name":"@cyberlangke/tokkit-abeja","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-abeja","version":"1.11.0","description":"ABEJA official model aliases for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"npm run build --workspace @cyberlangke/tokkit-core && npm run build --workspace @cyberlangke/tokkit-qwen && tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0","@cyberlangke/tokkit-qwen":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"31c295eed6fd96b7a9fb91f574da6cc0ffbecba2","_id":"@cyberlangke/tokkit-abeja@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-QX79+wwnfnoznnmkmCn/WOXWbxHj70iDek7oR4qfm8JdMzLmpf55ajXvoV3U5dm3D1YOgaTdUS1A+J3m4PBIfg==","shasum":"f76dcda8d057ad6f3ec2701d9e3c7c4531334869","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-abeja/-/tokkit-abeja-1.11.0.tgz","fileCount":8,"unpackedSize":15354,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQDPNM1pyAcGVL1GnDUKCn/0S+CZjwKqOmjfbVHhxuwOkQIgSIvZR+4UW8XHjiexP0GpGEhKjojtqCRzwzqLEO1jmlQ="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-abeja_1.11.0_1775034822900_0.9270326854408972"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-01T09:13:42.818Z","1.11.0":"2026-04-01T09:13:43.034Z","modified":"2026-04-01T09:13:43.239Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"ABEJA official model aliases for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-abeja\n\nABEJA 官方文本模型的 tokkit 子包。\n\n当前纳入的官方主线模型：\n\n- `abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0`\n- `abeja/ABEJA-Qwen2.5-7b-Japanese-v0.1`\n- `abeja/ABEJA-QwQ32b-Reasoning-Japanese-v1.0`\n- `abeja/ABEJA-Qwen3-14B-Agentic-256k-v0.1`\n\n当前不纳入：\n\n- `abeja/ABEJA-Qwen2.5-32b-Japanese-v0.1`\n  - 已被 `v1.0` 取代，不再作为当前主线收口对象\n- `abeja/gpt-neox-japanese-2.7b`\n- `abeja/gpt2-large-japanese`\n  - 旧研究线，不作为当前官方主线 LLM\n- `abeja/Mixtral-8x7B-*`\n  - `alpha` / `merged` 实验线，不作为当前稳定主线\n\n说明：\n\n- 以上纳入模型的 `tokenizer.json` 已验 hash。\n- `abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0`\n- `abeja/ABEJA-Qwen2.5-7b-Japanese-v0.1`\n- `abeja/ABEJA-QwQ32b-Reasoning-Japanese-v1.0`\n  - 当前都与已支持的 `qwen2.5` family 完全一致。\n- `abeja/ABEJA-Qwen3-14B-Agentic-256k-v0.1`\n  - 当前与已支持的 `qwen3` family 完全一致。\n- 该子包不重复分发 tokenizer 快照，而是复用现有 family，并注册 ABEJA 官方模型别名。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-abeja\n```\n\n```ts\nimport { getEncoding } from \"@cyberlangke/tokkit-abeja\"\n\nconst tokenizer = await getEncoding(\"abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0\")\n\nconsole.log(tokenizer.encode(\"こんにちは、ABEJA!\"))\n```\n","readmeFilename":"README.md","_rev":"1-a4924b08b5e6832a0a77abbdc969f8b2"}