{"_id":"@cyberlangke/tokkit-deci","name":"@cyberlangke/tokkit-deci","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-deci","version":"1.11.0","description":"Deci tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"npm run build --workspace @cyberlangke/tokkit-core && npm run build --workspace @cyberlangke/tokkit-mistral && tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0","@cyberlangke/tokkit-mistral":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"6d7b164cdcb511e7f5815e7278708c780b05d938","_id":"@cyberlangke/tokkit-deci@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-MtZnsPFnIBD8FQOe/ytkM5pKVLNt1HMaI7RdConP+OO7X03RyGKlJ35Mce46x/QGUG6uRS82zFY9Gr0TuAALOw==","shasum":"a03a55ef35c1bf6cdcea68a6d2dc9329a5b7fc3a","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-deci/-/tokkit-deci-1.11.0.tgz","fileCount":10,"unpackedSize":918799,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCbzfrQe7pCGJB1PZuJp0VVRHaK2OXMDIWf7bX+obLozwIgbyw/U12u6Ipr5OEKVH1ood2reOJNwtsJXBJizQ5Ymy0="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-deci_1.11.0_1775230166043_0.4934351932359382"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-03T15:29:25.919Z","1.11.0":"2026-04-03T15:29:26.304Z","modified":"2026-04-03T15:29:26.562Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"Deci tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-deci\n\nDeci 官方文本 tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `decicoder-1b`\n  - 覆盖 `Deci/DeciCoder-1b`\n- `mistral-7b-v0.1`\n  - 额外接受 `Deci/DeciLM-7B`\n  - 额外接受 `Deci/DeciLM-7B-instruct`\n\n说明：\n\n- 当前只纳入 `Deci` 官方组织下公开 `tokenizer.json`、非 gated 且 `license: apache-2.0` 的文本主线。\n- `DeciLM-6b`、`DeciLM-6b-instruct` 当前是 `license: llama2`，不纳入。\n- `DeciCoder-6B` 当前缺少 `tokenizer.json`，不纳入。\n- `DeciLM-7B`、`DeciLM-7B-instruct` 的 `tokenizer.json` SHA256 与现有 `mistral-7b-v0.1` 一致，因此作为模型别名复用。\n- `DeciCoder-1b` 的 `tokenizer.json` SHA256 未命中当前仓库已有快照，因此单独保留为 `decicoder-1b` family。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-deci\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-deci\"\n\nconst coder = await getTokenizer(\"Deci/DeciCoder-1b\")\nconst lm = await getTokenizer(\"Deci/DeciLM-7B-instruct\")\n\nconsole.log(coder !== lm)\n```\n","readmeFilename":"README.md","_rev":"1-a7afb28f81cd21bb00dca79b5963acdd"}