{"_id":"@cyberlangke/tokkit-dream-org","name":"@cyberlangke/tokkit-dream-org","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-dream-org","version":"1.11.0","description":"Dream-org tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"6d7b164cdcb511e7f5815e7278708c780b05d938","_id":"@cyberlangke/tokkit-dream-org@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-noRqGVSurFoaLd67WbXpy5C5Ih2TiJN4WD84LiMCar2dRUpNy3v03yT1cyIgMJV7Nq5zQT9fwjMCLl9/INaMZg==","shasum":"4126557b970b290aad6ea72cd8125749a2f72516","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-dream-org/-/tokkit-dream-org-1.11.0.tgz","fileCount":12,"unpackedSize":5940926,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIFIqhdkCjPbn4Luv6koKhHlrKf2zCwk97XI2xSGrit7MAiEArltq/H1teDxZkf93hLVj2nzGrevXS71psjXA8WTHlh8="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-dream-org_1.11.0_1775230183755_0.22898504615469983"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-03T15:29:43.685Z","1.11.0":"2026-04-03T15:29:44.054Z","modified":"2026-04-03T15:29:44.276Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"Dream-org tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-dream-org\n\nDream-org 官方文本 tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `dream-v0`\n  - 覆盖 `Dream-org/Dream-v0-Base-7B`\n  - 也聚合当前确认完全复用同一 tokenizer 的：\n    - `Dream-org/Dream-v0-Instruct-7B`\n    - `Dream-org/Dream-Coder-v0-Base-7B`\n    - `Dream-org/Dream-Coder-v0-Instruct-7B`\n- `dreamon-v0`\n  - 覆盖 `Dream-org/DreamOn-v0-7B`\n\n说明：\n\n- 当前纳入范围只包含 `Dream-org` 官方组织下公开的纯文本主线模型。\n- `Dream-v0-*` 与 `Dream-Coder-v0-*` 官方仓库没有直接提供 `tokenizer.json`；当前快照由官方 `vocab.json`、`merges.txt`、`added_tokens.json`、`tokenizer_config.json` 按官方 slow tokenizer 规则组装成 HF 风格 `tokenizer.json` 后压缩入库。\n- `DreamOn-v0-7B` 在同一条 regex / ByteLevel 规则上使用独立 `vocab` 与 `added_tokens`，因此单独保留为 `dreamon-v0` family。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-dream-org\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-dream-org\"\n\nconst dream = await getTokenizer(\"Dream-org/Dream-Coder-v0-Instruct-7B\")\nconst dreamon = await getTokenizer(\"dreamon-v0\")\n\nconsole.log(dream === dreamon)\n```\n","readmeFilename":"README.md","_rev":"1-7ba640186efbb0d855535936b4ec9015"}