{"_id":"@cyberlangke/tokkit-h2oai","name":"@cyberlangke/tokkit-h2oai","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-h2oai","version":"1.11.0","description":"H2O.ai tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"32571c4fb6d6aaa81214fb04cfb73b1934151425","_id":"@cyberlangke/tokkit-h2oai@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-0btUNY110P0Hlm6Wq0ZFiJRQyg2lZSPBP3A42ueHJ1gFhTLzBnvkcD1nwUwgSSrdEBiAw/JKWBkbwCedlObRIg==","shasum":"b8cab0424af5bb8378ebe34668e7966f69db255d","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-h2oai/-/tokkit-h2oai-1.11.0.tgz","fileCount":18,"unpackedSize":4157429,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIBkZtkC3CpBNMIluolPRrjMd3IAh9Iz4bLUaTskDj+HcAiEAwYurLQr0I45OgARB2EwPmF1UU2FszJiGnkXeMXwUgKU="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-h2oai_1.11.0_1774931113653_0.31872755721905466"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-31T04:25:13.600Z","1.11.0":"2026-03-31T04:25:13.941Z","modified":"2026-03-31T04:25:14.112Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"H2O.ai tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-h2oai\n\nH2O.ai 官方 Danube tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `danube`\n  - 覆盖 `h2oai/h2o-danube-1.8b-base`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `h2oai/h2o-danube-1.8b-chat`\n- `danube2`\n  - 覆盖 `h2oai/h2o-danube2-1.8b-base`\n  - 也聚合当前确认完全复用同一 tokenizer 的：\n    - `h2oai/h2o-danube2-1.8b-chat`\n    - `h2oai/h2o-danube3-500m-base`\n    - `h2oai/h2o-danube3-4b-base`\n- `danube3-500m-chat`\n  - 覆盖 `h2oai/h2o-danube3-500m-chat`\n- `danube3-4b-chat`\n  - 覆盖 `h2oai/h2o-danube3-4b-chat`\n- `danube3.1-4b-chat`\n  - 覆盖 `h2oai/h2o-danube3.1-4b-chat`\n\n当前不纳入：\n\n- `h2oai/h2o-danube-1.8b-sft`\n- `h2oai/h2o-danube2-1.8b-sft`\n- `GGUF` / `AWQ` / `ONNX` 等导出对象\n- `danube2-singlish-finetuned`\n- `h2ogpt-*`、`h2ovl-*` 与其他旧微调 / 多模态路线\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-h2oai\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-h2oai\"\n\nconst base = await getTokenizer(\"danube2\")\nconst chat = await getTokenizer(\"h2oai/h2o-danube3-4b-chat\")\n\nconsole.log(base.encode(\"Hello, Danube!\"))\nconsole.log(chat.encode(\"Hello, Danube!\"))\n```\n","readmeFilename":"README.md","_rev":"1-13daa7729a60105175d1b631cff64e3b"}