{"_id":"@cyberlangke/tokkit-bsc-lt","name":"@cyberlangke/tokkit-bsc-lt","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-bsc-lt","version":"1.11.0","description":"BSC-LT tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"npm run build --workspace @cyberlangke/tokkit-core && tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"6d7b164cdcb511e7f5815e7278708c780b05d938","_id":"@cyberlangke/tokkit-bsc-lt@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-WwoysVCiJskDrjSUiVKewveNNxhm/C5jrX3p7wpfCCg9zcaIJ5jp7m9Q2y0NLcubfG9jif5lFC+tEcEoo2JHJw==","shasum":"cda87be3ae32aa9c93052b563fd98c4ecbd862c2","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-bsc-lt/-/tokkit-bsc-lt-1.11.0.tgz","fileCount":12,"unpackedSize":18419749,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCtEMNbhb2p9nfw0b7CFJqRK3ioccxPscjKimcMrRTvTgIgS/dLYo5bJqvxWNxvO/fOVFDs1wvh/IOL2zRCsi6njFU="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-bsc-lt_1.11.0_1775230130736_0.6160420207576136"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-03T15:28:50.645Z","1.11.0":"2026-04-03T15:28:51.312Z","modified":"2026-04-03T15:28:51.521Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"BSC-LT tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-bsc-lt\n\nBSC-LT 官方文本 tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `salamandra`\n  - 覆盖 `BSC-LT/salamandra-2b`\n  - 覆盖 `BSC-LT/salamandra-7b`\n  - 覆盖 `BSC-LT/ALIA-40b`\n  - 覆盖 `BSC-LT/ALIA-40b-instruct-2601`\n- `salamandra-instruct`\n  - 覆盖 `BSC-LT/salamandra-2b-instruct`\n  - 覆盖 `BSC-LT/salamandra-7b-instruct`\n\n说明：\n\n- 当前只纳入 `BSC-LT` 官方组织下公开 `tokenizer.json`、非 gated、`license: apache-2.0` 且 `model.type = BPE` 的文本主线。\n- `salamandra-7b-instruct-tools`、`salamandra-7b-instruct-tools-16k` 属于工具化变体，当前不纳入。\n- `*-GGUF`、`*-GPTQ`、`*-FP8`、`*rag*`、`*aina-hack*`、`experimental*`、`Checkpoint_*`、`Flor-*` 等衍生或非当前主线对象不纳入。\n- `salamandra-2b`、`salamandra-7b`、`ALIA-40b`、`ALIA-40b-instruct-2601` 的 `tokenizer.json` SHA256 一致，因此复用 `salamandra`。\n- `salamandra-2b-instruct`、`salamandra-7b-instruct` 的 `tokenizer.json` SHA256 一致，因此复用 `salamandra-instruct`。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-bsc-lt\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-bsc-lt\"\n\nconst base = await getTokenizer(\"BSC-LT/salamandra-7b\")\nconst instruct = await getTokenizer(\"BSC-LT/salamandra-7b-instruct\")\n\nconsole.log(base !== instruct)\n```\n","readmeFilename":"README.md","_rev":"1-02b679a4f7f331ca590dd7a5967dda2b"}