{"_id":"@cyberlangke/tokkit-openai","name":"@cyberlangke/tokkit-openai","dist-tags":{"latest":"1.11.0"},"versions":{"1.11.0":{"name":"@cyberlangke/tokkit-openai","version":"1.11.0","description":"OpenAI tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"Apache-2.0","gitHead":"79a92a7820469fb39c1d8c1d9db2e9bbcfc8bdf2","_id":"@cyberlangke/tokkit-openai@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-ndnc6dbvblZFhn8ztZYI4r37LGHysohtIqy/bM0Mr62tMzhv4bZINwY/qLKpMrtGWYqM9PbvIauZOnvuLNI1Sw==","shasum":"7d92a114dd55c239754cfa27dbbc10ca212125e8","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-openai/-/tokkit-openai-1.11.0.tgz","fileCount":10,"unpackedSize":6841695,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQDVpDsS7YOGA4az+x+IZjKBGLxGuqtBwv1F4kRSGXje7wIhAND3tdMoZ13CrFzBNkqNbfWSMuTmE7KDxNdijrF+Njtb"}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-openai_1.11.0_1774937437430_0.34636014780172597"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-31T06:10:37.350Z","1.11.0":"2026-03-31T06:10:37.793Z","modified":"2026-03-31T06:10:38.015Z"},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"description":"OpenAI tokenizer families for tokkit.","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"author":{"name":"cyberlangke"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"license":"Apache-2.0","readme":"# @cyberlangke/tokkit-openai\n\nOpenAI 官方 `gpt-oss` tokenizer 的 tokkit 子包。\n\n当前内置 family：\n\n- `gpt-oss`\n  - 覆盖 `openai/gpt-oss-20b`\n  - 也聚合当前确认完全复用同一 tokenizer 的 `openai/gpt-oss-120b`\n\n当前不纳入：\n\n- `openai/gpt-oss-safeguard-20b`\n- `openai/gpt-oss-safeguard-120b`\n- `openai/circuit-sparsity`\n\n说明：\n\n- `gpt-oss-safeguard-*` 官方 model card 明确标注 `base_model_relation: finetune`，属于专项安全微调线，不作为当前公开维护目标。\n- `openai/circuit-sparsity` 不是当前 `gpt-oss` 主线。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-openai\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-openai\"\n\nconst tokenizer = await getTokenizer(\"gpt-oss\")\nconst larger = await getTokenizer(\"openai/gpt-oss-120b\")\n\nconsole.log(tokenizer.encode(\"Hello, OpenAI!\"))\nconsole.log(larger.encode(\"Hello, OpenAI!\"))\n```\n","readmeFilename":"README.md","_rev":"1-1363efddb6013355a7732019427e9179"}