{"_id":"@cyberlangke/tokkit-microsoft","_rev":"14-cf5c9f993535dfc06c43c71f8c9890fc","name":"@cyberlangke/tokkit-microsoft","dist-tags":{"latest":"1.11.0"},"versions":{"1.1.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.1.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.1.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"b8a0b8a68a21a7e9fc37f98e122257523e62cf3e","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.1.0.tgz","fileCount":28,"integrity":"sha512-Lr4/1I0tld9vucVsdP649H/k2CkcX9lWGq07vfaRRWrTOFTzWlullrZdP5T3GRFL7mbiyXORCRvoB5zI3G68AA==","signatures":[{"sig":"MEYCIQD0XEe4qWTQ+BmoD0cSUc7bdWDKJR54cdH9a2o5HxugFgIhAPUKP0WTmkTSadZo49iKOY1XoNS0gmYIHXllPkHOCrUP","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"835e3f36d2e2ca495a19b34064dce9d763416632","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.1.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.1.0_1774683697685_0.19938616223952876","host":"s3://npm-registry-packages-npm-production"}},"1.2.1":{"name":"@cyberlangke/tokkit-microsoft","version":"1.2.1","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.2.1","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"5e390d02687897f12cb0ae6c89a4e8e367bcff4d","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.2.1.tgz","fileCount":28,"integrity":"sha512-cU8nXuj4/pm8NGJaStgL7h4zPlp5dx/DjTCP3YIMFn7Aup/Tl5ItkAs6jJsk26sxuQF6USu8xjSDqBTTvJ4jwQ==","signatures":[{"sig":"MEYCIQDf+qSlsHRqXcryqYPSXDLcU8kT1wuJhL0s4aNTouSCjwIhANEDmFRqKO9KWGULTH2BBKk47qVJWH+2HmdyxzA6Ah0e","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"4c6d4f8a7edcbebfd58cdf5301eb675242f4bb21","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.2.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.2.1_1774688733668_0.5881629661305519","host":"s3://npm-registry-packages-npm-production"}},"1.2.2":{"name":"@cyberlangke/tokkit-microsoft","version":"1.2.2","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.2.2","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"8411e5a1a8a8ab41cae4402c92eca9326648d386","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.2.2.tgz","fileCount":28,"integrity":"sha512-IQ3CbU2tmUr13piimjybpIc5Oj+VpHN32HCXkJuvb6txpwnBLlkSgI+tJ7G6mmJHim4sfeeG/qrE1XhFKpsXqg==","signatures":[{"sig":"MEQCIAmu7hjHXYHtC2ACnlYi3/l2wB0LzCrHIfnhtTqDe76NAiAfvG1jnm0p/MQN/3y66QuFk7inCY+bOF1fNFZ7DTmV0g==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"0853008708fc4912c226f8f4810a3c84ce86ac26","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.2.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.2.2_1774689987478_0.015122480968892349","host":"s3://npm-registry-packages-npm-production"}},"1.2.3":{"name":"@cyberlangke/tokkit-microsoft","version":"1.2.3","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.2.3","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"e7d2e8c76af379650b2d7a998afab7ce4247b348","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.2.3.tgz","fileCount":28,"integrity":"sha512-yrxIq14zO32zvVZxGqPQV2PddX07hIdn/r5CTq1hJITLJXsTgflG17wtUO309gsYW05KIwi1cX6/lENp7fLXqg==","signatures":[{"sig":"MEQCIG8eBLwwwZQfeFGdeXyYzwVQSPosJvp4y+1nkylhmPAAAiBGIzBysLq4hrjC6nTMIbS81vOykMsx/4oUZC84+PDkHg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"9a55c7924d481a97a9059aeb9b31ec6d8c7aaa95","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.2.3"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.2.3_1774691440854_0.6474186705260789","host":"s3://npm-registry-packages-npm-production"}},"1.3.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.3.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.3.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"8ff2e3ae36da2f7a8fdc07fed6d87020f16a011a","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.3.0.tgz","fileCount":28,"integrity":"sha512-qK91/F7WX4DoBar5WKv7TJ2L4evdaTkSv/1THw+U74IRKLZy6A/v6VBY5GNKIaSiLadnAI+qSd6ER2gALntgAg==","signatures":[{"sig":"MEYCIQCwBRWgKXrXIB8aSrjnWamBp69iv0ZhSRz+M2i2eSvLLgIhAOnxuzmh1opcpC9iJ4iK2YHjVIrGIlYTMJam3L/dgA6R","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"8a2217da552ba62538637e780157ae780baf6228","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.3.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.3.0_1774694516857_0.15920474927832795","host":"s3://npm-registry-packages-npm-production"}},"1.4.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.4.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.4.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"daeeea6eba74289acb667c314144d9eb7e3171f2","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.4.0.tgz","fileCount":28,"integrity":"sha512-KuNDCrHxZQ8S8IzhMf/AAPCqEi1Q70rMa2k6GI+IAM8KN1c0RxIWrRiKoAUArV8bMHmRYMx/gwK/o2L/fVKiwA==","signatures":[{"sig":"MEUCIDHn6uq6RsolEzbUN2gwF7A48QgmhRVshoc8+A/ElfWXAiEAl6HltLqBtze5uhHy7pIbhUlV4CC7T7kcmmX1BpM4oWE=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20705948},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"4956169455ee4dd6aa70086d50da9d79ecb4232d","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.4.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.4.0_1774697331132_0.17136436593315052","host":"s3://npm-registry-packages-npm-production"}},"1.4.1":{"name":"@cyberlangke/tokkit-microsoft","version":"1.4.1","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.4.1","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"e1328c64e1caa07dd9065133ebf9dcae17276f7f","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.4.1.tgz","fileCount":28,"integrity":"sha512-DXECTOYiozGa8MF5sbIUE9Vv3hCcLd8qjqEmqt/CqN5Yr2r/pSGbj1cnz9KAV3dU4TqdF2fLqBuES/j+AKIxDQ==","signatures":[{"sig":"MEUCIQDkKtMv8nZwDYd4HifzWen8QstTVZzG2Gdjw+RllhOZ3QIgY+0dhK5siX64uQ4KzmI4Jjerfn22hV3oaj5yfTxWOkg=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20700604},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"a6dd2eb8a3c18dae80561a28114c73ed73ffbcc1","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.4.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.4.1_1774766107305_0.1619976843532538","host":"s3://npm-registry-packages-npm-production"}},"1.5.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.5.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.5.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"0a19b227c5f45efcd1bb74c6d6a648a629c89ae8","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.5.0.tgz","fileCount":28,"integrity":"sha512-MdWFO97S3KpDa1IxsZumyEQHxWr7k8Ka2WOkkoCy3uFGlb+sqLUEbjKOBWWoPPh70YISLalEqKVYQ1cA8FaC6A==","signatures":[{"sig":"MEUCIBhK8MtrNRnFmBWwOzEITB2bV833YM7/q1C1LtsH3DrRAiEAlZL730mDr0MTjTFsJNUzcVMG+TS6OdFehMWZegaeCik=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20700604},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"d6c68540588c0952d2f3d26b9a3a8f96f4069384","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.5.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.5.0_1774853682231_0.5617236573812732","host":"s3://npm-registry-packages-npm-production"}},"1.6.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.6.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.6.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"6d033b6aba9ff2409088ce6fadf8008b0ed69116","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.6.0.tgz","fileCount":28,"integrity":"sha512-KseljLTQ+z+xqHsxYvLlP6OKgTaz8ZTNIeBGSk5AYfhiZlDDX2/9jpH9qwFbqnusu9ElsNys7CW3AY7DSDmHUg==","signatures":[{"sig":"MEUCIQCOpWsYJoCpVFlcTE59xx3LturEXu1heJ+dD+8xFOnqAQIgLIhINJSifhfdbg+4V2yeUPzrzOv42bxWHABmKtBRzmY=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20700604},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"f5596e039fcd805883f5c0cdecf7ab6cc8ce0015","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.6.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.6.0_1774861764307_0.8931342766678794","host":"s3://npm-registry-packages-npm-production"}},"1.7.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.7.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.7.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"b4cda8c04b6c23674215bc457efb3bdec8fe313c","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.7.0.tgz","fileCount":28,"integrity":"sha512-DGnY9LhRsK52/hD89Eig7h60Xc5SPFriXZyvPrLgWu/DSiUvD66JeRkr6zOctwgSiqB5YiBLX5RZksv/MDQANQ==","signatures":[{"sig":"MEYCIQC6h27AckeFT6UvmsquwHUNWhvpQKajPVyEN5I87pCBHAIhAL9f8RaDcq/IY9ZpaymJZRstwDsUnf3nlxtprdio8aPY","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20700604},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"d30f8c18c181d49654ac20cb0bf2f47eb012cabb","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.7.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.7.0_1774866240255_0.8427156527681605","host":"s3://npm-registry-packages-npm-production"}},"1.8.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.8.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.8.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"d9872d177e1aebc0fcfdc1c478dea7bbaa7fa243","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.8.0.tgz","fileCount":28,"integrity":"sha512-G/v2DuBZoGy1YvKK6hiUU8kRXHiNCGmlo+6gNC19HTiqg5kb5Y5N+wZBN37zW0b/llKH75YovRi6VmH7nymteA==","signatures":[{"sig":"MEYCIQDI+5NTzKt7JLleiQkSG+jfMFGp+YBlW1FizXmHBiB5dQIhANBk8Le/ltYqE9CEYwQAkkEWhggzRBvFpApUnmspuDhf","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":20700604},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"7e5f33d0fbdb6c8184701b624b61652bea7f8a99","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.8.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.8.0_1774869320813_0.7891686401821045","host":"s3://npm-registry-packages-npm-production"}},"1.9.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.9.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.9.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"66f229c0daf3d5fd84fc7c698ddf3b40bc756a56","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.9.0.tgz","fileCount":30,"integrity":"sha512-vNoz2fu5DAuySzx9GIi8WQT+YC1h1PYWuVCF6PIPLd/QLZTdk0J+uR9ogfLnXblAHc1Wo6XztRX4Sg7AX91vjg==","signatures":[{"sig":"MEUCIEFOMIUo+EYIuu3mEIJaE2AUtUuA9eT+YnRGj9y8upSTAiEAkmO6pNtd4wNqzIAZe3ATRNwUVJXQfOSVj8uU3NJIZ3c=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":24787972},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"966b217b2c25444ce482773a6fedf7b5b9d24608","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.9.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.9.0_1774872840281_0.07108820958472495","host":"s3://npm-registry-packages-npm-production"}},"1.10.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.10.0","author":{"name":"cyberlangke"},"license":"MIT","_id":"@cyberlangke/tokkit-microsoft@1.10.0","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"dist":{"shasum":"7d8bc4d9c6a3b6fb78e97f345e641488eaef2db7","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.10.0.tgz","fileCount":30,"integrity":"sha512-zkQMqwrLuuFsmt02Q0WF+9IDL8wHr7g6r4wjSrNV7ZeS4bSmk7mYQCdDIigdqNlT2gR66GAV4Ri+MHRjB4n2sQ==","signatures":[{"sig":"MEQCICR4GI2agvmo9fCpdgMO4QN4WHE7TnaDZlwrLUSfohVCAiBy2yNcR80H4hCWEbsocxgYOAm+pv7W8jhfDRNPCfApMg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":24787974},"main":"./dist/index.cjs","type":"module","types":"./dist/index.d.ts","module":"./dist/index.js","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"gitHead":"c086a200b2f732709de752de4b7120a28d55cfd2","scripts":{"dev":"tsup --watch","build":"tsup"},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"repository":{"url":"git+https://github.com/1cyberlangke1/tokkit.git","type":"git"},"_npmVersion":"11.9.0","description":"Microsoft tokenizer families for tokkit.","directories":{},"_nodeVersion":"24.14.0","dependencies":{"@cyberlangke/tokkit-core":"1.10.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/tokkit-microsoft_1.10.0_1774875397685_0.7827995626556119","host":"s3://npm-registry-packages-npm-production"}},"1.11.0":{"name":"@cyberlangke/tokkit-microsoft","version":"1.11.0","description":"Microsoft tokenizer families for tokkit.","type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js","require":"./dist/index.cjs"}},"scripts":{"build":"tsup","dev":"tsup --watch"},"dependencies":{"@cyberlangke/tokkit-core":"1.11.0"},"publishConfig":{"access":"public"},"repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"homepage":"https://github.com/1cyberlangke1/tokkit#readme","bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"MIT","gitHead":"6d21c3bb1df377d5de599859bf065685223c2dc3","_id":"@cyberlangke/tokkit-microsoft@1.11.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-W9SJPG2Wh5AJzIWNZRUaSZvFDx6ankz2H7U4O/rmJQM0lfeI+krHPSGvYIrRiyROz9tBL+9xPLnTy/O5BEhO9w==","shasum":"0f1953c1e61b3d06bb8e92db1e6ec09a9c1687d0","tarball":"https://registry.npmjs.org/@cyberlangke/tokkit-microsoft/-/tokkit-microsoft-1.11.0.tgz","fileCount":30,"unpackedSize":24787974,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCgseMCQ+CW8ZkgPIROEok9St9j0yEmw2XQI8kmJuoyBQIgFJt995YwBVWLkdmmB8hXcYexYV1l+DqNrMtBJdctQwE="}]},"_npmUser":{"name":"cyberlangke","email":"3531702083@qq.com"},"directories":{},"maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/tokkit-microsoft_1.11.0_1774877967214_0.8068204441320237"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-28T07:41:37.580Z","modified":"2026-03-30T13:39:27.973Z","1.1.0":"2026-03-28T07:41:38.204Z","1.2.1":"2026-03-28T09:05:34.201Z","1.2.2":"2026-03-28T09:26:28.031Z","1.2.3":"2026-03-28T09:50:41.385Z","1.3.0":"2026-03-28T10:41:57.481Z","1.4.0":"2026-03-28T11:28:51.654Z","1.4.1":"2026-03-29T06:35:07.813Z","1.5.0":"2026-03-30T06:54:42.734Z","1.6.0":"2026-03-30T09:09:24.825Z","1.7.0":"2026-03-30T10:24:00.811Z","1.8.0":"2026-03-30T11:15:21.309Z","1.9.0":"2026-03-30T12:14:01.047Z","1.10.0":"2026-03-30T12:56:38.297Z","1.11.0":"2026-03-30T13:39:27.837Z"},"bugs":{"url":"https://github.com/1cyberlangke1/tokkit/issues"},"author":{"name":"cyberlangke"},"license":"MIT","homepage":"https://github.com/1cyberlangke1/tokkit#readme","repository":{"type":"git","url":"git+https://github.com/1cyberlangke1/tokkit.git"},"description":"Microsoft tokenizer families for tokkit.","maintainers":[{"name":"cyberlangke","email":"3531702083@qq.com"}],"readme":"# @cyberlangke/tokkit-microsoft\n\nMicrosoft 官方 tokenizer 的 tokkit 子包，只包含当前 MIT 兼容、且能用官方 tokenizer 资产直接验证的纯文本 tokenizer。\n\n## 支持的 family\n\n- `bitnet-b1.58-2b-4t`：覆盖 `bitnet-b1.58-2B-4T`\n- `phi-1`：覆盖 `phi-1 / phi-1_5 / phi-2`\n- `phi-3-mini`：覆盖 `Phi-3-mini-4k-instruct / Phi-3-mini-128k-instruct`\n- `phi-3-medium`：覆盖 `Phi-3-medium-4k-instruct / Phi-3-medium-128k-instruct`\n- `phi-3.5`：覆盖 `Phi-3.5-mini-instruct / Phi-3.5-MoE-instruct`\n- `phi-moe`：覆盖 `Phi-mini-MoE-instruct / Phi-tiny-MoE-instruct`\n- `phi-4`：覆盖 `phi-4`\n- `phi-4-mini`：覆盖 `Phi-4-mini-instruct`\n- `phi-4-mini-reasoning`：覆盖 `Phi-4-mini-reasoning`\n- `phi-4-mini-flash`：覆盖 `Phi-4-mini-flash-reasoning`\n- `phi-4-reasoning`：覆盖 `Phi-4-reasoning / Phi-4-reasoning-plus`\n\n## 当前不纳入\n\n- `Phi-3-small-8k-instruct`\n- `Phi-3-small-128k-instruct`\n\n这两条官方文本主线仓库当前仍然没有直接公开根目录 `tokenizer.json`，不满足当前“下载官方 tokenizer 快照并直接对拍”的接入门槛。\n\n- `Phi-3-vision-128k-instruct`\n- `Phi-3.5-vision-instruct`\n- `Phi-4-multimodal-instruct`\n- `Phi-4-reasoning-vision-15B`\n\n这些模型属于 vision / multimodal 路线，不在当前纯文本 BPE 主线范围内。\n\n- `NextCoder-*`\n- `UserLM-8b`\n- `FrogBoss-*`\n- `FrogMini-*`\n\n这些仓库当前都带 `base_model:finetune` 信号，属于官方微调 / 派生模型，不作为当前主线 tokenizer 的维护目标。\n\n- `*-onnx`\n- `*-gguf`\n- `*-pytdml`\n\n这些仓库属于导出格式或衍生分发，不作为当前官方主线 tokenizer 的维护目标。\n\n## 使用方法\n\n```bash\nnpm install @cyberlangke/tokkit-microsoft\n```\n\n```ts\nimport { getTokenizer } from \"@cyberlangke/tokkit-microsoft\"\n\nconst tokenizer = await getTokenizer(\"bitnet-b1.58-2b-4t\")\n\nconsole.log(tokenizer.vocabSize)\n```\n","readmeFilename":"README.md"}