{"_id":"@bureau-of-weights-and-measures/content-distillation","_rev":"2-7a34698ca8aaff3fee4259c24b061461","name":"@bureau-of-weights-and-measures/content-distillation","dist-tags":{"latest":"0.2.1"},"versions":{"0.2.0":{"name":"@bureau-of-weights-and-measures/content-distillation","version":"0.2.0","license":"MIT","_id":"@bureau-of-weights-and-measures/content-distillation@0.2.0","maintainers":[{"name":"hwclass","email":"harderworking.class@gmail.com"}],"dist":{"shasum":"a21a078e0181b51cd224ebfb288e3f621218f1ee","tarball":"https://registry.npmjs.org/@bureau-of-weights-and-measures/content-distillation/-/content-distillation-0.2.0.tgz","fileCount":14,"integrity":"sha512-wug+W+CaK9ko36w0DRLfd2BjbpnP7Y9A36p/NE0iaZQKYpVIJ3MOiQh+06rpsq7sNehQMpjjDpxf0OaxLt6b+g==","signatures":[{"sig":"MEUCIQDiBVeQnzlf25qGxw1zDfX3b2MeSytXBnvZjbzl7xVZSQIgIjxlgShhpifFw4scB2nspv7Nl6bC0atO8wQrjchZPxY=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":953935},"main":"./distill.js","type":"module","exports":{".":"./distill.js"},"_npmUser":{"name":"hwclass","email":"harderworking.class@gmail.com"},"_npmVersion":"6.14.15","description":"Deterministic content signal extraction for agentic web environments.","directories":{},"_nodeVersion":"14.17.6","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/content-distillation_0.2.0_1772714210817_0.6427045499952333","host":"s3://npm-registry-packages-npm-production"}},"0.2.1":{"name":"@bureau-of-weights-and-measures/content-distillation","version":"0.2.1","description":"Deterministic content signal extraction for agentic web environments.","license":"MIT","type":"module","main":"./distill.js","exports":{".":"./distill.js"},"gitHead":"cb5041eeccd51445eaf91d8b92cc3b3889e7ebf0","_id":"@bureau-of-weights-and-measures/content-distillation@0.2.1","_nodeVersion":"14.17.6","_npmVersion":"6.14.15","dist":{"integrity":"sha512-o66aW44aFRmsEeBubt4uMV1nR6Pl+OA8zzppyN8kBhZnJlXT6ipz78J0AaPuDTt6qnk8k6JCQOsszKA5bry+Sw==","shasum":"bb092d54053179d9ba9d8dcb5c195216bd2d0cf1","tarball":"https://registry.npmjs.org/@bureau-of-weights-and-measures/content-distillation/-/content-distillation-0.2.1.tgz","fileCount":14,"unpackedSize":954982,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIH52xDKyRsZIVxt7s4MDMNMrlbEqmxIHV8QEXMYxG9TuAiBtceoE6FYY8wM4SMCvC9Ep7Edbt0cFmiYNlSMSbtySVg=="}]},"_npmUser":{"name":"hwclass","email":"harderworking.class@gmail.com"},"directories":{},"maintainers":[{"name":"hwclass","email":"harderworking.class@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/content-distillation_0.2.1_1772715315098_0.02848420530098905"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-05T12:36:50.677Z","modified":"2026-03-05T12:55:15.406Z","0.2.0":"2026-03-05T12:36:51.048Z","0.2.1":"2026-03-05T12:55:15.290Z"},"license":"MIT","description":"Deterministic content signal extraction for agentic web environments.","maintainers":[{"name":"hwclass","email":"harderworking.class@gmail.com"}],"readme":"# @bureau-of-weights-and-measures/content-distillation\n\nDeterministic HTML → Markdown conversion for AI agents.\n\nExtracts the primary content signal from web pages and converts it into **clean Markdown optimized for LLM consumption**.\n\nPart of the **Bureau of Weights and Measures** ecosystem.\n\n---\n\n## Install\n\nRun:\n\n```bash\nnpm install @bureau-of-weights-and-measures/content-distillation\n```\n\n---\n\n## Usage\n\n```js\nimport { distill } from \"@bureau-of-weights-and-measures/content-distillation\"\n\nconst result = await distill(document)\n```\n\n---\n\n## Overview\n\nThis package extracts the **primary content signal** from web pages and converts it into structured **Markdown suitable for AI agents, crawlers, and LLM systems**.\n\nInstead of parsing noisy HTML, agents can request Markdown directly.\n\nFeatures:\n\n- deterministic HTML → Markdown conversion\n- WASM-powered pipeline (Rust → `wasm32-unknown-unknown`)\n- browser, worker, and edge compatible\n- optional ONNX intent classification — local inference, no external services\n- fully inspectable\n\n---\n\n## Agent access via `Accept` header\n\nDrop in one script tag.\n\nAny HTTP client that sends:\n\n```sh\nAccept: text/markdown\n```\n\nwill receive **clean Markdown instead of HTML**.\n\nHuman visitors in a browser see nothing different.\n\n```html\n<script type=\"module\" src=\"./distill.js\" data-wasm-url=\"./dist/wasm/distill_core_bg.wasm\" data-onnx-url=\"./models/intent.onnx\" data-workers=\"2\"> </script>\n```\n\nExample:\n\n```sh\ncurl https://yoursite.com/page\n -H \"Accept: text/markdown\"\n\nResponse headers:\n\nContent-Type: text/markdown\nX-Markdown-Tokens: 465\nX-Conversion-Ms: 3\nX-Actions: 1\n```\n\nThe response body contains the **distilled Markdown representation of the page**.\n\n---\n\n## Example output\n\nArticle Title\n\nThis is the primary content of the page...\n\nSection\n\nMore text...\n\n---\n\n## Bureau ecosystem\n\nThis package implements the **content signal detection layer** within the **Bureau of Weights and Measures** measurement architecture.  \n\nIt is the first primitive in a growing set of standards and instruments for **agentic web measurement**, including:\n\n- agent discovery\n- bot detection\n- content permission signals\n- content freshness\n- interaction traces\n- automation scoring\n\n---\n\n## Notes\n\n- Designed for **AI agents, crawlers, and LLM systems**.\n- Fully deterministic — repeated distillation of the same page produces identical Markdown.\n- Works in browser, Service Worker, and edge environments.\n- Optional ONNX intent classifier enables **local semantic understanding** of content.","readmeFilename":"README.md"}