{"_id":"@dornick/parsers-ml","name":"@dornick/parsers-ml","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@dornick/parsers-ml","version":"0.1.0","description":"Lazy transformers.js wrappers for Dornick — local embeddings, NER, summarisation. WebGPU preferred, WASM fallback.","type":"module","main":"./dist/index.js","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"}},"scripts":{"build":"tsup","dev":"tsup --watch","test":"vitest run","typecheck":"tsc --noEmit","clean":"rm -rf dist","size":"size-limit"},"dependencies":{"@dornick/capabilities":"0.1.0","@dornick/parsers":"0.1.0","@huggingface/transformers":"^3.0.2"},"devDependencies":{"@size-limit/file":"^11.1.6","size-limit":"^11.1.6","tsup":"^8.3.0","typescript":"^5.5.4","vitest":"^4.1.7"},"license":"MIT","sideEffects":false,"publishConfig":{"access":"public"},"size-limit":[{"name":"@dornick/parsers-ml (gz)","path":"dist/index.js","limit":"1.5 KB","gzip":true}],"gitHead":"622e4ebed9c3aebadd808a2dd1f5f29c7d226040","_id":"@dornick/parsers-ml@0.1.0","_nodeVersion":"26.7.0","_npmVersion":"11.19.0","dist":{"integrity":"sha512-xnMZZC/W7lGxMz/65b758iBu5u64Zz6f82oenf6QlfQlq12Y2wuNjU0N+MqsWYAI06hyFPzCB8wPRl87Ub5lSg==","shasum":"91b1b8ee9f2624121b98ccb0540e1222af29f4c5","tarball":"https://registry.npmjs.org/@dornick/parsers-ml/-/parsers-ml-0.1.0.tgz","fileCount":7,"unpackedSize":19920,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQCSn8sT74kl9IQuDrhyT4mq/fEu11jBYSIFaYF699BxNwIhAJ7u/QGU5q/ZfZoU59+P+zqWEjDuM9FkFg+er+LVmNUX"}]},"_npmUser":{"name":"dipankarsarkar","email":"me@dipankar.name"},"directories":{},"maintainers":[{"name":"dipankarsarkar","email":"me@dipankar.name"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/parsers-ml_0.1.0_1788784934948_0.25646674691081106"},"_hasShrinkwrap":false}},"time":{"created":"2026-09-07T12:42:14.593Z","0.1.0":"2026-09-07T12:42:15.083Z","modified":"2026-09-07T12:42:15.487Z"},"maintainers":[{"name":"dipankarsarkar","email":"me@dipankar.name"}],"description":"Lazy transformers.js wrappers for Dornick — local embeddings, NER, summarisation. WebGPU preferred, WASM fallback.","license":"MIT","readme":"# @dornick/parsers-ml\n\n**Lazy transformers.js wrappers for Dornick — local embeddings, NER, summarisation.**\n\nThree text-only parsers. WebGPU preferred (auto-detected); falls back to WASM. Model downloads are cached by the browser.\n\n[![npm](https://img.shields.io/npm/v/@dornick/parsers-ml.svg?style=flat&color=4c1)](https://www.npmjs.com/package/@dornick/parsers-ml)\n[![license](https://img.shields.io/npm/l/@dornick/parsers-ml.svg?style=flat&color=blue)](../../LICENSE)\n\n## Install\n\n```sh\nnpm i @dornick/parsers-ml\n```\n\nDepends on [`@huggingface/transformers`](https://huggingface.co/docs/transformers.js). The library is ~1-2 MB and the per-task models 80-300 MB on first use, cached by the browser thereafter.\n\n## Use\n\n```ts\nimport { Dornick } from \"@dornick/core\";\nimport { embeddingsParser, nerParser, summaryParser } from \"@dornick/parsers-ml\";\n\nconst dornick = new Dornick({ config, transport });\ndornick.parsers.add(embeddingsParser);\ndornick.parsers.add(nerParser);\ndornick.parsers.add(summaryParser);\n```\n\n## Paths produced\n\nFor any text-like upload (`text/*`, `.md`, `.json`):\n\n| Path | Content |\n|---|---|\n| `embeddings.jsonl` | One JSON object per ~1.2KB chunk: `{ chunk, vector }` (MiniLM, 384-dim, mean-pooled + L2-normalised) |\n| `ner.jsonl` | Detected entity spans: `{ entity_group, word, score, start, end }` |\n| `summary.txt` | Short summary (~80 tokens) via DistilBART CNN |\n\n## Models\n\n| Parser | Default model |\n|---|---|\n| `embeddingsParser` | `Xenova/all-MiniLM-L6-v2` |\n| `nerParser` | `Xenova/bert-base-NER` |\n| `summaryParser` | `Xenova/distilbart-cnn-6-6` |\n\nThese models are first-class compatible with `@huggingface/transformers`. The model files load from the Hugging Face CDN on first parse and cache via the browser's HTTP cache. For offline / strict-CSP deployments, vendor the model files yourself and configure transformers.js's `env` accordingly.\n\n## Capability gate\n\n`requires: [\"wasm\"]` — all three parsers run anywhere WebAssembly is supported. WebGPU is preferred and used automatically when available (`navigator.gpu`); otherwise the parser falls back to WASM (slower but identical results).\n\n## Pairs nicely with\n\n- [`@dornick/parsers-pdf`](https://www.npmjs.com/package/@dornick/parsers-pdf) — pipe PDF-extracted text through embeddings for local semantic search.\n- [`@dornick/parsers-archive`](https://www.npmjs.com/package/@dornick/parsers-archive) — index entries inside a Takeout / Slack export.\n\n## See also\n\n- [Concept: Parsers](https://docs.neullabs.com/dornick/concepts/parsers/)\n- [Use case: Personal data explorer](https://docs.neullabs.com/dornick/use-cases/personal-data-explorer/)\n\n## License\n\nMIT — © Dipankar Sarkar\n","readmeFilename":"README.md","_rev":"1-689f91c5c8828e57b8b41f1f6503c63d"}