{"_id":"@1qh/ragcore","_rev":"2-07d20b77375fa20d39a9c12ce16013fb","name":"@1qh/ragcore","dist-tags":{"latest":"0.0.0"},"versions":{"0.0.0":{"name":"@1qh/ragcore","version":"0.0.0","author":{"name":"1qh"},"license":"Apache-2.0","_id":"@1qh/ragcore@0.0.0","maintainers":[{"name":"1qh","email":"laiquanghuy24122001@gmail.com"}],"homepage":"https://github.com/1qh/ragcore#readme","bugs":{"url":"https://github.com/1qh/ragcore/issues"},"dist":{"shasum":"c42b945b9fbb2b21d27ee08f5d5ee5ceee373be1","tarball":"https://registry.npmjs.org/@1qh/ragcore/-/ragcore-0.0.0.tgz","fileCount":15,"integrity":"sha512-OKbuIMSmu5hSbzeSxxba7f15OVPH4BLBj3JLwjf/++IpTWfOr7YpumYWmFR71xIrd6M8pjAcrBDCK5joy2RkjA==","signatures":[{"sig":"MEYCIQD+Zqt/5ZOfj+rTnrSqpBnjpEPy8VyIeccFz+7l2lpJEwIhAP+sk+wWIByp4/ZFto1RmaV7uQRxMTZEi8SKtDoqcH3T","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":245651},"main":"./dist/index.mjs","type":"module","types":"./dist/index.d.mts","exports":{".":{"types":"./dist/index.d.mts","default":"./dist/index.mjs"}},"gitHead":"d5cad57cf2229d228943f9263da75f1573f99ee3","scripts":{"build":"tsdown","prepublishOnly":"tsdown"},"_npmUser":{"name":"1qh","email":"laiquanghuy24122001@gmail.com"},"repository":{"url":"git+https://github.com/1qh/ragcore.git","type":"git"},"_npmVersion":"11.17.0","description":"The RAG engine minus the store: multi-engine document understanding with per-page routing, chunking, provenance that survives chunking, embedding, and retrieval logic.","directories":{},"sideEffects":false,"_nodeVersion":"26.5.0","dependencies":{"ai":"latest","zod":"latest","pino":"latest","mupdf":"latest","exceljs":"latest","cockatiel":"latest","smol-toml":"latest","@mastra/rag":"latest","node-html-parser":"latest","@opentelemetry/api":"latest","fractional-indexing":"latest","google-auth-library":"latest","@docling/docling-core":"latest","@opentelemetry/resources":"latest","@ai-sdk/openai-compatible":"latest","@flatten-js/interval-tree":"latest","@opentelemetry/sdk-trace-base":"latest","@opensearch-project/opensearch":"latest","@opentelemetry/semantic-conventions":"latest","@opentelemetry/exporter-trace-otlp-http":"latest"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/ragcore_0.0.0_1785217013711_0.6326133285115665","host":"s3://npm-registry-packages-npm-production"},"deprecated":"Use the unscoped package: npm i ragworks"}},"time":{"created":"2026-07-28T05:36:53.522Z","modified":"2026-07-28T07:29:46.070Z","0.0.0":"2026-07-28T05:36:53.848Z"},"bugs":{"url":"https://github.com/1qh/ragcore/issues"},"author":{"name":"1qh"},"license":"Apache-2.0","homepage":"https://github.com/1qh/ragcore#readme","repository":{"url":"git+https://github.com/1qh/ragcore.git","type":"git"},"description":"The RAG engine minus the store: multi-engine document understanding with per-page routing, chunking, provenance that survives chunking, embedding, and retrieval logic.","maintainers":[{"name":"1qh","email":"laiquanghuy24122001@gmail.com"}],"readme":"# rag-ingest\n\nTurn a document into retrievable chunks that still know where they came from.\n\nMost RAG stacks treat ingest as a preamble: extract some text, split it every N characters, embed. That is where the answers are lost. A scanned page returns nothing, a table’s value slides one column, and a citation points at a passage no reader can find on the page. This package is the ingest half done properly — per-page engine routing, parsing, chunking, and an offset-to-region bridge that keeps every chunk anchored to the pixels it came from.\n\n## What this is not\n\nIt is not a whole RAG engine, and the name says so. Retrieval — query understanding, hybrid search, fusion, reranking, the graph index — needs a store interface and a record store, and neither belongs in a library like this. Bring your own index; this package decides what goes into it.\n\n## Why the ingest half is worth its own package\n\n- **Per-page routing, not per-document.** A mixed document has clean pages and scanned ones. One engine for the whole file caps quality on the rest. Each page is scored on four orthogonal signals — character count, control-character ratio, script validity, and already-decoded mojibake — and escalated to a vision model only when its text layer is genuinely unusable.\n- **The corruption signal measures corruption.** The control-character ratio excludes the C0 layout whitespace every text layer carries by the line. Counting `\\n` measures line density instead, which makes the densest table on the page look like the most corrupt one — across a sampled corpus that mistake escalated all 39 pages when 9 needed it, and the vision model then rewrote text the source never contained.\n- **Provenance survives chunking.** `buildChunks` returns each chunk’s character span into the markdown _and_ its regions on the page, joined by an interval tree over the parser’s element geometry. That join is the one capability here no library owns.\n- **A page assigned to an absent engine still gets read.** If a structure engine is not configured, its pages re-route to the vision model rather than silently keeping the parse the router already rejected.\n\n## Install\n\n```sh\nbun add @1qh/ragcore      # or npm / pnpm\n```\n\nNeeds a [docling](https://github.com/docling-project/docling) service for parsing, and a provider registry file naming your OpenAI-compatible endpoints for embedding. Both are configuration, not vendors: any OpenAI-compatible host works, local or managed.\n\n## Use\n\n```ts\nimport { buildChunks, configureEngine, parseDocument } from '@1qh/ragcore'\n\nconfigureEngine({\n  DOCLING_URL: 'http://localhost:5001',\n  PROVIDERS_FILE: './providers.toml'\n})\n\nconst parsed = await parseDocument({ bytes, name: 'policy.pdf' })\nconst chunks = await buildChunks({\n  blocks: parsed.blocks ?? [],\n  markdown: parsed.markdown,\n  maxSize: 800,\n  overlap: 120,\n  strategy: 'recursive'\n})\n\nfor (const c of chunks) console.log(c.text, c.charspan, c.regions)\n```\n\nEvery step stands alone. If you already parse your own documents, take only `buildChunks`. If you already chunk, take only `locateChunks` and `buildRegionIndex` — the provenance bridge works on any markdown plus any block geometry.\n\n`configureEngine` fails fast, by name, on the two values the pipeline cannot run without. It never substitutes a default, because a pipeline pointed at the wrong service reports success.\n\n## Verify it yourself\n\n```sh\nbun smoke.ts <path-to-document>\n```\n\nDrives the public API against real services and fails if a spatial parse produces chunks with no page regions.\n\n## Maintenance\n\nThis is a living project: the code runs in production and keeps moving, and issues get answered. It is published under Apache-2.0.\n\nTwo honest caveats. The Vietnamese-language behaviour is the best-measured part, because that is the corpus it was built against; other languages are expected to work and are not equally measured. And the routing thresholds are defaults drawn from one corpus — they are exported so you can measure your own rather than inherit ours.\n","readmeFilename":"README.md"}