{"_id":"@aiblox/xform","name":"@aiblox/xform","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@aiblox/xform","version":"0.1.0","description":"Context optimization engine for LLM-ready structured data","type":"module","main":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"}},"publishConfig":{"access":"public"},"scripts":{"build":"tsc -p tsconfig.build.json","test":"vitest run","test:watch":"vitest","docs:sync":"npm run build && node scripts/sync-usage-examples.mjs","prepublishOnly":"npm run build"},"keywords":["llm","context","optimization","json","toon","token-efficiency"],"license":"MIT","engines":{"node":">=18"},"dependencies":{"@toon-format/toon":"^2.3.0","fast-xml-parser":"^5.2.5"},"devDependencies":{"@types/node":"^22.15.21","typescript":"^5.8.3","vitest":"^3.1.4"},"_id":"@aiblox/xform@0.1.0","gitHead":"546644d2d23ee29487ade7af273ebd3e4fc06a7d","_nodeVersion":"20.20.2","_npmVersion":"10.8.2","dist":{"integrity":"sha512-cG5MDzk9c5NBKInadA1j7dSaaxiKPkNNXIjWzzy3wiESgCHG4w1XRpPoGazxbuyMYAyVW4NneqiZoc2CcN8GnA==","shasum":"80102af5647f19ef781d3611d6b9974b294bb63c","tarball":"https://registry.npmjs.org/@aiblox/xform/-/xform-0.1.0.tgz","fileCount":79,"unpackedSize":121970,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQD5daHCxttqucui+EIiT33lb9yzIL/fr60FeeCDezbrMAIgIoqnznjyxHvH1K/itDTSUpTdTiq+02wRlrcSWXPWpsQ="}]},"_npmUser":{"name":"metinsaylan","email":"metinsaylan@gmail.com"},"directories":{},"maintainers":[{"name":"metinsaylan","email":"metinsaylan@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/xform_0.1.0_1779580581535_0.784018078538073"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-23T23:56:21.365Z","0.1.0":"2026-05-23T23:56:21.692Z","modified":"2026-05-23T23:56:21.970Z"},"maintainers":[{"name":"metinsaylan","email":"metinsaylan@gmail.com"}],"description":"Context optimization engine for LLM-ready structured data","keywords":["llm","context","optimization","json","toon","token-efficiency"],"license":"MIT","readme":"# @aiblox/xform\n\nContext optimization engine for LLM-ready structured data. Converts JSON, XML, CSV, and TSV into compact, high-signal output formats.\n\nThis is **not** a general file-format library — it scans, reduces, transforms, and describes data specifically for AI context windows.\n\n## Install\n\n```bash\nnpm install @aiblox/xform\n```\n\n## Quick start\n\n```typescript\nimport { xform } from '@aiblox/xform';\n\nconst largeJsonData = [\n  { id: 1, tenant: 'acme', name: 'Ada', status: 'active', note: null },\n  { id: 2, tenant: 'acme', name: 'Bob', status: 'active', note: null },\n  ...\n];\n\nconst context = await xform(largeJsonData);\n```\n\nSample output:\n```\nDataset with {X} records and {Y} columns.\nConstant across all records: status=\"active\"; tenant=\"acme\".\n\nRecords:\n  [X|]{id|name|region|score}:\n    1|Ada|us-east|10\n    2|Bob|us-east|12\n    3|Cora|eu-west|99\n    ...\n```\n\n`xform` is an alias for `transform`. See **[USAGE.md](./USAGE.md)** for examples of every output format with sample inputs and outputs.\n\n## API\n\n| Function | Description |\n|----------|-------------|\n| `xform(input, options)` | Alias for `transform` — full pipeline → `context`, `json_compact`, or `toon` |\n| `transform(input, options)` | Full pipeline → `context`, `json_compact`, or `toon` |\n| `scan(input, options)` | Column profiles, types, constants, null ratios |\n| `reduce(input, options)` | Remove null-only columns, collapse constants |\n| `describe(input, options)` | Concise natural-language data summary |\n| `toJsonCompact(input, options)` | Minified JSON with metadata |\n| `toToon(input, options)` | TOON-encoded output (tabular-friendly) |\n| `toDSV(data, delimiter)` | Delimiter-separated values (records or pipeline result) |\n| `toCSV` / `toTSV` / `toPSV` | `toDSV` with `,`, tab, or pipe delimiters |\n| `fromJson` / `fromXml` / `fromCsv` / `fromTsv` | Parse inputs to record arrays |\n\n## Options\n\n```typescript\ninterface TransformOptions {\n  output?: 'context' | 'json_compact' | 'toon';\n  /** TOON field separator: `|`, `,`, tab, or `pipe` / `comma` / `tab`. Default `|`. */\n  delimiter?: string;\n  schema?: SchemaDefinition[];\n  hints?: { groupby?: string[] };\n  compact?: boolean;\n  preserveOutliers?: boolean;\n  includeStats?: boolean;\n  format?: 'json' | 'xml' | 'csv' | 'tsv';\n}\n```\n\n### Schema\n\nSchemas are JSON arrays with `name`, optional `_extends`, and nested `_type`:\n\n```typescript\nconst schemas = [\n  { name: 'Base', status: 'string' },\n  { name: 'User', _extends: 'Base', email: 'string' },\n];\n\nawait transform(records, { schema: schemas });\n```\n\n### Grouping\n\nRecord grouping runs **only** when you pass explicit hints — no fuzzy clustering by default:\n\n```typescript\nawait transform(records, {\n  hints: { groupby: ['department'] },\n});\n```\n\n## Pipeline\n\n1. **Scan** — column types, null ratios, cheap constant detection\n2. **Reduce** — drop null-only columns, collapse constants, summarize repeats\n3. **Transform** — schema-aware normalization\n4. **Describe** — token-efficient natural language summary\n\nIf the final output is **longer** than the serialized input, results automatically **fall back to the original** (token safety). Disable with `fallbackToOriginal: false`.\n\n## License\n\nMIT\n","readmeFilename":"README.md","_rev":"1-bddae6d43d87a559fa68f368cbc4f539"}