{"_id":"@ayushsoam51/llmx","name":"@ayushsoam51/llmx","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@ayushsoam51/llmx","version":"0.1.0","description":"Lightweight, provider-neutral LLM API execution runtime","type":"module","sideEffects":false,"engines":{"node":">=22"},"exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"},"./providers/*":{"types":"./dist/providers/*.d.ts","import":"./dist/providers/*.js"},"./telemetry/*":{"types":"./dist/telemetry/*.d.ts","import":"./dist/telemetry/*.js"},"./testing":{"types":"./dist/testing.d.ts","import":"./dist/testing.js"}},"bin":{"llmx":"cli/index.mjs"},"scripts":{"build":"tsc -p tsconfig.json","test":"npm run build && npm run typecheck && node --test test/*.test.mjs","benchmark":"npm run build && node benchmarks/runtime.mjs","prepack":"npm run build","typecheck":"tsc -p tsconfig.test.json","docs":"npm run build && node scripts/api-docs.mjs","format":"prettier --write src test cli examples scripts benchmarks","format:check":"prettier --check src test cli examples scripts benchmarks"},"devDependencies":{"prettier":"^3.9.6","typescript":"~5.9.3","valibot":"^1.5.0","zod":"^4.6.2"},"repository":{"type":"git","url":"git+https://github.com/HriGrit/LLMX.git"},"license":"MIT","gitHead":"b35610fcb568f519c7f1d2a80b8bf799b7eb9931","_id":"@ayushsoam51/llmx@0.1.0","bugs":{"url":"https://github.com/HriGrit/LLMX/issues"},"homepage":"https://github.com/HriGrit/LLMX#readme","_nodeVersion":"25.2.1","_npmVersion":"11.6.2","dist":{"integrity":"sha512-Sqh8pZ0unoH4gS5Sh7FpEWGyDTere2g9cQN56AiUC63A97h3CfsooEDAWrqKSU4y05TH4SOsQb4O53UK30R20g==","shasum":"61f22ae045de57793c18e7cac722bed1a2605b2f","tarball":"https://registry.npmjs.org/@ayushsoam51/llmx/-/llmx-0.1.0.tgz","fileCount":69,"unpackedSize":307416,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQCRLRT0/LSepBsVpAlcoPt5NbQnwdQQNWfEM5dsdwu97AIhAKj9dr/LiL5oy7hzQcjPIhZcSf12dfGFk5TMQSws9Lrp"}]},"_npmUser":{"name":"ayushsoam51","email":"ayushsoam15@gmail.com"},"directories":{},"maintainers":[{"name":"ayushsoam51","email":"ayushsoam15@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/llmx_0.1.0_1789198243953_0.3251181273829118"},"_hasShrinkwrap":false}},"time":{"created":"2026-09-12T07:30:43.830Z","0.1.0":"2026-09-12T07:30:44.103Z","modified":"2026-09-12T07:30:44.330Z"},"maintainers":[{"name":"ayushsoam51","email":"ayushsoam15@gmail.com"}],"description":"Lightweight, provider-neutral LLM API execution runtime","homepage":"https://github.com/HriGrit/LLMX#readme","repository":{"type":"git","url":"git+https://github.com/HriGrit/LLMX.git"},"bugs":{"url":"https://github.com/HriGrit/LLMX/issues"},"license":"MIT","readme":"# LLMX\n\nA lightweight TypeScript runtime for executing LLM API calls with consistent scheduling, retries, validation, wire inspection, usage accounting, and tracing.\n\n**Status: initial v0.1 implementation, not yet published to npm.** The repository implements the local runtime slice of the product requirements. Production controls and remaining release gates are tracked in [scope](docs/SCOPE.md). No model rankings, automatic model recommendations, mandatory proxy, or mandatory infrastructure.\n\n## Run locally\n\nRequires Node.js 22 or newer.\n\n```sh\ngit clone https://github.com/HriGrit/LLMX.git\ncd LLMX\nnpm ci\nnpm test\nnpm run benchmark\nnpm pack\n```\n\nInstall the resulting `ayushsoam51-llmx-0.1.0.tgz` into your application. The package is published under Ayush's verified npm scope: `@ayushsoam51/llmx`.\n\nThe core has **zero runtime dependencies**. TypeScript, Prettier, Zod, and Valibot are development tools; the two validation libraries are used only to verify interoperability.\n\n## One gateway file\n\n```ts\nimport { createGateway } from '@ayushsoam51/llmx';\nimport { openai } from '@ayushsoam51/llmx/providers/openai';\nimport { gemini } from '@ayushsoam51/llmx/providers/gemini';\n\nexport const llm = createGateway({\n  providers: {\n    google: gemini({ apiKey: process.env.GEMINI_API_KEY! }),\n    openai: openai({ apiKey: process.env.OPENAI_API_KEY! }),\n  },\n  models: {\n    main_model: { provider: 'google', model: process.env.MAIN_MODEL!, fallback: ['backup_model'] },\n    eval_model: { provider: 'google', model: process.env.EVAL_MODEL! },\n    backup_model: { provider: 'openai', model: process.env.BACKUP_MODEL! },\n  },\n  limits: { concurrency: 20, rpm: 600, tpm: 1_000_000, queueSize: 1000 },\n  reliability: { timeoutMs: 30_000, maxAttempts: 3 },\n  logging: { payloads: 'metadata' },\n});\n\nconst response = await llm.generate({\n  model: 'main_model',\n  messages: [{ role: 'user', content: 'Explain token reservation in one sentence.' }],\n  maxOutputTokens: 200,\n});\nconsole.log(response.text, response.usage, response.attempts);\n```\n\nAliases are opaque application-owned mappings. A new model identifier needs no package release when the provider API remains compatible. Direct calls also work:\n\n```ts\nawait llm.generate({ provider: 'google', model: process.env.EVAL_MODEL!, messages });\n```\n\n## Streaming\n\n```ts\nconst stream = llm.stream({ model: 'main_model', messages, signal: controller.signal });\nfor await (const text of stream.textStream()) process.stdout.write(text);\nconst final = await stream.final();\n```\n\nAlternatively iterate canonical events or call `final()` directly to drain the stream. Streams are lazy, single-consumer, and bounded by `maxResponseBytes`. Breaking iteration closes the transport. Once an event has been exposed, failures never silently retry or switch providers.\n\n## Structured output\n\nUse raw JSON Schema, a Standard Schema validator such as Zod or Valibot, or a custom `{ parse, jsonSchema? }` adapter.\n\n```ts\nimport { z } from 'zod';\n\nconst Customer = z.object({ name: z.string() });\nconst response = await llm.generate({\n  model: 'main_model', messages,\n  output: Customer,\n  jsonSchema: z.toJSONSchema(Customer),\n});\nresponse.data?.name; // inferred string\n```\n\n`output` validates the returned JSON locally. `jsonSchema` supplies the provider's native structured-output format. Standard Schema alone does not expose a JSON Schema converter. LLMX never silently rewrites prompts or repairs invalid output. The dependency-free raw validator supports a documented subset and rejects unsupported assertions before a call.\n\n## Tools and embeddings\n\n```ts\nconst response = await llm.generate({\n  model: 'main_model', messages,\n  tools: [{ name: 'lookup', description: 'Look up a product', parameters: {\n    type: 'object', properties: { sku: { type: 'string' } }, required: ['sku'], additionalProperties: false,\n  } }],\n});\n// Arguments are validated against registered schemas before returning the final result.\n// Execute approved tools in your application; streamed fragments are not validated yet.\nconsole.log(response.toolCalls);\n\nconst vectors = await llm.embed({ provider: 'openai', model: process.env.EMBED_MODEL!, input: ['first text', 'second text'] });\n```\n\nAnthropic embeddings fail locally with `UNSUPPORTED_FEATURE`. Gemini embeds use `batchEmbedContents`; input arrays must fit the provider's batch limits.\n\n## Inspection and optional telemetry\n\n```ts\nlogging: {\n  payloads: 'full', // explicit opt-in for prompt/response inspection\n  redact: ['metadata.userId', 'wire.body.customer.email'],\n  sink: event => console.log(JSON.stringify(event)),\n}\n```\n\nFull inspection captures the serialized outbound body just before transport. Credentials in standard authentication headers and secret query parameters are redacted. Metadata is the default; no sink or exporter is enabled implicitly. Full mode requires application-specific redaction of sensitive prompt content. See [privacy and failure semantics](docs/ARCHITECTURE.md).\n\n```ts\nimport { langfuse } from '@ayushsoam51/llmx/telemetry/langfuse';\n// Add to createGateway:\ntelemetry: { exporters: [langfuse({\n  publicKey: process.env.LANGFUSE_PUBLIC_KEY!,\n  secretKey: process.env.LANGFUSE_SECRET_KEY!,\n  baseURL: process.env.LANGFUSE_BASE_URL,\n})] }\n```\n\nExport is asynchronous, bounded, and fail-open. Call `await llm.flush()` before serverless suspension, or pass that promise to your platform's `waitUntil`. `await llm.close()` stops admission, waits for active work up to its deadline, cancels remaining work, and flushes telemetry. No Docker discovery, installation, or environment modification occurs.\n\n## CLI\n\nAfter installing the package in an application:\n\n```sh\nnpx llmx init\nnpx llmx doctor\nnpx llmx config\nnpx llmx providers\nnpx llmx models\nnpx llmx test --model main_model\nnpx llmx inspect --file saved-events.json\n```\n\n`init` reports dependency and source-pattern findings, previews a minimal `llmx.config.mjs`, and asks before creating it. Existing files are never overwritten. `test` shows the configured route and token cap before requesting confirmation for a paid API call. For deliberate noninteractive execution, pass `--yes`. Other commands do not probe providers. Config files are application code and should come from a trusted source.\n\n## Documentation\n\n- [Architecture, behavior, defaults, and limitations](docs/ARCHITECTURE.md)\n- [Requirements coverage and roadmap](docs/SCOPE.md)\n- [Generated public API](docs/API.md)\n- [Benchmark evidence](docs/benchmark.json)\n- [Original requirements, extracted from the supplied document](docs/PRODUCT_REQUIREMENTS.md)\n- [Working mock example](examples/mock.mjs)\n\nCI is configured for Node 22 and 24 on Linux, macOS, and Windows. Local validation was performed on Node 24/Linux. Bun, Deno, edge environments, live provider credentials, and real Langfuse delivery remain unverified. Compatibility with a provider API does not imply every model supports every operation.\n\nLicense selection and npm publication are intentionally pending. This repository is currently marked `UNLICENSED`.\n","readmeFilename":"README.md","_rev":"1-b2dcfc4514872fbf3d7ecd6ae78650a1"}