![\"vern-llm](\"https://raw.githubusercontent.com/LakBud/vernLLM/main/apps/docs/public/logo.png\"){kind=logo}
\n\n \n
\n GitHub ·\n Documentation ·\n npm\n
\n\n\n ![\"npm](\"https://img.shields.io/npm/v/vern-llm.svg\")
\n ![\"npm](\"https://img.shields.io/npm/dm/vern-llm.svg\")
\n ![\"CI](\"https://github.com/LakBud/vernLLM/actions/workflows/ci.yml/badge.svg\"){kind=icon}
\n ![](\"https://codecov.io/gh/LakBud/vernLLM/graph/badge.svg?token=NKKW54MODY\"/){kind=icon}
\n ![\"license\"](\"https://img.shields.io/npm/l/vern-llm.svg\")
\n ![\"node](\"https://img.shields.io/node/v/vern-llm.svg\")
\n ![\"TypeScript\"](\"https://img.shields.io/badge/TypeScript-strict-3178C6?logo=typescript&logoColor=white\"){kind=icon}
The LLM call framework. Resilience, observability, and control for every call, in your own process.
\n\n**Full documentation: [vernllm.dev](https://vernllm.dev)** for installation, structured output, caching, circuit breaker, provider fallback, rate limiting, observability, every adapter, and the complete API reference. This README is a quick reference, not the manual.\n\n## Install\n\n```bash\nnpm i vern-llm\n```\n\n## Quick start\n\n```ts\nimport Anthropic from '@anthropic-ai/sdk';\nimport OpenAI from 'openai';\nimport { fromAnthropic, fromOpenAI, VernLLM } from 'vern-llm';\n\nconst openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));\nconst anthropic = fromAnthropic(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }));\n\nconst llm = new VernLLM({\n client: openai,\n model: 'gpt-4o',\n fallback: { client: anthropic, model: 'claude-sonnet-5', circuitBreaker: true },\n rateLimit: { requestsPerMinute: 500, tokensPerMinute: 100_000, maxConcurrent: 20 },\n retryBudget: { windowMs: 60_000, minCalls: 20, retryRatio: 0.2 },\n maxRetries: 3,\n timeoutMs: 10_000,\n defaultMaxTokens: 1000,\n defaultReasoningEffort: 'medium',\n});\n\nconst result = await llm.call({ userContent: \"What's the weather in New York?\" });\n```\n\n## Why vern-llm?\n\n- **Retries with backoff**: transient failures retry automatically; validation errors and non-retryable status codes fail fast instead\n- **Provider fallback**: declare an ordered list of backup targets, tried in order after the primary, with no scoring or health-checking, `fallback` on the same constructor\n- **Client-side rate limiting**: queue locally against requests-per-minute, tokens-per-minute, and concurrency ceilings instead of letting the provider reject the call\n- **Structured output**: pass a Zod schema, get a typed, validated result back\n- **Tool calling**: pass `tools`, vern-llm handles retries and validation around them the same as any other call; you run the tools and continue the conversation\n- **Streaming**: set `stream: true` on any call and get live chunks alongside the same validated result the call would otherwise resolve to\n- **Provider-native JSON Schema mode**: constrain generation itself, not just validate after the fact\n- **Caching**: wrap any LLM call with `cachedCall`, bring your own cache adapter\n- **Middleware pipeline**: `transform` patches the outgoing request per attempt, `wrap` runs once around the whole logical call regardless of retries or fallback, patches from separate middleware merge instead of clobbering each other, order is controlled by `priority`, and each entry can be conditionally `enabled` per call\n- **Circuit breaker**: trips after repeated failures, recovers automatically once the provider's back, independent per fallback target too\n- **Observability**: one `onEvent` stream reports retries, fallovers, circuit transitions, and rate-limit waits\n- **Usage tracking**: `onUsage` and `onUsageFailure` report token spend on success and on failure, so nothing goes unaccounted for when a call fails after the provider already responded\n- **One interface, every provider**: OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together, Fireworks, Ollama, Anthropic, Gemini, Bedrock, or raw HTTP via `fromFetch`\n- **Zero runtime dependencies**: `zod` and provider SDKs are not required dependencies; vern-llm relies on compatible interfaces rather than specific implementations.\n\n### Why not a gateway?\n\nThe customization is something a gateway can't match. vern-llm runs in your process, so you bring your own rate limiter, cache, and middlewares which transform the call instead of picking from a config panel, and hooks like `reserveUsage`, `onEvent` and even more give you superior control over billing and observability.\n\nSee the [docs](https://vernllm.dev) for adapter setup, caching, the circuit breaker, provider fallback, rate limiting, and structured output in depth.\n\n## License\n\n[MIT](https://github.com/LakBud/vernLLM/blob/main/LICENSE.md) © LakBud\n","readmeFilename":""}