![\"NodeLLM](\"https://nodellm.dev/assets/images/logo.png\"){kind=logo}
\n \n\n \n \n \n
![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/openai.svg\"){kind=icon}
**OpenAI** | Chat, Streaming, Tools, Vision, Audio, Images, Transcription, **Reasoning** |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/anthropic-text.svg\"){kind=icon}
**Anthropic** | Chat, Streaming, Tools, Vision, PDF, Structured Output, **Extended Thinking (Claude 3.7)** |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/gemini-color.svg\"){kind=icon}
**Gemini** | Chat, Streaming, Tools, Vision, Audio, Video, Embeddings |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/deepseek-color.svg\"){kind=icon}
**DeepSeek** | Chat (V3), **Extended Thinking (R1)**, Streaming, Tools |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/bedrock-color.svg\"){kind=icon}
**Bedrock** | Chat, Streaming, Tools, Image Gen (Titan/SD), Embeddings, **Prompt Caching** |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/openrouter.svg\"){kind=icon}
**OpenRouter** | 540+ models, Chat, Streaming, Tools, Vision, Embeddings, **Reasoning** |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/ollama.svg\"){kind=icon}
**Ollama** | **Local Inference**, Chat, Streaming, Tools, Vision, Embeddings |\n| ![](\"https://registry.npmmirror.com/@lobehub/icons-static-svg/latest/files/icons/mistral-color.svg\"){kind=icon}
**Mistral** | Chat, Streaming, Tools, Vision, Embeddings, Transcription, Moderation, **Reasoning (Magistral)** |\n| ![](\"https://nodellm.dev/assets/images/providers/typesafe.svg\")
**TypeSafe** | **Judgments**: typed probability, choice and score answers (Jev) |\n\n---\n\n## ⚡ Quick Start\n\n### Installation\n\n```bash\nnpm install @node-llm/core\n```\n\n### Basic Chat & Streaming\n\nNodeLLM automatically reads your API keys from environment variables (e.g., `OPENAI_API_KEY`).\n\n```ts\nimport { createLLM } from \"@node-llm/core\";\n\nconst llm = createLLM({ provider: \"openai\" });\n\n// 1. Standard Request\nconst res = await llm.chat(\"gpt-4o\").ask(\"What is the speed of light?\");\nconsole.log(res.content);\n\n// 2. Real-time Streaming\nfor await (const chunk of llm.chat().stream(\"Tell me a long story\")) {\n process.stdout.write(chunk.content);\n}\n```\n\n### Structured Output (Zod)\n\nStop parsing markdown. Get typed objects directly.\n\n```ts\nimport { z } from \"@node-llm/core\";\n\nconst PlayerSchema = z.object({\n name: z.string(),\n powerLevel: z.number(),\n abilities: z.array(z.string())\n});\n\nconst chat = llm.chat(\"gpt-4o-mini\").withSchema(PlayerSchema);\nconst response = await chat.ask(\"Generate a random RPG character\");\n\nconsole.log(response.parsed.name); // Fully typed!\n```\n\n---\n\n## 🛡️ Security Circuit Breakers\n\nNodeLLM protects your production environment with four built-in safety pillars:\n\n```ts\nconst llm = createLLM({\n requestTimeout: 15000, // 15s DoS Protection\n maxTokens: 4096, // Cost Protection\n maxRetries: 3, // Retry Storm Protection\n maxToolCalls: 5 // Infinite Loop Protection\n});\n```\n\n---\n\n## 🔌 Middleware System\n\nNodeLLM 1.9.0 introduces a powerful lifecycle hook system for audit, security, and observability.\n\n```ts\nimport { createLLM, PIIMaskMiddleware, UsageLoggerMiddleware } from \"@node-llm/core\";\n\nconst llm = createLLM({\n provider: \"openai\",\n middlewares: [\n new PIIMaskMiddleware(), // Redact emails/phone numbers automatically\n new UsageLoggerMiddleware() // Log structured token usage & costs\n ]\n});\n\n// All chats created from this instance inherit these middlewares\nconst chat = llm.chat(\"gpt-4o\");\n```\n\n### Decisive Tool Safety\n\nMiddlewares can control the engine's recovery strategy during tool failures.\n\n```ts\nconst safetyMiddleware = {\n name: \"Audit\",\n onToolCallError: async (ctx, tool, error) => {\n if (tool.function.name === \"delete_user\") return \"STOP\"; // Kill the loop\n return \"RETRY\"; // Attempt recovery\n }\n};\n```\n\n---\n\n## 🤖 Agent Class\n\nDefine reusable, class-configured agents with a declarative DSL:\n\n```ts\nimport { Agent, Tool, z } from \"@node-llm/core\";\n\nclass LookupOrderTool extends Tool<{ orderId: string }> {\n name = \"lookup_order\";\n description = \"Look up an order by ID\";\n schema = z.object({ orderId: z.string() });\n\n async execute({ orderId }: { orderId: string }) {\n return { status: \"shipped\", eta: \"Tomorrow\" };\n }\n}\n\nclass SupportAgent extends Agent {\n static model = \"gpt-4.1\";\n static instructions = \"You are a helpful support agent.\";\n static tools = [LookupOrderTool];\n static temperature = 0.2;\n}\n\n// Use anywhere in your app\nconst agent = new SupportAgent();\nconst response = await agent.ask(\"Where is order #123?\");\nconsole.log(response.content);\n```\n\n### ToolHalt - Early Loop Termination\n\nStop the agentic loop early when a definitive answer is found:\n\n```ts\nclass FinalAnswerTool extends Tool<{ answer: string }> {\n name = \"final_answer\";\n description = \"Return the final answer to the user\";\n schema = z.object({ answer: z.string() });\n\n async execute({ answer }: { answer: string }) {\n return this.halt(answer); // Stops the loop, returns this result\n }\n}\n```\n\n---\n\n## 💾 Ecosystem\n\nLooking for persistence? use **[@node-llm/orm](https://www.npmjs.com/package/@node-llm/orm)**.\n\n- Automatically saves chat history to PostgreSQL/MySQL/SQLite via Prisma.\n- Tracks tool execution results and API metrics (latency, cost, tokens).\n\n---\n\n## 📚 Full Documentation\n\nVisit **[nodellm.dev](https://nodellm.dev/)** for:\n\n- [Deep Dive into Tool Calling](https://nodellm.dev/core-features/tools)\n- [Multi-modal Vision & Audio Guide](https://nodellm.dev/core-features/multimodal)\n- [Custom Provider Plugin System](https://nodellm.dev/advanced/custom-providers)\n\n---\n\n## License\n\nMIT © [NodeLLM Contributors]\n","readmeFilename":"README.md"}