{"_id":"@contextaisdk/provider-ollama","name":"@contextaisdk/provider-ollama","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@contextaisdk/provider-ollama","version":"0.1.0","description":"Ollama local LLM provider for ContextAI SDK","license":"MIT","author":{"name":"ContextAI Team"},"homepage":"https://github.com/draco28/contextai#readme","bugs":{"url":"https://github.com/draco28/contextai/issues"},"repository":{"type":"git","url":"git+https://github.com/draco28/contextai.git","directory":"packages/provider-ollama"},"sideEffects":false,"type":"module","main":"./dist/index.cjs","module":"./dist/index.js","types":"./dist/index.d.ts","exports":{".":{"import":{"types":"./dist/index.d.ts","default":"./dist/index.js"},"require":{"types":"./dist/index.d.cts","default":"./dist/index.cjs"}}},"engines":{"node":">=18.0.0"},"dependencies":{"@contextaisdk/core":"0.1.0"},"devDependencies":{"tsup":"^8.0.0","typescript":"^5.5.0","vitest":"^2.1.0","@contextaisdk/tsconfig":"0.1.0"},"keywords":["contextai","ollama","llm","ai","provider","local"],"publishConfig":{"access":"public"},"scripts":{"build":"tsup","dev":"tsup --watch","test":"vitest run","test:watch":"vitest","test:integration":"vitest run --config vitest.integration.config.ts","typecheck":"tsc --noEmit","clean":"rm -rf dist"},"_id":"@contextaisdk/provider-ollama@0.1.0","_integrity":"sha512-pbZbnNadiKYl6oEzofswRWgR0RXL31AtgolJrkkklG8LefBFZ1awngDj8uEvN1f2d9H7B3X5vqRBUmX9xMlEew==","_resolved":"/private/var/folders/nh/3wp6hgt537b3z2ygfjj637m00000gn/T/3e4beaa422222d0f58588e4f7788363d/contextaisdk-provider-ollama-0.1.0.tgz","_from":"file:contextaisdk-provider-ollama-0.1.0.tgz","_nodeVersion":"25.1.0","_npmVersion":"11.6.2","dist":{"integrity":"sha512-pbZbnNadiKYl6oEzofswRWgR0RXL31AtgolJrkkklG8LefBFZ1awngDj8uEvN1f2d9H7B3X5vqRBUmX9xMlEew==","shasum":"01f0ebb79bd2626ef6aecc4757c989d599730dce","tarball":"https://registry.npmjs.org/@contextaisdk/provider-ollama/-/provider-ollama-0.1.0.tgz","fileCount":15,"unpackedSize":183279,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQDNrS22tHwiMqTLiPn0BizYThNtKo29SaihluA0xc43mAIgXLJ/bg53ACcEN+JUuqVi+t01uzSHvNtN/nb8VrK4TtE="}]},"_npmUser":{"name":"draco28","email":"praveensingh2897@gmail.com"},"directories":{},"maintainers":[{"name":"draco28","email":"praveensingh2897@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/provider-ollama_0.1.0_1769140054422_0.5637330329911505"},"_hasShrinkwrap":false}},"time":{"created":"2026-01-23T03:47:34.343Z","0.1.0":"2026-01-23T03:47:34.587Z","modified":"2026-01-23T03:47:34.823Z"},"maintainers":[{"name":"draco28","email":"praveensingh2897@gmail.com"}],"description":"Ollama local LLM provider for ContextAI SDK","homepage":"https://github.com/draco28/contextai#readme","keywords":["contextai","ollama","llm","ai","provider","local"],"repository":{"type":"git","url":"git+https://github.com/draco28/contextai.git","directory":"packages/provider-ollama"},"author":{"name":"ContextAI Team"},"bugs":{"url":"https://github.com/draco28/contextai/issues"},"license":"MIT","readme":"# @contextaisdk/provider-ollama\n\n> Local LLM provider for ContextAI SDK via Ollama\n\n[![npm version](https://img.shields.io/npm/v/@contextaisdk/provider-ollama.svg?style=flat-square)](https://www.npmjs.com/package/@contextaisdk/provider-ollama)\n[![TypeScript](https://img.shields.io/badge/TypeScript-5.5+-blue.svg?style=flat-square)](https://www.typescriptlang.org/)\n\n## Installation\n\n```bash\nnpm install @contextaisdk/provider-ollama\n# or\npnpm add @contextaisdk/provider-ollama\n```\n\n**No peer dependencies!** This package has no external dependencies beyond `@contextaisdk/core`.\n\n## Prerequisites\n\nInstall and run Ollama:\n\n```bash\n# macOS\nbrew install ollama\n\n# Linux\ncurl -fsSL https://ollama.com/install.sh | sh\n\n# Start the server\nollama serve\n\n# Pull a model\nollama pull llama3.2\n```\n\n## Quick Start\n\n```typescript\nimport { OllamaProvider, OllamaModels } from '@contextaisdk/provider-ollama';\nimport { Agent } from '@contextaisdk/core';\n\n// Create the provider (no API key needed!)\nconst ollama = new OllamaProvider({\n  model: OllamaModels.LLAMA_3_2,\n  host: 'http://localhost:11434', // default\n});\n\n// Check if Ollama is running\nif (await ollama.isAvailable()) {\n  const agent = new Agent({\n    name: 'Local Assistant',\n    systemPrompt: 'You are a helpful assistant.',\n    llm: ollama,\n  });\n\n  const response = await agent.run('Hello!');\n  console.log(response.output);\n} else {\n  console.log('Start Ollama with: ollama serve');\n}\n```\n\n## Configuration\n\n```typescript\nconst provider = new OllamaProvider({\n  // Required\n  model: string;               // e.g., 'llama3.2', 'mistral', 'codellama'\n\n  // Optional settings\n  host?: string;               // Server URL (default: 'http://localhost:11434')\n  timeout?: number;            // Request timeout (default: 120000ms - 2 min)\n  headers?: Record<string, string>;  // Custom headers\n  keepAlive?: string;          // Memory management (e.g., '5m', '0')\n\n  // Default generation options\n  defaultOptions?: {\n    temperature?: number;      // 0-2 (default: 0.8)\n    maxTokens?: number;        // Max response tokens\n    topP?: number;             // Nucleus sampling\n    topK?: number;             // Top-K sampling\n    stopSequences?: string[];  // Stop generation triggers\n  };\n});\n```\n\n## Available Models\n\n```typescript\nimport { OllamaModels } from '@contextaisdk/provider-ollama';\n\n// Llama 3.2\nOllamaModels.LLAMA_3_2        // 'llama3.2' (default size)\nOllamaModels.LLAMA_3_2_1B    // 'llama3.2:1b' (small, fast)\nOllamaModels.LLAMA_3_2_3B    // 'llama3.2:3b' (balanced)\n\n// Llama 3.1\nOllamaModels.LLAMA_3_1_8B    // 'llama3.1:8b'\nOllamaModels.LLAMA_3_1_70B   // 'llama3.1:70b' (requires 64GB+ RAM)\n\n// Mistral\nOllamaModels.MISTRAL          // 'mistral'\nOllamaModels.MISTRAL_NEMO     // 'mistral-nemo'\n\n// Code Models\nOllamaModels.CODELLAMA        // 'codellama'\nOllamaModels.DEEPSEEK_CODER   // 'deepseek-coder'\nOllamaModels.QWEN_CODER       // 'qwen2.5-coder'\n\n// Small Models\nOllamaModels.PHI3             // 'phi3' (Microsoft)\nOllamaModels.GEMMA2           // 'gemma2' (Google)\n```\n\nOr use any model string: `'llama3.2:latest'`, `'mixtral:8x7b'`, etc.\n\n## Features\n\n### Streaming Responses\n\nOllama uses NDJSON (newline-delimited JSON) for streaming:\n\n```typescript\nfor await (const chunk of provider.streamChat([\n  { role: 'user', content: 'Write a story about a robot' }\n])) {\n  process.stdout.write(chunk.content);\n}\n```\n\n### Tool Calling\n\nOllama supports function calling with compatible models:\n\n```typescript\nimport { defineTool } from '@contextaisdk/core';\nimport { z } from 'zod';\n\nconst calculatorTool = defineTool({\n  name: 'calculate',\n  description: 'Perform mathematical calculations',\n  parameters: z.object({\n    expression: z.string().describe('Math expression to evaluate'),\n  }),\n  execute: async ({ expression }) => {\n    // Simple eval (use a proper math parser in production)\n    return { result: eval(expression) };\n  },\n});\n\nconst agent = new Agent({\n  name: 'Math Assistant',\n  systemPrompt: 'Help users with calculations.',\n  llm: ollama,\n  tools: [calculatorTool],\n});\n```\n\n**Note:** Tool calling works best with Llama 3.1+, Mistral, and newer models.\n\n### List Available Models\n\n```typescript\nconst models = await provider.listModels();\n\nfor (const model of models) {\n  console.log(`${model.name} - ${model.details.parameter_size}`);\n}\n// Output:\n// llama3.2:latest - 3B\n// mistral:latest - 7B\n// codellama:latest - 7B\n```\n\n### Multimodal (Vision Models)\n\n```typescript\nimport { OllamaProvider } from '@contextaisdk/provider-ollama';\n\nconst provider = new OllamaProvider({\n  model: 'llava', // Vision model\n});\n\n// Send images as base64\nconst response = await provider.chat([\n  {\n    role: 'user',\n    content: 'What is in this image?',\n    images: [base64ImageString],\n  },\n]);\n```\n\n### Memory Management (keepAlive)\n\nControl how long models stay loaded in memory:\n\n```typescript\n// Keep model loaded for 5 minutes (default Ollama behavior)\nconst provider = new OllamaProvider({\n  model: 'llama3.2',\n  keepAlive: '5m',\n});\n\n// Unload immediately after request (saves RAM)\nconst lowMemoryProvider = new OllamaProvider({\n  model: 'llama3.2',\n  keepAlive: '0',\n});\n\n// Keep loaded indefinitely\nconst alwaysReadyProvider = new OllamaProvider({\n  model: 'llama3.2',\n  keepAlive: '-1',\n});\n```\n\n## Direct Chat API\n\nUse the provider directly without an agent:\n\n```typescript\n// Non-streaming\nconst response = await provider.chat([\n  { role: 'system', content: 'You are a coding assistant.' },\n  { role: 'user', content: 'Write a TypeScript function' },\n], {\n  temperature: 0.7,\n  maxTokens: 500,\n});\n\nconsole.log(response.content);\n\n// Response includes timing metrics\nconsole.log('Generation speed:', response.metrics?.tokensPerSecond, 'tokens/s');\n\n// Streaming\nfor await (const chunk of provider.streamChat(messages)) {\n  process.stdout.write(chunk.content);\n}\n```\n\n## Error Handling\n\n```typescript\nimport { OllamaProviderError } from '@contextaisdk/provider-ollama';\n\ntry {\n  const response = await provider.chat(messages);\n} catch (error) {\n  if (error instanceof OllamaProviderError) {\n    switch (error.code) {\n      case 'CONNECTION_REFUSED':\n        console.log('Ollama not running. Start with: ollama serve');\n        break;\n      case 'MODEL_NOT_FOUND':\n        console.log('Model not installed. Run: ollama pull', error.details.model);\n        break;\n      case 'MODEL_LOADING':\n        console.log('Model still loading, please wait...');\n        break;\n      case 'OUT_OF_MEMORY':\n        console.log('Not enough RAM. Try a smaller model.');\n        break;\n      case 'TIMEOUT':\n        console.log('Request timed out. Local inference can be slow.');\n        break;\n      default:\n        console.log('Error:', error.message);\n    }\n  }\n}\n```\n\n### Error Codes\n\n| Code | Description |\n|------|-------------|\n| `CONNECTION_REFUSED` | Ollama server not running |\n| `MODEL_NOT_FOUND` | Model not pulled locally |\n| `MODEL_LOADING` | Model still loading into memory |\n| `OUT_OF_MEMORY` | Insufficient RAM for model |\n| `CONTEXT_LENGTH_EXCEEDED` | Input too long for model |\n| `TIMEOUT` | Request timed out |\n| `INVALID_RESPONSE` | Malformed response from Ollama |\n\n## Performance Tips\n\n### 1. Choose the Right Model Size\n\n| RAM | Recommended Models |\n|-----|-------------------|\n| 8GB | llama3.2:1b, phi3, gemma2:2b |\n| 16GB | llama3.2:3b, mistral, codellama:7b |\n| 32GB | llama3.1:8b, mixtral:8x7b |\n| 64GB+ | llama3.1:70b |\n\n### 2. Use Quantized Models\n\n```bash\n# 4-bit quantization (smaller, faster, slightly lower quality)\nollama pull llama3.2:3b-q4_0\n\n# 8-bit quantization (balanced)\nollama pull llama3.2:3b-q8_0\n```\n\n### 3. Pre-load Models\n\n```bash\n# Load model into memory before use\ncurl http://localhost:11434/api/generate -d '{\"model\": \"llama3.2\", \"keep_alive\": \"10m\"}'\n```\n\n### 4. Increase Timeout for Large Models\n\n```typescript\nconst provider = new OllamaProvider({\n  model: 'llama3.1:70b',\n  timeout: 300000, // 5 minutes for large models\n});\n```\n\n## Troubleshooting\n\n### \"Connection refused\"\n\n```bash\n# Check if Ollama is running\ncurl http://localhost:11434/api/tags\n\n# Start Ollama\nollama serve\n```\n\n### \"Model not found\"\n\n```bash\n# List available models\nollama list\n\n# Pull the model\nollama pull llama3.2\n```\n\n### Slow Generation\n\n1. Use a smaller model or quantized version\n2. Reduce `maxTokens` in generation options\n3. Close other memory-intensive applications\n4. Consider GPU acceleration if available\n\n### Using a Remote Ollama Server\n\n```typescript\nconst provider = new OllamaProvider({\n  model: 'llama3.2',\n  host: 'http://192.168.1.100:11434', // Remote server\n});\n```\n\n## Comparison: Local vs Cloud\n\n| Aspect | Ollama (Local) | Cloud Providers |\n|--------|---------------|-----------------|\n| Privacy | Data stays local | Data sent to API |\n| Cost | Free (hardware only) | Per-token pricing |\n| Speed | Depends on hardware | Consistent |\n| Models | Open-source only | Proprietary available |\n| Internet | Not required | Required |\n| Setup | Requires Ollama | API key only |\n\n## License\n\nMIT\n","readmeFilename":"README.md","_rev":"1-3da62d535b42b0348a45705e5ff7e30a"}