{"_id":"@aivue/browser-llm","name":"@aivue/browser-llm","dist-tags":{"latest":"1.0.0"},"versions":{"1.0.0":{"name":"@aivue/browser-llm","version":"1.0.0","description":"High-performance in-browser LLM inference for Vue.js using WebLLM and WebGPU","main":"dist/index.js","module":"dist/index.mjs","types":"dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.mjs","require":"./dist/index.js"},"./browser-llm.css":"./dist/browser-llm.css","./dist/browser-llm.css":"./dist/browser-llm.css"},"scripts":{"build":"npm run clean && vite build","dev":"vite build --watch","lint":"eslint \"src/**/*.{ts,vue}\"","clean":"rm -rf dist","prepublishOnly":"npm run build"},"keywords":["vue","ai","llm","browser","webllm","webgpu","local-ai","privacy","offline","machine-learning","chatbot","inference","llama","phi","gemma","mistral"],"author":{"name":"reachbrt"},"license":"MIT","repository":{"type":"git","url":"git+https://github.com/reachbrt/vueai.git","directory":"packages/browser-llm"},"homepage":"https://github.com/reachbrt/vueai#readme","bugs":{"url":"https://github.com/reachbrt/vueai/issues"},"publishConfig":{"access":"public"},"peerDependencies":{"@aivue/core":"^1.3.5","vue":"^2.6.0 || ^3.0.0"},"dependencies":{"@mlc-ai/web-llm":"^0.2.80"},"devDependencies":{"@types/node":"^20.16.0","@vitejs/plugin-vue":"^5.0.0","@vue/compiler-sfc":"^3.5.13","eslint":"^8.56.0","typescript":"^5.3.0","vite":"^6.3.5","vite-plugin-dts":"^4.5.3","vue":"^3.5.0","vue-tsc":"^2.2.10"},"_id":"@aivue/browser-llm@1.0.0","gitHead":"990e5f8ec751fe70b28af43f4c1ed9acec60846f","_nodeVersion":"22.14.0","_npmVersion":"10.9.2","dist":{"integrity":"sha512-9HAp6KA6nzcGOeZv5lzcBQyLEt5AE3ZyVGgp2SXWz89xn6fj5IpermAl0OWdNgXQgomZRSiW/MRrhsKq3DbQTg==","shasum":"11cfc66e64340a3948a5ac4d73fa5cab1181816a","tarball":"https://registry.npmjs.org/@aivue/browser-llm/-/browser-llm-1.0.0.tgz","fileCount":16,"unpackedSize":180739,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQC/IFIe3SoI1gjb6WOkCbpW6OZUCz1Wp30df8bikiSOVAIhANhQ4irGq6c03GdBFs2zJgddkmU3Elrk9Pgxrp7zm7Bf"}]},"_npmUser":{"name":"aivue","email":"reachbrt@gmail.com"},"directories":{},"maintainers":[{"name":"aivue","email":"reachbrt@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/browser-llm_1.0.0_1768622295511_0.1391852691776021"},"_hasShrinkwrap":false}},"time":{"created":"2026-01-17T03:58:15.402Z","1.0.0":"2026-01-17T03:58:15.672Z","modified":"2026-01-17T03:58:15.923Z"},"maintainers":[{"name":"aivue","email":"reachbrt@gmail.com"}],"description":"High-performance in-browser LLM inference for Vue.js using WebLLM and WebGPU","homepage":"https://github.com/reachbrt/vueai#readme","keywords":["vue","ai","llm","browser","webllm","webgpu","local-ai","privacy","offline","machine-learning","chatbot","inference","llama","phi","gemma","mistral"],"repository":{"type":"git","url":"git+https://github.com/reachbrt/vueai.git","directory":"packages/browser-llm"},"author":{"name":"reachbrt"},"bugs":{"url":"https://github.com/reachbrt/vueai/issues"},"license":"MIT","readme":"# @aivue/browser-llm\n\n<div align=\"center\">\n\n### 🚀 High-Performance In-Browser LLM Inference for Vue.js\n\n[![npm version](https://img.shields.io/npm/v/@aivue/browser-llm.svg)](https://www.npmjs.com/package/@aivue/browser-llm)\n[![npm downloads](https://img.shields.io/npm/dm/@aivue/browser-llm.svg)](https://www.npmjs.com/package/@aivue/browser-llm)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nRun powerful AI models **completely locally** in your browser with **zero API costs** and **100% privacy**!\n\n[Demo](https://aivue.netlify.app) | [Documentation](https://github.com/reachbrt/vueai) | [Report Bug](https://github.com/reachbrt/vueai/issues)\n\n</div>\n\n---\n\n## ✨ Features\n\n### 🔒 **100% Privacy** - All processing happens locally, no data sent to servers\n### 💰 **Zero Cost** - No API keys needed, no usage fees\n### ⚡ **WebGPU Accelerated** - Hardware-accelerated inference using GPU\n### 📦 **Model Caching** - Models cached in browser for offline use\n### 🔄 **OpenAI-Compatible API** - Same interface as existing chatbot packages\n### 🎯 **Multiple Models** - Support for Llama, Phi, Gemma, Mistral, Qwen, and more\n### 📱 **Progressive Loading** - Show download progress for models\n### 🔌 **Drop-in Replacement** - Works with existing @aivue/chatbot components\n### 🌐 **Vue 2 & 3 Compatible** - Works with both Vue 2.6+ and Vue 3.x\n\n---\n\n## 📦 Installation\n\n```bash\nnpm install @aivue/browser-llm\n# or\nyarn add @aivue/browser-llm\n# or\npnpm add @aivue/browser-llm\n```\n\n---\n\n## 🚀 Quick Start\n\n### Basic Usage\n\n```vue\n<template>\n  <div>\n    <!-- Model Selector -->\n    <ModelSelector\n      :available-models=\"availableModels\"\n      :current-model=\"currentModel\"\n      :is-loading=\"isLoading\"\n      :download-progress=\"downloadProgress\"\n      :error=\"error\"\n      :is-web-g-p-u-supported=\"isWebGPUSupported\"\n      @load-model=\"loadModel\"\n    />\n\n    <!-- Chat Interface -->\n    <BrowserLLMChat\n      :messages=\"messages\"\n      :is-model-loaded=\"isModelLoaded\"\n      :is-loading=\"isLoading\"\n      :current-model=\"currentModel\"\n      :error=\"error\"\n      :tokens-per-second=\"tokensPerSecond\"\n      @send-message=\"sendMessage\"\n      @stream-message=\"handleStreamMessage\"\n      @clear=\"clearMessages\"\n    />\n  </div>\n</template>\n\n<script setup>\nimport { useBrowserLLM, ModelSelector, BrowserLLMChat } from '@aivue/browser-llm';\n\nconst {\n  availableModels,\n  selectedModel,\n  loadModel,\n  messages,\n  sendMessage,\n  streamMessage,\n  clearMessages,\n  isModelLoaded,\n  isLoading,\n  downloadProgress,\n  error,\n  isWebGPUSupported,\n  tokensPerSecond,\n  currentModel,\n} = useBrowserLLM({\n  defaultModel: 'Llama-3.2-1B-Instruct-q4f16_1-MLC',\n  temperature: 0.7,\n  systemPrompt: 'You are a helpful AI assistant.',\n});\n\nconst handleStreamMessage = async (content) => {\n  for await (const chunk of streamMessage(content)) {\n    // Handle streaming chunks if needed\n  }\n};\n</script>\n```\n\n---\n\n## 📚 Available Models\n\n| Model | Size | Speed | Use Case |\n|-------|------|-------|----------|\n| **Llama 3.2 1B** | ~800MB | ⚡⚡⚡ | Fast chat, mobile-friendly |\n| **Llama 3.2 3B** | ~2GB | ⚡⚡ | Balanced performance |\n| **Llama 3.1 8B** | ~5GB | ⚡ | High quality responses |\n| **Phi 3.5 Mini** | ~2.5GB | ⚡⚡ | Excellent for reasoning |\n| **Gemma 2 2B** | ~1.5GB | ⚡⚡⚡ | Google's efficient model |\n| **Mistral 7B** | ~4GB | ⚡ | High quality general purpose |\n| **Qwen 2.5 1.5B** | ~1GB | ⚡⚡⚡ | Fast multilingual |\n| **Qwen 2.5 7B** | ~4.5GB | ⚡ | Excellent multilingual |\n\n---\n\n## 🎯 API Reference\n\n### `useBrowserLLM(options)`\n\nMain composable for browser LLM functionality.\n\n#### Options\n\n```typescript\ninterface BrowserLLMOptions {\n  defaultModel?: string;        // Default model to load\n  temperature?: number;          // 0.0 - 2.0 (default: 0.7)\n  topP?: number;                // 0.0 - 1.0 (default: 0.9)\n  maxTokens?: number;           // Max tokens to generate (default: 2048)\n  systemPrompt?: string;        // System prompt for the model\n}\n```\n\n#### Returns\n\n```typescript\ninterface UseBrowserLLMReturn {\n  // Model Management\n  availableModels: Ref<ModelInfo[]>;\n  selectedModel: Ref<string>;\n  loadModel: (modelId: string) => Promise<void>;\n  unloadModel: () => Promise<void>;\n  isModelLoaded: Ref<boolean>;\n  \n  // Chat\n  messages: Ref<BrowserLLMMessage[]>;\n  sendMessage: (content: string, options?: ChatOptions) => Promise<string>;\n  streamMessage: (content: string, options?: ChatOptions) => AsyncGenerator<string>;\n  clearMessages: () => void;\n  \n  // Progress & Status\n  downloadProgress: Ref<ModelDownloadProgress | null>;\n  isLoading: Ref<boolean>;\n  error: Ref<Error | null>;\n  isWebGPUSupported: Ref<boolean>;\n  \n  // Performance\n  tokensPerSecond: Ref<number>;\n  generationStats: Ref<GenerationStats | null>;\n}\n```\n\n---\n\n## 🔧 Advanced Usage\n\n### Using Composable Only\n\n```vue\n<script setup>\nimport { useBrowserLLM } from '@aivue/browser-llm';\n\nconst { loadModel, sendMessage, messages } = useBrowserLLM();\n\n// Load a model\nawait loadModel('Llama-3.2-1B-Instruct-q4f16_1-MLC');\n\n// Send a message\nconst response = await sendMessage('Hello, how are you?');\nconsole.log(response);\n```\n\n### Streaming Responses\n\n```vue\n<script setup>\nimport { useBrowserLLM } from '@aivue/browser-llm';\n\nconst { loadModel, streamMessage } = useBrowserLLM();\n\nawait loadModel('Llama-3.2-1B-Instruct-q4f16_1-MLC');\n\n// Stream a response\nfor await (const chunk of streamMessage('Tell me a story')) {\n  console.log(chunk); // Print each chunk as it arrives\n}\n```\n\n### Custom Chat Options\n\n```vue\n<script setup>\nimport { useBrowserLLM } from '@aivue/browser-llm';\n\nconst { sendMessage } = useBrowserLLM();\n\n// Send with custom options\nconst response = await sendMessage('Explain quantum physics', {\n  temperature: 0.3,      // More focused\n  max_tokens: 500,       // Limit response length\n  top_p: 0.95,\n});\n```\n\n### Checking WebGPU Support\n\n```vue\n<script setup>\nimport { checkWebGPUSupport, getWebGPUErrorMessage } from '@aivue/browser-llm';\n\nconst gpuInfo = await checkWebGPUSupport();\n\nif (!gpuInfo.supported) {\n  console.error(getWebGPUErrorMessage());\n} else {\n  console.log('WebGPU supported!', gpuInfo);\n}\n```\n\n---\n\n## 🌐 Browser Requirements\n\n- **WebGPU Support**: Chrome 113+, Edge 113+, or WebGPU-enabled browser\n- **RAM**: 4GB+ recommended (8GB+ for larger models)\n- **Storage**: 2-8GB for model caching\n- **Internet**: Required for initial model download (then works offline)\n\n### Browser Compatibility\n\n| Browser | Version | Status |\n|---------|---------|--------|\n| Chrome | 113+ | ✅ Fully Supported |\n| Edge | 113+ | ✅ Fully Supported |\n| Firefox | Nightly (with flag) | ⚠️ Experimental |\n| Safari | Not yet | ❌ Not Supported |\n\n---\n\n## 💡 Use Cases\n\n### 🔒 **Privacy-First Applications**\n- Medical/Healthcare apps with sensitive data\n- Legal document analysis\n- Personal journaling with AI assistance\n\n### 💰 **Cost-Effective Solutions**\n- Educational platforms\n- Prototyping and development\n- High-volume applications\n\n### 📱 **Offline-Capable Apps**\n- Field work applications\n- Remote area tools\n- Airplane mode functionality\n\n### 🎓 **Learning & Education**\n- AI/ML education without API costs\n- Student projects\n- Research and experimentation\n\n---\n\n## 🎨 Components\n\n### `<BrowserLLMChat>`\n\nFull-featured chat interface component.\n\n**Props:**\n- `title` (string): Chat window title\n- `placeholder` (string): Input placeholder text\n- `messages` (BrowserLLMMessage[]): Chat messages\n- `isModelLoaded` (boolean): Whether model is loaded\n- `isLoading` (boolean): Loading state\n- `currentModel` (string | null): Current model ID\n- `error` (Error | null): Error state\n- `tokensPerSecond` (number): Performance metric\n- `useStreaming` (boolean): Enable streaming (default: true)\n\n**Events:**\n- `@send-message`: Emitted when user sends a message\n- `@stream-message`: Emitted when streaming a message\n- `@clear`: Emitted when clearing chat\n\n### `<ModelSelector>`\n\nModel selection and loading component.\n\n**Props:**\n- `availableModels` (ModelInfo[]): Available models\n- `currentModel` (string | null): Currently loaded model\n- `isLoading` (boolean): Loading state\n- `downloadProgress` (ModelDownloadProgress | null): Download progress\n- `error` (Error | null): Error state\n- `isWebGPUSupported` (boolean): WebGPU support status\n\n**Events:**\n- `@load-model`: Emitted when loading a model\n- `@select-model`: Emitted when selecting a model\n\n---\n\n## 🔍 Troubleshooting\n\n### WebGPU Not Supported\n\n**Problem:** Browser doesn't support WebGPU\n\n**Solution:**\n- Update to Chrome 113+ or Edge 113+\n- Enable WebGPU in browser flags (chrome://flags)\n- Check GPU drivers are up to date\n\n### Model Download Fails\n\n**Problem:** Model fails to download\n\n**Solution:**\n- Check internet connection\n- Clear browser cache\n- Try a smaller model first\n- Check available disk space\n\n### Out of Memory\n\n**Problem:** Browser runs out of memory\n\n**Solution:**\n- Use a smaller model (1B-3B parameters)\n- Close other browser tabs\n- Increase system RAM if possible\n- Use low-resource models\n\n### Slow Performance\n\n**Problem:** Model runs slowly\n\n**Solution:**\n- Ensure WebGPU is enabled (not CPU fallback)\n- Close background applications\n- Try a smaller model\n- Check GPU is being utilized\n\n---\n\n## 📊 Performance Tips\n\n1. **Start Small**: Begin with 1B-3B parameter models\n2. **Cache Models**: Models are cached after first download\n3. **Use Streaming**: Better UX with streaming responses\n4. **Monitor Performance**: Check `tokensPerSecond` metric\n5. **Optimize Prompts**: Shorter prompts = faster responses\n\n---\n\n## 🤝 Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n---\n\n## 📄 License\n\nMIT © [reachbrt](https://github.com/reachbrt)\n\n---\n\n## 🙏 Acknowledgments\n\n- Built with [WebLLM](https://webllm.mlc.ai/) by MLC AI\n- Powered by [WebGPU](https://www.w3.org/TR/webgpu/)\n- Part of the [@aivue](https://github.com/reachbrt/vueai) ecosystem\n\n---\n\n## 🔗 Links\n\n- [GitHub Repository](https://github.com/reachbrt/vueai)\n- [npm Package](https://www.npmjs.com/package/@aivue/browser-llm)\n- [Live Demo](https://aivue.netlify.app)\n- [Report Issues](https://github.com/reachbrt/vueai/issues)\n\n---\n\n<div align=\"center\">\n\n**Made with ❤️ by [reachbrt](https://github.com/reachbrt)**\n\nIf you find this package useful, please consider giving it a ⭐ on [GitHub](https://github.com/reachbrt/vueai)!\n\n</div>\n\n","readmeFilename":"README.md","_rev":"1-74d774fe6bc12a19193188e96e2c80c6"}