{"_id":"@ariaflowagents/realtime-audio","name":"@ariaflowagents/realtime-audio","dist-tags":{"latest":"1.0.0"},"versions":{"1.0.0":{"name":"@ariaflowagents/realtime-audio","version":"1.0.0","description":"Realtime audio pipeline for AriaFlow — multi-provider speech-to-speech and orchestration.","keywords":["realtime-audio","openai","multi-provider"],"publishConfig":{"access":"public"},"type":"module","main":"dist/index.js","types":"dist/index.d.ts","exports":{".":{"types":"./dist/index.d.ts","default":"./dist/index.js"}},"dependencies":{"@google/genai":"^1.0.0","ws":"^8.19.0","zod-to-json-schema":"^3.24.0"},"peerDependencies":{"ai":"^6.0.0","zod":"^3.0.0","@ariaflowagents/core":"1.0.0"},"devDependencies":{"@types/node":"^20.11.0","@types/ws":"^8.5.13","ai":"^6.0.0","typescript":"^5.3.0","zod":"^3.23.0","@ariaflowagents/core":"1.0.0"},"scripts":{"build":"tsc -p tsconfig.json","clean":"rm -rf dist","test":"npm run build && node --test test/*.test.js"},"_id":"@ariaflowagents/realtime-audio@1.0.0","_integrity":"sha512-/B+yNZsuAezkoiYr6L6DCWFlUlqMsVgDJ/i5rjJLKCHH5oSYKbvBUfHwOdfg0kXRk2jZi2XYVsNpqAfZoI5ZnA==","_resolved":"/private/var/folders/j5/7dkmn95d0_9b5fdnfsmpjs580000gn/T/bda37af1043e326929a79f32c4ad53ab/ariaflowagents-realtime-audio-1.0.0.tgz","_from":"file:ariaflowagents-realtime-audio-1.0.0.tgz","_nodeVersion":"22.22.2","_npmVersion":"10.9.7","dist":{"integrity":"sha512-/B+yNZsuAezkoiYr6L6DCWFlUlqMsVgDJ/i5rjJLKCHH5oSYKbvBUfHwOdfg0kXRk2jZi2XYVsNpqAfZoI5ZnA==","shasum":"f6d102a63214ccc3205bcb683314d124c1d53fee","tarball":"https://registry.npmjs.org/@ariaflowagents/realtime-audio/-/realtime-audio-1.0.0.tgz","fileCount":52,"unpackedSize":236327,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIHBoEFW7wgLUBOOjDuQf2wglRxoE1Pjq7dzYMF1HIJ4pAiBrqL+7PMGqM3QWPrzXGUkacxHsEx54icjc01yCw2litA=="}]},"_npmUser":{"name":"octalpixel","email":"mithushancj@gmail.com"},"directories":{},"maintainers":[{"name":"octalpixel","email":"mithushancj@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/realtime-audio_1.0.0_1776978163142_0.1623916547878128"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-23T21:02:43.061Z","1.0.0":"2026-04-23T21:02:43.303Z","modified":"2026-04-23T21:02:43.517Z"},"maintainers":[{"name":"octalpixel","email":"mithushancj@gmail.com"}],"description":"Realtime audio pipeline for AriaFlow — multi-provider speech-to-speech and orchestration.","keywords":["realtime-audio","openai","multi-provider"],"readme":"# @ariaflowagents/realtime-audio\n\nRealtime audio pipeline for AriaFlow — the multi-provider foundation for speech-to-speech voice agents and their orchestration. Ships provider clients for Google Gemini Live and OpenAI Realtime today, a provider-agnostic `RealtimeAudioClient` interface other providers plug into, and a `VoiceEngine` / `CallWorker` pair that bridges any audio transport (WebSocket, LiveKit, etc.) to the chosen provider while handling tools, session state, and event logging via AriaFlow's Foundation primitives. (Renamed from `@ariaflowagents/gemini-native-audio` at v0.10.0; the historical \"Gemini Live native audio\" docs below reflect the original Gemini-specific slice and remain accurate for that provider.)\n\n## What This Does\n\nUnlike traditional voice pipelines (STT → LLM → TTS), Gemini Live accepts raw audio input and produces raw audio output in a single model call. This package wraps that capability for AriaFlow agents:\n\n- **`VoiceEngine`** — Call acceptor. Accepts incoming audio connections and creates per-call workers.\n- **`CallWorker`** — Per-call lifecycle manager. Bridges your audio transport (WebSocket, LiveKit, etc.) to a Gemini Live session. Handles tool calls, session state, and event logging using AriaFlow's Foundation primitives.\n- **`GeminiLiveSession`** — Thin wrapper around `@google/genai` `ai.live.connect()`. Manages the WebSocket connection to Gemini, audio encoding (base64 PCM ↔ Uint8Array), tool dispatch, and session resumption.\n- **`toolSetToGeminiDeclarations`** — Converts AriaFlow/AI SDK tool definitions (Zod schemas) to Gemini's FunctionDeclaration format.\n\n## Architecture\n\n```\n┌─────────────┐     ┌─────────────┐     ┌────────────────────┐\n│   Client     │────>│ CallWorker  │────>│ GeminiLiveSession  │\n│  (WebSocket) │     │             │     │                    │\n│              │<────│  audio +    │<────│  Gemini Live API   │\n│  audio in/out│     │  tool calls │     │  (native audio)    │\n└─────────────┘     └─────────────┘     └────────────────────┘\n                          │\n                          ├── ToolExecutor (runs AriaFlow tools)\n                          ├── ConversationState (persists transcripts)\n                          └── ConversationEventLog (records events)\n```\n\n## Usage\n\n```typescript\nimport { VoiceEngine } from '@ariaflowagents/realtime-audio';\nimport { createFoundation } from '@ariaflowagents/core/foundation';\n\nconst foundation = createFoundation({ /* ... */ });\n\nconst engine = new VoiceEngine({\n  foundation,\n  agents: [\n    {\n      id: 'receptionist',\n      name: 'Hospital Receptionist',\n      prompt: 'You are a hospital receptionist. Help patients schedule appointments.',\n      voice: 'Charon', // Gemini voice preset\n      tools: { /* AriaFlow tools */ },\n    },\n  ],\n  defaultAgentId: 'receptionist',\n  gemini: {\n    apiKey: process.env.GOOGLE_API_KEY!,\n    model: 'gemini-2.5-flash-native-audio-preview', // default\n  },\n});\n\n// Accept a call from any audio transport\nconst worker = await engine.acceptCall({\n  callId: crypto.randomUUID(),\n  transport: myWebSocketTransport, // implements TransportSession\n});\n\nawait worker.start();\n```\n\n## TransportSession Interface\n\nImplement this to connect any audio source/sink:\n\n```typescript\ninterface TransportSession {\n  sendAudio(data: Uint8Array): void;       // Send audio to client\n  onAudio(handler: (data: Uint8Array) => void): void;  // Receive audio from client\n  onClose(handler: () => void): void;      // Handle disconnect\n  close(): void;                           // Close the transport\n}\n```\n\n## Events\n\n`GeminiLiveSession` emits `RealtimeEvent`s:\n\n| Event | Description |\n|-------|-------------|\n| `audio` | Raw PCM audio from Gemini (send to client) |\n| `transcript` | Text transcript (user or assistant) |\n| `tool-call` | Gemini wants to call a tool |\n| `tool-result` | Tool execution result |\n| `turn-complete` | Model finished speaking |\n| `interrupted` | User interrupted the model |\n| `session-resumed` | Session resumption handle updated |\n| `error` | Error from Gemini |\n\n## Key Details\n\n- **Audio format**: 16-bit PCM at 24kHz\n- **Default model**: `gemini-2.5-flash-native-audio-preview`\n- **Session resumption**: Automatic — `GeminiLiveSession` tracks resumption handles\n- **Tool execution**: Uses AriaFlow's `ToolExecutor` with timeout support\n- **State persistence**: Transcripts are saved to session via `ConversationState`\n\n## Peer Dependencies\n\n- `@ariaflowagents/core` — Foundation primitives (ToolExecutor, ConversationState, etc.)\n- `ai` (v6+) — Vercel AI SDK\n- `zod` — Schema definitions for tools\n","readmeFilename":"README.md","_rev":"1-4d3d957c909893fe46148da9d20bb7fe"}