{"_id":"@beledevere/traffic-cop-client","name":"@beledevere/traffic-cop-client","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@beledevere/traffic-cop-client","version":"0.1.0","description":"JavaScript client for Traffic Cop API, an intelligent middleware for optimizing LLM API usage","main":"dist/index.js","types":"dist/index.d.ts","module":"dist/index.mjs","exports":{".":{"require":"./dist/index.js","import":"./dist/index.mjs","types":"./dist/index.d.ts"}},"scripts":{"build":"tsc && tsc -p tsconfig.esm.json","test":"jest","lint":"eslint src --ext .ts","format":"prettier --write \"src/**/*.ts\"","prepublishOnly":"npm run build","prepare":"npm run build"},"keywords":["llm","ai","middleware","traffic-cop","openai","anthropic","claude","gpt","machine-learning"],"author":{"name":"Traffic Cop Team","email":"support@trafficcop.ai"},"homepage":"https://trafficcop.ai","repository":{"type":"git","url":"git+https://github.com/traffic-cop/traffic-cop-js-sdk.git"},"bugs":{"url":"https://github.com/traffic-cop/traffic-cop-js-sdk/issues"},"license":"MIT","dependencies":{"axios":"^1.4.0","uuid":"^9.0.0"},"devDependencies":{"@types/jest":"^29.5.0","@types/node":"^18.15.0","@types/uuid":"^9.0.1","@typescript-eslint/eslint-plugin":"^5.54.0","@typescript-eslint/parser":"^5.54.0","eslint":"^8.35.0","jest":"^29.5.0","prettier":"^2.8.4","ts-jest":"^29.1.0","typescript":"^5.0.2"},"engines":{"node":">=14.0.0"},"_id":"@beledevere/traffic-cop-client@0.1.0","_nodeVersion":"23.11.0","_npmVersion":"10.9.2","dist":{"integrity":"sha512-Fl1+fFSl03fVw+Q7ZFDOPx6w1tWjeWUcih48qvhPRSOB+GvskWJOtZJAgl7GBSv6SqrEUIR6XIfdAYVGjt1Ryw==","shasum":"2419e40d211a036ad88c81db27f455dcfaa440cb","tarball":"https://registry.npmjs.org/@beledevere/traffic-cop-client/-/traffic-cop-client-0.1.0.tgz","fileCount":13,"unpackedSize":69371,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIFqk6sYrs1zFdppR++RTAqqNpj+00WM++kLLuag5XdrgAiAFqDk2cWnH75B51Imdh/WUHR9koYvqSWSsV5j46IhBmw=="}]},"_npmUser":{"name":"beledevere","email":"dumko.raj@gmail.com"},"directories":{},"maintainers":[{"name":"beledevere","email":"dumko.raj@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/traffic-cop-client_0.1.0_1746753629291_0.8072838341014339"},"_hasShrinkwrap":false}},"time":{"created":"2025-05-09T01:20:29.168Z","0.1.0":"2025-05-09T01:20:29.455Z","modified":"2025-05-09T01:20:29.725Z"},"maintainers":[{"name":"beledevere","email":"dumko.raj@gmail.com"}],"description":"JavaScript client for Traffic Cop API, an intelligent middleware for optimizing LLM API usage","homepage":"https://trafficcop.ai","keywords":["llm","ai","middleware","traffic-cop","openai","anthropic","claude","gpt","machine-learning"],"repository":{"type":"git","url":"git+https://github.com/traffic-cop/traffic-cop-js-sdk.git"},"author":{"name":"Traffic Cop Team","email":"support@trafficcop.ai"},"bugs":{"url":"https://github.com/traffic-cop/traffic-cop-js-sdk/issues"},"license":"MIT","readme":"# Traffic Cop JavaScript Client\n\n[![npm version](https://badge.fury.io/js/traffic-cop-client.svg)](https://badge.fury.io/js/traffic-cop-client)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![TypeScript](https://img.shields.io/badge/TypeScript-4.9.5-blue.svg)](https://www.typescriptlang.org/)\n\nA JavaScript/TypeScript client for the Traffic Cop API, an intelligent middleware for optimizing LLM API usage.\n\n## What is Traffic Cop?\n\nTraffic Cop is a middleware SaaS that optimizes LLM API usage by intelligently routing requests through cost-effective 'Draft' models and high-fidelity 'Verify' models. It helps you:\n\n- **Reduce LLM API costs** by using smaller models when appropriate\n- **Maintain high quality** by verifying with larger models when needed\n- **Collect valuable data** on model performance and confidence\n- **Optimize your LLM strategy** with data-driven insights\n\nTraffic Cop supports two execution modes:\n1. **Advise Mode** (default): Traffic Cop provides recommendations, your application executes the LLM calls\n2. **Proxy Mode**: Traffic Cop executes the LLM calls on your behalf\n\n## Installation\n\n```bash\nnpm install traffic-cop-client\n# or\nyarn add traffic-cop-client\n```\n\n## Usage\n\nTraffic Cop supports two execution modes:\n\n- **Advise Mode** (default): Traffic Cop provides recommendations on whether to use a draft model or verify model, but the client executes the LLM calls.\n- **Proxy Mode**: Traffic Cop executes the LLM calls on behalf of the client.\n\n### Advise Mode (Client Executes LLM Calls)\n\nAdvise Mode is the default and recommended mode for most users. In this mode:\n\n1. Traffic Cop provides recommendations on whether to use a draft model or verify model\n2. Your application executes the LLM calls based on this advice\n3. You report the outcome back to Traffic Cop to help improve future recommendations\n\n#### Example with OpenAI\n\n```typescript\nimport { TrafficCopClient, ExecutionMode, AdviseDecision } from 'traffic-cop-client';\nimport OpenAI from 'openai';\n\n// Initialize clients\nconst trafficCop = new TrafficCopClient({\n  apiKey: 'your-traffic-cop-api-key',\n});\n\nconst openai = new OpenAI({\n  apiKey: 'your-openai-api-key',\n});\n\n// Route a prompt in advise mode\nasync function adviseExample() {\n  try {\n    const prompt = 'What is the capital of France?';\n    const draftModelId = 'gpt-3.5-turbo';\n    const verifyModelId = 'gpt-4';\n\n    // Step 1: Get advice from Traffic Cop\n    const advice = await trafficCop.route({\n      prompt,\n      draftModelId,\n      verifyModelId,\n      executionMode: ExecutionMode.ADVISE, // Default, can be omitted\n      userId: 'user-123', // Optional, will be generated if not provided\n      metadata: { // Optional\n        source: 'web-app',\n        sessionId: 'session-456',\n      },\n    });\n\n    console.log(`Decision: ${advice.decision}`);\n    console.log(`Suggested draft model: ${advice.suggestedDraftModelId}`);\n    console.log(`Suggested verify model: ${advice.suggestedVerifyModelId}`);\n\n    // Step 2: Execute the draft model call\n    const startDraftTime = Date.now();\n    const draftResponse = await openai.chat.completions.create({\n      model: draftModelId,\n      messages: [{ role: 'user', content: prompt }],\n    });\n    const draftLatencyMs = Date.now() - startDraftTime;\n\n    const draftContent = draftResponse.choices[0].message.content;\n    const draftTokenCount = draftResponse.usage.total_tokens;\n\n    console.log(`Draft response: ${draftContent}`);\n\n    // Step 3: Decide whether to verify based on Traffic Cop's advice\n    const shouldVerify = advice.decision === AdviseDecision.VERIFICATION_RECOMMENDED;\n    let verifyContent = null;\n    let verifyTokenCount = null;\n    let verifyLatencyMs = null;\n\n    if (shouldVerify) {\n      // Execute the verify model call\n      const startVerifyTime = Date.now();\n      const verifyResponse = await openai.chat.completions.create({\n        model: verifyModelId,\n        messages: [{ role: 'user', content: prompt }],\n      });\n      verifyLatencyMs = Date.now() - startVerifyTime;\n\n      verifyContent = verifyResponse.choices[0].message.content;\n      verifyTokenCount = verifyResponse.usage.total_tokens;\n\n      console.log(`Verify response: ${verifyContent}`);\n    }\n\n    // Step 4: Choose the final response\n    const finalContent = verifyContent || draftContent;\n\n    // Step 5: Report the outcome back to Traffic Cop\n    const outcome = await trafficCop.reportExecutionOutcome({\n      trafficCopRequestId: advice.trafficCopRequestId,\n      userId: 'user-123', // Use the same userId that was passed to route()\n      actualDraftModelUsed: draftModelId,\n      draftTokenCount: draftTokenCount,\n      draftLatencyMs: draftLatencyMs,\n      wasVerificationPerformed: shouldVerify,\n      finalResponse: finalContent,\n      actualVerifyModelUsed: shouldVerify ? verifyModelId : undefined,\n      verifyTokenCount: verifyTokenCount,\n      verifyLatencyMs: verifyLatencyMs,\n      qualityFeedback: 0.95, // Optional feedback score (0-1)\n    });\n\n    console.log(`Outcome reported: ${outcome.success}`);\n    console.log(`Final response: ${finalContent}`);\n  } catch (error) {\n    console.error('Error:', error);\n  }\n}\n\nadviseExample();\n```\n\n#### Example with Anthropic\n\n```typescript\nimport { TrafficCopClient, ExecutionMode, AdviseDecision } from 'traffic-cop-client';\nimport Anthropic from '@anthropic-ai/sdk';\n\n// Initialize clients\nconst trafficCop = new TrafficCopClient({\n  apiKey: 'your-traffic-cop-api-key',\n});\n\nconst anthropic = new Anthropic({\n  apiKey: 'your-anthropic-api-key',\n});\n\nasync function adviseWithAnthropic() {\n  try {\n    const prompt = 'What is the capital of France?';\n    const draftModelId = 'claude-instant-1';\n    const verifyModelId = 'claude-2';\n\n    // Step 1: Get advice from Traffic Cop\n    const advice = await trafficCop.route({\n      prompt,\n      draftModelId,\n      verifyModelId,\n    });\n\n    console.log(`Decision: ${advice.decision}`);\n\n    // Step 2: Execute the draft model call\n    const startDraftTime = Date.now();\n    const draftResponse = await anthropic.messages.create({\n      model: draftModelId,\n      messages: [{ role: 'user', content: prompt }],\n      max_tokens: 1000,\n    });\n    const draftLatencyMs = Date.now() - startDraftTime;\n\n    const draftContent = draftResponse.content[0].text;\n    // Anthropic doesn't provide token count directly in the response\n    // This is a rough estimation - in production, consider using a proper tokenizer\n    // like @anthropic-ai/tokenizer or a similar library for accurate counts\n    const draftTokenCount = Math.ceil((prompt.length + draftContent.length) / 4);\n\n    // Step 3: Decide whether to verify based on Traffic Cop's advice\n    const shouldVerify = advice.decision === AdviseDecision.VERIFICATION_RECOMMENDED;\n    let verifyContent = null;\n    let verifyTokenCount = null;\n    let verifyLatencyMs = null;\n\n    if (shouldVerify) {\n      const startVerifyTime = Date.now();\n      const verifyResponse = await anthropic.messages.create({\n        model: verifyModelId,\n        messages: [{ role: 'user', content: prompt }],\n        max_tokens: 1000,\n      });\n      verifyLatencyMs = Date.now() - startVerifyTime;\n\n      verifyContent = verifyResponse.content[0].text;\n      // Same token count estimation as for draft model\n      verifyTokenCount = Math.ceil((prompt.length + verifyContent.length) / 4);\n    }\n\n    // Step 4: Choose the final response\n    const finalContent = verifyContent || draftContent;\n\n    // Step 5: Report the outcome back to Traffic Cop\n    await trafficCop.reportExecutionOutcome({\n      trafficCopRequestId: advice.trafficCopRequestId,\n      userId: 'user-123', // Use the same userId that was passed to route() or that was auto-generated\n      actualDraftModelUsed: draftModelId,\n      draftTokenCount: draftTokenCount,\n      draftLatencyMs: draftLatencyMs,\n      wasVerificationPerformed: shouldVerify,\n      finalResponse: finalContent,\n      actualVerifyModelUsed: shouldVerify ? verifyModelId : undefined,\n      verifyTokenCount: verifyTokenCount,\n      verifyLatencyMs: verifyLatencyMs,\n    });\n\n    console.log(`Final response: ${finalContent}`);\n  } catch (error) {\n    console.error('Error:', error);\n  }\n}\n```\n\n### Proxy Mode (Traffic Cop Executes LLM Calls)\n\nIn Proxy Mode, Traffic Cop executes the LLM calls on your behalf. Important notes about API keys:\n\n- **Gemini models**: Traffic Cop can use its own managed API keys for Gemini models (e.g., `gemini-pro`, `text-bison`).\n- **Non-Gemini models**: You must provide your own API keys for OpenAI (e.g., `gpt-3.5-turbo`, `gpt-4`) and Anthropic (e.g., `claude-instant`, `claude-2`) models.\n\n```typescript\nimport { TrafficCopClient, ExecutionMode } from 'traffic-cop-client';\n\n// Create a client\nconst client = new TrafficCopClient({\n  apiKey: 'your-api-key',\n});\n\n// Example with OpenAI models (requires customer API key)\nasync function proxyOpenAIExample() {\n  try {\n    const response = await client.route({\n      prompt: 'What is the capital of France?',\n      draftModelId: 'gpt-3.5-turbo',\n      verifyModelId: 'gpt-4',\n      executionMode: ExecutionMode.PROXY,\n      userId: 'user-123', // Optional, will be generated if not provided\n      customerApiKeys: { // Required for OpenAI models\n        openai: 'sk-your-openai-key',\n      },\n      metadata: { // Optional\n        source: 'web-app',\n        sessionId: 'session-456',\n      },\n    });\n\n    console.log(`Final response: ${response.finalResponse}`);\n    console.log(`Verification used: ${response.verificationUsed}`);\n    console.log(`Estimated cost saved: $${response.estimatedCostSaved?.toFixed(6)}`);\n  } catch (error) {\n    console.error('Error:', error);\n  }\n}\n\n// Example with Gemini models (Traffic Cop's managed key can be used)\nasync function proxyGeminiExample() {\n  try {\n    const response = await client.route({\n      prompt: 'What is the capital of France?',\n      draftModelId: 'gemini-pro',\n      verifyModelId: 'gemini-pro-1.5',\n      executionMode: ExecutionMode.PROXY,\n      userId: 'user-123',\n      // No customerApiKeys needed for Gemini models\n    });\n\n    console.log(`Final response: ${response.finalResponse}`);\n    console.log(`Verification used: ${response.verificationUsed}`);\n  } catch (error) {\n    console.error('Error:', error);\n  }\n}\n\nproxyOpenAIExample();\nproxyGeminiExample();\n```\n\n### CommonJS\n\n```javascript\nconst { TrafficCopClient, ExecutionMode } = require('traffic-cop-client');\n\n// Create a client\nconst client = new TrafficCopClient({\n  apiKey: 'your-api-key',\n});\n\n// Route a prompt in advise mode (default)\nclient.route({\n  prompt: 'What is the capital of France?',\n  draftModelId: 'gpt-3.5-turbo',\n  verifyModelId: 'gpt-4',\n})\n  .then(advice => {\n    console.log(`Decision: ${advice.decision}`);\n    // Client would make their own LLM calls here\n    // ...\n\n    // Then report the outcome\n    return client.reportExecutionOutcome({\n      trafficCopRequestId: advice.trafficCopRequestId,\n      userId: 'user-123', // Use the same userId that was passed to route() or that was auto-generated\n      actualDraftModelUsed: advice.suggestedDraftModelId,\n      draftTokenCount: 15,\n      draftLatencyMs: 250,\n      wasVerificationPerformed: false,\n      finalResponse: \"Paris is the capital of France.\",\n    });\n  })\n  .then(outcome => {\n    console.log(`Outcome reported: ${outcome.success}`);\n  })\n  .catch(error => {\n    console.error('Error:', error);\n  });\n```\n\n## Configuration\n\nThe client can be configured with the following options:\n\n```typescript\ninterface TrafficCopClientOptions {\n  /**\n   * API key for authentication\n   */\n  apiKey: string;\n\n  /**\n   * Base URL for the Traffic Cop API\n   * @default \"https://traffic-cop-api-pbo3cvpjua-uc.a.run.app\"\n   */\n  baseUrl?: string;\n\n  /**\n   * Request timeout in milliseconds\n   * @default 60000\n   */\n  timeout?: number;\n}\n```\n\n## Response Format\n\nThe response from the `route` method depends on the execution mode:\n\n### Advise Mode Response\n\n```typescript\ninterface AdviseRouteResponse {\n  /**\n   * Unique request identifier\n   */\n  requestId: string;\n\n  /**\n   * Execution mode (always 'advise' for this response type)\n   */\n  executionMode: 'advise';\n\n  /**\n   * Unique identifier for tracking this request through the system\n   */\n  trafficCopRequestId: string;\n\n  /**\n   * Decision on whether verification is recommended\n   */\n  decision: 'verification_recommended' | 'draft_sufficient';\n\n  /**\n   * Suggested draft model to use\n   */\n  suggestedDraftModelId: string;\n\n  /**\n   * Suggested verify model to use if verification is needed\n   */\n  suggestedVerifyModelId: string;\n\n  /**\n   * Confidence threshold used for routing decision\n   */\n  thresholdUsed: number;\n}\n```\n\n### Proxy Mode Response\n\n```typescript\ninterface ProxyRouteResponse {\n  /**\n   * Unique request identifier\n   */\n  requestId: string;\n\n  /**\n   * Execution mode (always 'proxy' for this response type)\n   */\n  executionMode: 'proxy';\n\n  /**\n   * Unique identifier for tracking this request through the system\n   */\n  trafficCopRequestId: string;\n\n  /**\n   * Response from the draft model\n   */\n  draftResponse: {\n    content: string;\n    modelId: string;\n    tokensUsed: number;\n    latencyMs: number;\n    confidence?: number;\n    metadata?: Record<string, string>;\n  };\n\n  /**\n   * Response from the verify model (if used)\n   */\n  verifyResponse?: {\n    content: string;\n    modelId: string;\n    tokensUsed: number;\n    latencyMs: number;\n    confidence?: number;\n    metadata?: Record<string, string>;\n  };\n\n  /**\n   * The final response content to return to the user\n   */\n  finalResponse: string;\n\n  /**\n   * Whether the verify model was used\n   */\n  verificationUsed: boolean;\n\n  /**\n   * Estimated cost saved by using the draft model (if applicable)\n   */\n  estimatedCostSaved?: number;\n\n  /**\n   * Confidence threshold used for routing decision\n   */\n  thresholdUsed: number;\n}\n```\n\n### Report Execution Outcome Response\n\nThe response from the `reportExecutionOutcome` method:\n\n```typescript\ninterface ReportExecutionOutcomeResponse {\n  /**\n   * Whether the report was successfully processed\n   */\n  success: boolean;\n\n  /**\n   * The traffic_cop_request_id that was reported on\n   */\n  trafficCopRequestId: string;\n\n  /**\n   * Status message\n   */\n  message: string;\n}\n```\n\n> **Important Note on `userId`**: When calling `reportExecutionOutcome`, always use the same `userId` that was passed to the original `route()` call. This ensures consistent tracking of user interactions across the system. The `userId` represents the end-user identifier, while `trafficCopRequestId` is used to correlate the specific request-response pair.\n\n## Error Handling\n\nThe client will throw specific exceptions that you can catch to handle different types of errors:\n\n```typescript\nimport {\n  TrafficCopClient,\n  ExecutionMode,\n  TrafficCopError,\n  TrafficCopConnectionError,\n  TrafficCopAPIError\n} from 'traffic-cop-client';\n\nconst client = new TrafficCopClient({\n  apiKey: 'your-api-key',\n});\n\ntry {\n  const advice = await client.route({\n    prompt: 'What is the capital of France?',\n    draftModelId: 'gpt-3.5-turbo',\n    verifyModelId: 'gpt-4',\n  });\n\n  console.log(`Decision: ${advice.decision}`);\n\n  // Execute LLM calls based on the advice...\n\n} catch (error) {\n  if (error instanceof TrafficCopAPIError) {\n    // Handle API errors (e.g., invalid request, authentication error)\n    console.error(`API Error (Status ${error.statusCode}): ${error.detail}`);\n  } else if (error instanceof TrafficCopConnectionError) {\n    // Handle connection errors (e.g., network issues, timeouts)\n    console.error(`Connection Error: ${error.message}`);\n  } else if (error instanceof TrafficCopError) {\n    // Handle other Traffic Cop errors\n    console.error(`Traffic Cop Error: ${error.message}`);\n  } else {\n    // Handle unexpected errors\n    console.error(`Unexpected Error: ${error}`);\n  }\n}\n```\n\n### Exception Types\n\n- `TrafficCopError`: Base exception for all Traffic Cop client errors\n- `TrafficCopConnectionError`: Raised for connection errors (network issues, timeouts)\n- `TrafficCopAPIError`: Raised when the API returns an error response (includes statusCode and detail)\n\n## Contributing\n\nWe welcome contributions to the Traffic Cop JavaScript SDK! Please see [CONTRIBUTING.md](https://github.com/traffic-cop/traffic-cop-js-sdk/blob/main/CONTRIBUTING.md) for details on how to contribute.\n\n## Development\n\n### Setup\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/traffic-cop/traffic-cop-js-sdk.git\ncd traffic-cop-js-sdk\n```\n\n2. Install dependencies:\n```bash\nnpm install\n# or\nyarn\n```\n\n### Building\n\n```bash\nnpm run build\n# or\nyarn build\n```\n\n### Running Tests\n\n```bash\n# Run all tests\nnpm test\n# or\nyarn test\n\n# Run with coverage\nnpm test -- --coverage\n# or\nyarn test --coverage\n```\n\n### Code Style\n\nThis project uses:\n- [ESLint](https://eslint.org/) for linting\n- [Prettier](https://prettier.io/) for code formatting\n- [TypeScript](https://www.typescriptlang.org/) for type checking\n\n```bash\n# Lint code\nnpm run lint\n# or\nyarn lint\n\n# Format code\nnpm run format\n# or\nyarn format\n```\n\n## Support\n\nFor support, please:\n- Open an [issue](https://github.com/traffic-cop/traffic-cop-js-sdk/issues) on GitHub\n- Contact us at support@trafficcop.ai\n- Visit our [documentation](https://docs.trafficcop.ai)\n\n## License\n\nMIT\n\n## Links\n\n- [Traffic Cop Website](https://trafficcop.ai)\n- [Documentation](https://docs.trafficcop.ai)\n- [GitHub Repository](https://github.com/traffic-cop/traffic-cop-js-sdk)\n","readmeFilename":"README.md","_rev":"1-90b7bd77bafdf1c2edf738e754c4eb6c"}