{"_id":"@alicantorun/aimetrics-sdk","name":"@alicantorun/aimetrics-sdk","dist-tags":{"latest":"1.0.0"},"versions":{"1.0.0":{"name":"@alicantorun/aimetrics-sdk","version":"1.0.0","description":"SDK for AiMetrics - LLM Observability and Analytics","private":false,"main":"dist/index.js","types":"dist/index.d.ts","scripts":{"build":"tsc","test":"jest","prepare":"npm run build","prepublishOnly":"npm run build"},"dependencies":{"axios":"^1.6.0","ws":"^8.14.0"},"devDependencies":{"@types/node":"^20.0.0","@types/ws":"^8.5.0","typescript":"^5.0.0","jest":"^29.0.0","@types/jest":"^29.0.0","ts-jest":"^29.0.0"},"peerDependencies":{"openai":"^4.0.0"},"keywords":["llm","ai","metrics","observability","analytics","openai","monitoring","performance"],"author":{"name":"Alican Torun"},"license":"MIT","publishConfig":{"access":"public"},"gitHead":"542289496c68884eedbc681945dca50fa966b3ec","_id":"@alicantorun/aimetrics-sdk@1.0.0","_nodeVersion":"18.17.1","_npmVersion":"9.6.7","dist":{"integrity":"sha512-mJyAxdBrBGuKLLG5y9m5dLCK+q2G75G+P+71BcYOEm1D4am9FqbQ3GkacgYn/zMG2N4pXSaYlKO7CAzyoH096g==","shasum":"3061b4e749a6415cbf6154a1c55b6fb22106abe7","tarball":"https://registry.npmjs.org/@alicantorun/aimetrics-sdk/-/aimetrics-sdk-1.0.0.tgz","fileCount":20,"unpackedSize":36259,"signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIDGmsU7cAdIgBMDG3gRTGNbF8LwgRdVpBgk33YpM3e4jAiA017V2lnZwMKjgtCBm5VEVXnP+DBqwUKZLmINArj0phQ=="}]},"_npmUser":{"name":"alicantorun","email":"torun.alican@gmail.com"},"directories":{},"maintainers":[{"name":"alicantorun","email":"torun.alican@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/aimetrics-sdk_1.0.0_1733773208741_0.4651218946065403"},"_hasShrinkwrap":false}},"time":{"created":"2024-12-09T19:40:08.597Z","1.0.0":"2024-12-09T19:40:08.953Z","modified":"2024-12-09T19:40:09.188Z"},"maintainers":[{"name":"alicantorun","email":"torun.alican@gmail.com"}],"description":"SDK for AiMetrics - LLM Observability and Analytics","keywords":["llm","ai","metrics","observability","analytics","openai","monitoring","performance"],"author":{"name":"Alican Torun"},"license":"MIT","readme":"# AiMetrics SDK\n\nA comprehensive SDK for tracking and analyzing LLM/AI system metrics and performance.\n\n## Installation\n\n```bash\nnpm install @aimetrics/sdk\n```\n\nIf you're using OpenAI (which is a peer dependency):\n\n```bash\nnpm install @aimetrics/sdk openai\n```\n\n## Quick Start\n\n```typescript\nimport { AiMetricsTracker } from \"@aimetrics/sdk\";\nimport OpenAI from \"openai\";\n\n// Initialize the metrics tracker\nconst metrics = new AiMetricsTracker({\n    apiKey: \"your-metrics-api-key\", // Get this from AiMetrics dashboard\n    clientId: \"your-client-id\", // Your unique identifier\n    endpoint: \"https://api.aimetrics.ai\", // Optional: defaults to localhost:3001\n    batchSize: 10, // Optional: batch size for sending metrics\n    flushInterval: 5000, // Optional: flush interval in ms\n    debug: true, // Optional: enable debug logging\n});\n\n// Initialize your LLM client\nconst openai = new OpenAI({\n    apiKey: process.env.OPENAI_API_KEY,\n});\n\n// Track LLM calls\nasync function chatCompletion(messages) {\n    return await metrics.track(\n        {\n            model: \"gpt-3.5-turbo\",\n            messages,\n        },\n        async () => {\n            const response = await openai.chat.completions.create({\n                model: \"gpt-3.5-turbo\",\n                messages,\n            });\n            return response;\n        }\n    );\n}\n\n// Example usage\nasync function main() {\n    try {\n        const response = await chatCompletion([\n            { role: \"system\", content: \"You are a helpful assistant.\" },\n            { role: \"user\", content: \"Hello, how are you?\" },\n        ]);\n        console.log(response.choices[0].message);\n    } catch (error) {\n        console.error(\"Error:\", error);\n    }\n}\n\n// Get metrics\nasync function getMetrics() {\n    const startDate = new Date(Date.now() - 24 * 60 * 60 * 1000); // Last 24 hours\n    const metrics = await metrics.getMetrics(startDate);\n    console.log(\"Metrics:\", metrics);\n}\n\n// Clean up when done\nfunction cleanup() {\n    metrics.destroy();\n}\n```\n\n## Features\n\n-   Real-time metrics tracking\n-   Token usage monitoring\n-   Cost calculation\n-   Response quality analysis\n-   Performance metrics\n-   Content analysis\n-   Batch processing\n-   Error tracking\n\n## Configuration\n\nThe SDK accepts the following configuration options:\n\n| Option        | Type    | Required | Default                   | Description                               |\n| ------------- | ------- | -------- | ------------------------- | ----------------------------------------- |\n| apiKey        | string  | Yes      | -                         | Your AiMetrics API key                    |\n| clientId      | string  | Yes      | -                         | Your unique client identifier             |\n| endpoint      | string  | No       | http://localhost:3001/api | Custom metrics server endpoint            |\n| batchSize     | number  | No       | 10                        | Number of metrics to batch before sending |\n| flushInterval | number  | No       | 5000                      | Interval in ms to flush metrics           |\n| debug         | boolean | No       | false                     | Enable debug logging                      |\n\n## Metrics Tracked\n\n### Basic Metrics\n\n-   Total calls\n-   Success/failure rate\n-   Response times\n-   Token usage\n-   Costs\n\n### Quality Metrics\n\n-   Response coherence\n-   Relevance scores\n-   Toxicity detection\n-   Content analysis\n\n### Performance Metrics\n\n-   Time to first token\n-   Tokens per second\n-   Retry counts\n-   Error rates\n\n### Content Analysis\n\n-   Word count\n-   Code snippet detection\n-   Programming languages used\n-   Sentiment analysis\n-   Average word length\n\n## Error Handling\n\nThe SDK includes comprehensive error handling:\n\n```typescript\ntry {\n    const response = await metrics.track(\n        {\n            model: \"gpt-3.5-turbo\",\n            messages: [{ role: \"user\", content: \"Hello\" }],\n        },\n        async () => {\n            // Your LLM call here\n        }\n    );\n} catch (error) {\n    if (error.response) {\n        console.error(\"API Error:\", error.response.data);\n    } else {\n        console.error(\"Error:\", error.message);\n    }\n}\n```\n\n## Best Practices\n\n1. Initialize the tracker once and reuse the instance\n2. Use appropriate batch sizes for your use case\n3. Call `destroy()` when shutting down your application\n4. Enable debug mode during development\n5. Handle errors appropriately\n6. Use environment variables for sensitive data\n\n## Support\n\nFor issues and feature requests, please visit our [GitHub repository](https://github.com/yourusername/aimetrics/issues).\n\n## License\n\nMIT License\n","readmeFilename":"README.md"}