{"_id":"@anzalabidi/agentreplay","_rev":"2-46eb5f0e07e1565b42b2263267fc8c7e","name":"@anzalabidi/agentreplay","dist-tags":{"latest":"0.2.0-alpha.2"},"versions":{"0.2.0-alpha.1":{"name":"@anzalabidi/agentreplay","version":"0.2.0-alpha.1","keywords":["ai-agents","evals","trace-protocol","replay","regression","harness"],"license":"Apache-2.0","_id":"@anzalabidi/agentreplay@0.2.0-alpha.1","maintainers":[{"name":"anzalabidi","email":"anzalabidi@gmail.com"}],"bin":{"agentreplay":"bin/agentreplay.js"},"dist":{"shasum":"cf9817491caa0b7a293442699c76996cd48506c7","tarball":"https://registry.npmjs.org/@anzalabidi/agentreplay/-/agentreplay-0.2.0-alpha.1.tgz","fileCount":49,"integrity":"sha512-XXQvk38VirtX9eJ2i6xtByzPJV91ylbs9clOQAOO+u80l2xHsoLqsugYcIl/EPANfDoFkTya6MiqGGX+QXAEwg==","signatures":[{"sig":"MEYCIQC6HuafECgSDlBTJ/wWdLhO1Ch3H/H4N5oL0lbAqChz4wIhAMoE9fAZ2w1tODvuHs+ddyLVZdxJK2w/ABbefyKvoVPj","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":212822},"main":"./src/index.js","type":"module","engines":{"node":">=20"},"exports":{".":"./src/index.js","./replay":"./src/replay/runner.js","./adapters/http":"./src/adapters/httpRecorder.js","./adapters/state":"./src/adapters/stateAdapter.js"},"gitHead":"d11e8b1384b0298c2dbfdde4758fd45423569573","scripts":{"demo":"node ./bin/agentreplay.js demo","test":"node --test","serve":"node ./bin/agentreplay.js serve --port 4177","validate":"node ./bin/agentreplay.js validate ./traces/billing-fixed-run.json"},"_npmUser":{"name":"anzalabidi","email":"anzalabidi@gmail.com"},"_npmVersion":"10.9.4","description":"Language-neutral trace protocol and replay harness for production AI agents.","directories":{},"_nodeVersion":"22.21.0","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/agentreplay_0.2.0-alpha.1_1778082007764_0.2815455350593674","host":"s3://npm-registry-packages-npm-production"}},"0.2.0-alpha.2":{"name":"@anzalabidi/agentreplay","version":"0.2.0-alpha.2","description":"Language-neutral trace protocol and replay harness for production AI agents.","type":"module","main":"./src/index.js","exports":{".":"./src/index.js","./adapters/http":"./src/adapters/httpRecorder.js","./adapters/state":"./src/adapters/stateAdapter.js","./replay":"./src/replay/runner.js"},"bin":{"agentreplay":"bin/agentreplay.js"},"scripts":{"demo":"node ./bin/agentreplay.js demo","serve":"node ./bin/agentreplay.js serve --port 4177","test":"node --test","validate":"node ./bin/agentreplay.js validate ./traces/billing-fixed-run.json"},"keywords":["ai-agents","evals","trace-protocol","replay","regression","harness"],"license":"Apache-2.0","engines":{"node":">=20"},"_id":"@anzalabidi/agentreplay@0.2.0-alpha.2","gitHead":"a14515e154cbfe05ffd54f15f777e9ef124d4500","_nodeVersion":"22.21.0","_npmVersion":"10.9.4","dist":{"integrity":"sha512-qo9HqsBYnM+1vj2EUb9M6JEw0zKyQ7vAGnHJ0ISju6Uta87LfZl+60bhFDNu6JVa5b8MnNmLraXmBs6neLQRAQ==","shasum":"3f6f72a82187959f0f596b157175f2533acaca0e","tarball":"https://registry.npmjs.org/@anzalabidi/agentreplay/-/agentreplay-0.2.0-alpha.2.tgz","fileCount":49,"unpackedSize":212906,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIDKKwJgtfOwkJKnC9dUlx2NrL6x2kBUVkdhXAE6HffmyAiBkPjaT40PpDQ5l+RyuRaMqLhNzFMuans3J4ef8ROlBPg=="}]},"_npmUser":{"name":"anzalabidi","email":"anzalabidi@gmail.com"},"directories":{},"maintainers":[{"name":"anzalabidi","email":"anzalabidi@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/agentreplay_0.2.0-alpha.2_1778082267646_0.3504206545218791"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-06T15:40:07.699Z","modified":"2026-05-06T15:44:27.921Z","0.2.0-alpha.1":"2026-05-06T15:40:08.025Z","0.2.0-alpha.2":"2026-05-06T15:44:27.808Z"},"license":"Apache-2.0","keywords":["ai-agents","evals","trace-protocol","replay","regression","harness"],"description":"Language-neutral trace protocol and replay harness for production AI agents.","maintainers":[{"name":"anzalabidi","email":"anzalabidi@gmail.com"}],"readme":"# AgentReplay\n\nAgentReplay is a language-neutral trace protocol and replay harness for production AI agents.\n\nIt is built around one product promise:\n\n> Turn every failed agent run into a replayable test.\n\nAgent observability tells you what happened. AgentReplay freezes the run as `agentreplay.trace.v1` JSON so you can replay, diff, and gate future model, prompt, tool, or workflow changes against the same production failure.\n\n## Why this exists\n\nAgents fail differently than chatbots. The final text can look fine while the agent touched the wrong external state, called tools in the wrong order, skipped approval, or acted on stale context.\n\nAgentReplay captures:\n\n- user input\n- agent metadata\n- tool schemas and calls\n- tool arguments and responses\n- pre/post state snapshots\n- side-effect diffs\n- approvals\n- final output\n- deterministic assertions\n\nThe resulting trace becomes a regression artifact that can run in CI.\n\n## Quick demo\n\n```bash\nnpm test\nnpm run demo\nnpm install @anzalabidi/agentreplay\nnpx --yes --package @anzalabidi/agentreplay agentreplay validate ./examples/protocol/raw-trace.json --json\nnode ./bin/agentreplay.js demo\nnode ./bin/agentreplay.js init ./my-agent-repo\nnode ./bin/agentreplay.js inspect ./traces/billing-bad-run.json\nnode ./bin/agentreplay.js gate ./traces/billing-bad-run.json\nnode ./bin/agentreplay.js gate ./traces/billing-fixed-run.json\nnode ./bin/agentreplay.js gate ./traces/billing-fixed-run.json --json\nnode ./bin/agentreplay.js diff ./traces/billing-bad-run.json ./traces/billing-fixed-run.json\n```\n\nThe demo records two billing-agent runs:\n\n- a bad run that refunds the wrong invoice and sends an email without approval\n- a fixed run that refunds the duplicate invoice and drafts the email\n\n## SDKs and raw JSON\n\nThe Node SDK is the reference implementation. Python and Go SDKs emit the same trace semantics for teams outside the Node ecosystem. Other languages can write raw `agentreplay.trace.v1` JSON and still use the CLI:\n\n```bash\nagentreplay validate ./examples/protocol/raw-trace.json --json\nagentreplay gate ./examples/protocol/raw-trace.json --json\n```\n\nSee [docs/TRACE_SPEC.md](./docs/TRACE_SPEC.md) for required fields, hashing, redaction, approvals, side effects, and replay semantics.\n\nConformance examples:\n\n- [examples/conformance/node.js](./examples/conformance/node.js)\n- [sdk/python/examples/conformance.py](./sdk/python/examples/conformance.py)\n- [sdk/go/examples/conformance/main.go](./sdk/go/examples/conformance/main.go)\n- [examples/protocol/raw-trace.json](./examples/protocol/raw-trace.json)\n\nFramework boundary examples:\n\n- [examples/integrations/openai-agents-tool-boundary.js](./examples/integrations/openai-agents-tool-boundary.js)\n- [examples/integrations/langgraph-tool-boundary.js](./examples/integrations/langgraph-tool-boundary.js)\n- [examples/integrations/http-webhook-raw-json.js](./examples/integrations/http-webhook-raw-json.js)\n\n## Node SDK\n\n```js\nimport { AgentReplayHarness } from \"@anzalabidi/agentreplay\";\n\nconst harness = new AgentReplayHarness({\n  project: \"billing-ops\",\n  agent: {\n    name: \"refund-agent\",\n    model: \"gpt-5.5\",\n    promptHash: \"sha256:...\"\n  }\n});\n\nconst refund = harness.wrapTool(\n  \"stripe.refund\",\n  async ({ invoiceId, amount }) => {\n    return stripe.refunds.create({ invoice: invoiceId, amount });\n  },\n  {\n    snapshot: async ({ invoiceId }) => stripe.invoices.retrieve(invoiceId),\n    diff: async (before, after) => [\n      {\n        type: \"invoice_status_changed\",\n        before: before.status,\n        after: after.status\n      }\n    ]\n  }\n);\n\nharness.recordInput({ message: \"Refund the duplicate invoice.\" });\nawait refund({ invoiceId: \"in_123\", amount: 4999 });\nharness.recordFinalOutput({ status: \"done\" });\nawait harness.save(\"./traces/prod-incident-123.json\", {\n  expectedOutcome: {\n    assertions: [\n      {\n        type: \"tool_not_called\",\n        name: \"never_send_email_without_approval\",\n        tool: \"gmail.send\"\n      }\n    ]\n  }\n});\n```\n\n## CLI\n\n```bash\nagentreplay inspect <trace.json> [--json]\nagentreplay validate <trace.json> [--json]\nagentreplay gate <trace.json> [--json]\nagentreplay diff <baseline.json> <candidate.json> [--json]\nagentreplay init [directory] [--force] [--json]\n```\n\nUse `--json` in CI so failures can be stored as build artifacts instead of scraped from terminal text.\n\nUse `agentreplay init` to scaffold `traces/gates`, `traces/incidents`, and a GitHub Actions workflow that validates and gates release-critical traces.\n\n## Current proof\n\nThese are checked by the repository test suite and public fixtures:\n\n- `npm test`: 21/21 tests passing.\n- Billing fixture: bad trace fails, fixed trace passes, diff shows changed tool arguments and side effects.\n- CRM adversarial pilot: 20/20 vulnerable runs caught, 20/20 hardened runs passed, 20/20 vulnerable-to-fixed diffs detected.\n- CRM bad trace fails on injection-shaped query syntax, missing approval, and multi-record mutation.\n- CRM fixed trace passes all gates.\n- Node, Python, and Go conformance traces validate, gate, redact secrets/PII, and produce matching tool-call hashes.\n- Hand-written raw JSON trace validates and gates without any SDK.\n- Public package sanitization check rejects private CRM pilot identifiers.\n- `npm pack --dry-run` ships only the public protocol, SDKs, examples, docs, console, and representative traces.\n\n## Assertion types\n\nCurrent deterministic assertions:\n\n- `tool_called`\n- `tool_not_called`\n- `requires_approval`\n- `tool_order`\n- `arg_equals`\n- `arg_matches`\n- `arg_not_matches`\n- `response_equals`\n- `max_tool_calls`\n- `side_effect_exists`\n- `side_effect_count`\n- `no_replay_mismatches`\n- `redaction_applied`\n\nThe product should stay deterministic first. LLM judges can be useful later, but they should not be the foundation of a regression harness.\n\n## Market readiness\n\nAgentReplay is currently suitable as an installable alpha for engineering teams building tool-using agents in high-stakes workflows. It has a working protocol, Node reference SDK/CLI, Python and Go trace SDKs, replay engine, deterministic gates, side-effect diffs, redaction, a local console, and passing product tests.\n\nIt is not yet a hosted enterprise platform. Before a public paid launch, the remaining decisions are package publishing, security review, hosted trace storage, and polished framework adapters for the ecosystems buyers already use.\n\nSee [docs/INTEGRATIONS.md](./docs/INTEGRATIONS.md) for the recommended tool-boundary integration pattern.\nSee [docs/READINESS.md](./docs/READINESS.md) for the current product-level checklist.\n\nThe repo also includes a generic CRM adversarial workflow in [examples/crm-agent-workflow](./examples/crm-agent-workflow). It proves the harness can catch injection-shaped CRM queries, unapproved CRM writes, and over-broad record mutation without exposing any private implementation. See [docs/CRM_PILOT.md](./docs/CRM_PILOT.md) for the 20-case pilot result.\n\n## Initial wedge\n\nThe first commercial wedge is billing, finance ops, and RevOps agents:\n\n- actions are high-stakes\n- APIs are structured\n- state diffs are clear\n- auditability matters\n- failures are easy to understand\n\nThe launch story:\n\n> Evals test imagined cases. Production creates the real cases. AgentReplay turns those real cases into regression tests.\n","readmeFilename":"README.md"}