{"_id":"@accelerate-data/promptfoo-eval-harness","_rev":"11-e71711da3b1c80a8b28621050b626a29","name":"@accelerate-data/promptfoo-eval-harness","dist-tags":{"latest":"1.8.0"},"versions":{"0.1.0":{"name":"@accelerate-data/promptfoo-eval-harness","version":"0.1.0","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@0.1.0","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"45fb3596365af4f6592160d966c6ee532328477d","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-0.1.0.tgz","fileCount":31,"integrity":"sha512-U2jXJpNhbcD8VouH1yhwEOTcZTRzUQ3+NCDIxTeXnYGbzQQXj3cruKo0q4VKP2lRqZVLc8eYloUdFA8l2a3yCw==","signatures":[{"sig":"MEYCIQDKVitaGEfhWtSII9vn2ZkhOhwVUhrzDprMedNKatSvIwIhAL1hQ9rToiN2JqGIwROVFVTz4Rhi3AiHjS64RD0ZTR+k","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@0.1.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":101334},"main":"scripts/framework/index.js","engines":{"node":">=18"},"gitHead":"1eeaf6e684f29e6d1b1da578b50d924d5eee13f3","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js","lint:md":"markdownlint '**/*.md' --ignore node_modules"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"js-yaml":"^4.1.1","promptfoo":"0.121.9","smol-toml":"^1.6.1"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.48.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_0.1.0_1777886540523_0.10977361213347558","host":"s3://npm-registry-packages-npm-production"}},"0.1.1":{"name":"@accelerate-data/promptfoo-eval-harness","version":"0.1.1","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@0.1.1","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"97b9cca765cc0bf381247445a4646ec911f04972","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-0.1.1.tgz","fileCount":32,"integrity":"sha512-izbULHGNnMCpOqNi+G7JRuS/KKKv+HV5/6mSCMO3o5iV2FofHTnIo0RE3WaxAHj2AyXJJMyFf4xrAiTk3VzLrA==","signatures":[{"sig":"MEUCIAu51rsy7gcdf/jWnXCAN7L6Jhuy+sHTDfi8c2wxwJ+BAiEA3BTrizbYHu7XVdshdJ3YxgPkoLiWHh1BiI367KgWsYM=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@0.1.1","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":103772},"main":"scripts/framework/index.js","engines":{"node":">=18"},"gitHead":"939ccf54242d5fabb512c37a7729d6694d65f004","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js","lint:md":"markdownlint '**/*.md' --ignore node_modules"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"js-yaml":"^4.1.1","promptfoo":"0.121.9","smol-toml":"^1.6.1"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.48.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_0.1.1_1777887768119_0.7052541806656818","host":"s3://npm-registry-packages-npm-production"}},"0.1.2":{"name":"@accelerate-data/promptfoo-eval-harness","version":"0.1.2","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@0.1.2","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"5b5c69891a5cd0026fd65824a7e489d3b2a60bfe","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-0.1.2.tgz","fileCount":32,"integrity":"sha512-eW8Q87DWQPL/Ze9uOWwITz7RwyXHMjU6x4WO8U3rOXZPHUEhzlmNof+2cIGCUhZX8VBWd6N5fZQZBIt619ybqQ==","signatures":[{"sig":"MEYCIQCEnKQVDILmEL3+N5NdVyzwekcItf8+h4g8RJqG+FArGwIhAJCNq+UdIZrL8HZ0YOjYbJBwKFZJfSEOqBeGhY29aaf2","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@0.1.2","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":103782},"main":"scripts/framework/index.js","engines":{"node":">=18"},"gitHead":"409c13b8c5d1eed9e8498db1a61b359335e61fac","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js","lint:md":"markdownlint '**/*.md' --ignore node_modules"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"js-yaml":"^4.1.1","promptfoo":"0.121.9","smol-toml":"^1.6.1"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.48.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_0.1.2_1777984727045_0.31857412444621414","host":"s3://npm-registry-packages-npm-production"}},"0.1.3":{"name":"@accelerate-data/promptfoo-eval-harness","version":"0.1.3","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@0.1.3","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"6f8f236eeacf3ce7bc688f46b98f61e1036005bd","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-0.1.3.tgz","fileCount":32,"integrity":"sha512-mMxZmE1GP1xuJR+92hIEJYZpU1JYEyugE2eoifYLiBPnU1rotc88UO5eS39+EVZY/Q9eFSWdhCD+WT0/djbwwA==","signatures":[{"sig":"MEQCIAdpTIvm8S7Lt2qQDSOegfYf8Y0Gc/KCZQu9Xkg1OW0fAiBOAYfduraUJ+gey54bmB3F7G9QgrHEg1UI707/9sPUGQ==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@0.1.3","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":103786},"main":"scripts/framework/index.js","engines":{"node":">=18"},"gitHead":"d61801a14fc6d0ccf5eb6c5014edb3f44bd0d0dd","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js","lint:md":"markdownlint '**/*.md' --ignore node_modules"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"js-yaml":"^4.1.1","promptfoo":"0.121.9","smol-toml":"^1.6.1"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.48.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_0.1.3_1777984895134_0.282785586446582","host":"s3://npm-registry-packages-npm-production"}},"1.5.0":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.5.0","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@1.5.0","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"0287ab2f66448766bca628256fae8023ae4576fd","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.5.0.tgz","fileCount":125,"integrity":"sha512-5H8loB8YQ3wTvDfoRNzxmT+tsKleoc7v0Jb/ksjJa0hU2A//7cymC+1Y2nBsDcQUydpw1r2bmRn6t5IcsbDEwg==","signatures":[{"sig":"MEQCIEgkI76V1irsIIJJevgumht23q6RBbN74YgVAmM/aBafAiABkUa7+DroGsEroMeSBmXyLkaC+MkK9R6NQctyuJq4lQ==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.5.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":836285},"main":"scripts/framework/index.js","engines":{"node":">=20"},"gitHead":"c4101aaa8e1e37ccde833e9fea5fe61797568d54","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","bench":"npm run bench:spawn-cost && npm run bench:throughput","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"ws":"^8.18.0","js-yaml":"^4.1.1","p-limit":"^3.1.0","promptfoo":"0.121.11","smol-toml":"^1.6.1","@openai/codex":"0.133.0","@opencode-ai/sdk":"1.15.10","@openai/codex-sdk":"0.133.0"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.48.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_1.5.0_1780043146448_0.4737750103107967","host":"s3://npm-registry-packages-npm-production"}},"1.6.0":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.6.0","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@1.6.0","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"15bd21cbfff72933a3cc4a3cabbeafc8c2c59afe","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.6.0.tgz","fileCount":125,"integrity":"sha512-yGGNZcJqoi/bN22zNmcDjaJZNz32qcILkLXzhIW4gkAhdQXiIfqPeSqK7M8IUphpCzA+Zv9QPegHhV5H25GMZQ==","signatures":[{"sig":"MEQCIDaSAeGIjFaNRd3PggRFKIwpgmtqspWASTIL4L1M+mGyAiAv65ksZmfZ2cZMdupfzKjntOP7mg3nHkKQ0jayHWI/Sg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.6.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":840788},"main":"scripts/framework/index.js","engines":{"node":">=20"},"gitHead":"64bda2b0c0d77a4df0768c1aa1ca3742601b0dfd","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","bench":"npm run bench:spawn-cost && npm run bench:throughput","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"ws":"^8.18.0","js-yaml":"^5.2.0","p-limit":"^3.1.0","promptfoo":"0.121.17","smol-toml":"^1.6.1","@openai/codex":"0.142.4","@opencode-ai/sdk":"1.17.13","@openai/codex-sdk":"0.142.5"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.49.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_1.6.0_1785138524254_0.18938589322705002","host":"s3://npm-registry-packages-npm-production"}},"1.7.0":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.7.0","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@1.7.0","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"88002d42ee8859e021326ca83fa2a16be54ff679","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.7.0.tgz","fileCount":125,"integrity":"sha512-HbUfq7I6CzX1y5GlKFsG0vamulmb0veTxiFPRhCNXMvC51Hz2R0nONMo7Lv7mWhWIOxLq7tuIrMKr5IzGmwspQ==","signatures":[{"sig":"MEQCIHAj//hMWtL0FaNlXzZmAsaj972Igoav+SdjzZMVuBWGAiBvYFfa3IfPqwQqZ8k9/L0aubpOJ4l4Pue2lNDMBfFu6A==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.7.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":850934},"main":"scripts/framework/index.js","engines":{"node":">=20"},"gitHead":"6b55f3486b884556f5c2d1adb501baaa9177bc1c","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","bench":"npm run bench:spawn-cost && npm run bench:throughput","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"ws":"^8.18.0","js-yaml":"^5.2.0","p-limit":"^3.1.0","promptfoo":"0.121.17","smol-toml":"^1.6.1","@openai/codex":"0.142.4","@opencode-ai/sdk":"1.17.13","@openai/codex-sdk":"0.142.5"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.49.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_1.7.0_1785201532332_0.42343585750461377","host":"s3://npm-registry-packages-npm-production"}},"1.7.1":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.7.1","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@1.7.1","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"831046abb81ecec749fdaf7f1abf00809d5d03b4","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.7.1.tgz","fileCount":125,"integrity":"sha512-iAA7u2mpQpou3DArSmMTYswuOhU90Hfoya0v1Cix4B7ex3Li7SAmoRM19bZ+yOAguE2pxb+YyJ6oKwIR/N5rrQ==","signatures":[{"sig":"MEYCIQDg1yspJeBcJIR+DIRuma9oJxjob1k3YlVBGXmmRbGawQIhAJdDhWVy4qy9+0id19PdIkdUmeV6YHix8dtYK7Aa36ck","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.7.1","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":860710},"main":"scripts/framework/index.js","engines":{"node":">=20"},"gitHead":"97315fd449a9740be2c5c4c29a59b801c1f26de7","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","bench":"npm run bench:spawn-cost && npm run bench:throughput","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"ws":"^8.18.0","js-yaml":"^5.2.0","p-limit":"^3.1.0","promptfoo":"0.121.17","smol-toml":"^1.6.1","@openai/codex":"0.142.4","@opencode-ai/sdk":"1.17.13","@openai/codex-sdk":"0.142.5"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.49.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_1.7.1_1785317578213_0.6079485815832069","host":"s3://npm-registry-packages-npm-production"}},"1.7.2":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.7.2","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"license":"Elastic-2.0","_id":"@accelerate-data/promptfoo-eval-harness@1.7.2","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"dist":{"shasum":"6343b64cdfd019ae9431f87e3c474750f24e8c8d","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.7.2.tgz","fileCount":125,"integrity":"sha512-vNFhgb7xcrHoY6pozDZRdC9FXzPEjJpoyaWIbGFpFN9cd9PUie3nyTgfB7KAyFLGUSPy5wykWT3j+C6pu4HXRA==","signatures":[{"sig":"MEUCIADLrxjYlu76D3A9tfY3PZCl2R/of5iGbDda/SQc4wJvAiEAtWXik2y5MUMri9D1DTJQSAymDbuiK53Q5ps6x0xCGpA=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.7.2","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":870550},"main":"scripts/framework/index.js","engines":{"node":">=20"},"gitHead":"f511627c21c3cc2277ef66ea583ad16a0bed09cd","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","bench":"npm run bench:spawn-cost && npm run bench:throughput","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js"},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"overrides":{"uuid":"^14.0.0","@anthropic-ai/sdk":">=0.91.1"},"repository":{"url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git","type":"git"},"_npmVersion":"10.8.2","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","directories":{},"_nodeVersion":"20.20.2","dependencies":{"ws":"^8.18.0","js-yaml":"^5.2.0","p-limit":"^3.1.0","promptfoo":"0.121.17","smol-toml":"^1.6.1","@openai/codex":"0.142.4","@opencode-ai/sdk":"1.17.13","@openai/codex-sdk":"0.142.5"},"_hasShrinkwrap":false,"devDependencies":{"markdownlint-cli":"^0.49.0"},"_npmOperationalInternal":{"tmp":"tmp/promptfoo-eval-harness_1.7.2_1785321767698_0.4103987330288641","host":"s3://npm-registry-packages-npm-production"}},"1.8.0":{"name":"@accelerate-data/promptfoo-eval-harness","version":"1.8.0","description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","license":"Elastic-2.0","repository":{"type":"git","url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git"},"engines":{"node":">=20"},"bin":{"ad-evals":"bin/ad-evals.js","eval-harness-init":"bin/eval-harness-init.sh"},"main":"scripts/framework/index.js","scripts":{"test":"node --test scripts/*.test.js scripts/framework/*.test.js scripts/framework/providers/**/*.test.js scripts/framework/providers/**/*.test.mjs","lint:md":"markdownlint '**/*.md' --ignore-path .markdownlintignore","bench:spawn-cost":"node bench/spawn-cost.js","bench:throughput":"node bench/parallel-throughput/driver.js","bench":"npm run bench:spawn-cost && npm run bench:throughput"},"dependencies":{"@opencode-ai/sdk":"1.17.13","@openai/codex":"0.142.4","@openai/codex-sdk":"0.142.5","js-yaml":"^5.2.0","p-limit":"^3.1.0","promptfoo":"0.121.17","smol-toml":"^1.6.1","ws":"^8.18.0"},"devDependencies":{"markdownlint-cli":"^0.49.0"},"overrides":{"@anthropic-ai/sdk":">=0.91.1","uuid":"^14.0.0"},"keywords":["promptfoo","opencode","eval","llm","agent","harness"],"_id":"@accelerate-data/promptfoo-eval-harness@1.8.0","gitHead":"4011b49c74f2775ee154274110434fe9aa7f05a1","bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","_nodeVersion":"20.20.2","_npmVersion":"10.8.2","dist":{"integrity":"sha512-gASxv2ELqxAuKm9CtjuZu3v4RT1AmBpaq73dei5waRFmbh/1mE0UzSrtp6sN2535xhSiA12suDM+ZxqNlvz8oQ==","shasum":"3937ae200b884efe40e675ba9e850343e2051c27","tarball":"https://registry.npmjs.org/@accelerate-data/promptfoo-eval-harness/-/promptfoo-eval-harness-1.8.0.tgz","fileCount":125,"unpackedSize":877719,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@accelerate-data%2fpromptfoo-eval-harness@1.8.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQC977P1kEARXJ2ToQBbp2LKiiPOVQQgsddILLr8v2zmaAIhAKpWY+4VgBkmfGXjRz72fsk7/0I7fTmwZ//PwBNnheyj"}]},"_npmUser":{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},"directories":{},"maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/promptfoo-eval-harness_1.8.0_1786323160846_0.46777231356398197"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-04T09:22:20.424Z","modified":"2026-08-10T00:52:41.506Z","0.1.0":"2026-05-04T09:22:20.658Z","0.1.1":"2026-05-04T09:42:48.275Z","0.1.2":"2026-05-05T12:38:47.193Z","0.1.3":"2026-05-05T12:41:35.348Z","1.5.0":"2026-05-29T08:25:46.647Z","1.6.0":"2026-07-27T07:48:44.427Z","1.7.0":"2026-07-28T01:18:52.473Z","1.7.1":"2026-07-29T09:32:58.381Z","1.7.2":"2026-07-29T10:42:47.860Z","1.8.0":"2026-08-10T00:52:41.092Z"},"bugs":{"url":"https://github.com/accelerate-data/promptfoo-eval-harness/issues"},"license":"Elastic-2.0","homepage":"https://github.com/accelerate-data/promptfoo-eval-harness#readme","keywords":["promptfoo","opencode","eval","llm","agent","harness"],"repository":{"type":"git","url":"git+https://github.com/accelerate-data/promptfoo-eval-harness.git"},"description":"Promptfoo + OpenCode eval harness for agent behavior. Owns model/tier policy, provider wiring, package discovery, state export, and artifact guards. Consumers own eval YAML, prompts, fixtures, and assertions.","maintainers":[{"name":"hbanerjee74","email":"hb@acceleratedata.ai"},{"name":"ntad27","email":"anhduynguyen1706@gmail.com"}],"readme":"# @accelerate-data/promptfoo-eval-harness\n\nShared Promptfoo + OpenCode eval harness. Owns model/tier policy, provider\nwiring, package discovery, Promptfoo state export, and artifact guards.\nConsumers own eval YAML, prompts, fixtures, and assertions.\n\n## Providers\n\nAll providers route through the single Promptfoo bridge\n`file://scripts/framework/_node_bridge.js`. Set `provider_kind` in the\nprovider `config:` block to select a provider.\n\nThe shipped tier config (`config/eval-tiers.toml`) defaults every tier to\n`openhands_sdk`. Provider selection is centralized there: the framework\noverrides any package-local `providers` block with the tier mapping, so\nswitch providers by editing the tier in your own `eval-tiers.toml`, not a\npackage.\n\n| `provider_kind` | Status | Model alias examples | Notes |\n| --- | --- | --- | --- |\n| `opencode_cli` | stable | `opencode-mock`, `opencode-anthropic` | In-process; requires `opencode` binary on `PATH`. |\n| `openhands_sdk` | stable | `mock/openhands-mock`, `openhands/anthropic-claude-3-5-sonnet` | Subprocess via `uv run --with openhands-sdk==1.22.1`; supports multi-turn. |\n| `claude_agent_sdk` | stable | `claude-sonnet-4-6`, `claude-opus-4-7`, `claude-haiku-4-5` (aliases: `opus`, `sonnet`, `haiku`) | Subprocess via `uv run --with claude-agent-sdk==0.2.85`; async lifecycle; multi-turn via stateful `ClaudeSDKClient`; default tools `Read`/`Write`/`Edit`/`Glob`/`Grep` (Bash & web gated via `permissions.allow_shell` / `allow_web`). |\n| `opencode_sdk` | stable | `anthropic/claude-sonnet-4-6`, `openai/gpt-4o` (any model the OpenCode server accepts) | In-proc Node provider via `@opencode-ai/sdk`; boots an ephemeral `127.0.0.1:0` OpenCode server per run, dispatches turns through one `client.session.prompt` per case; agents limited to `{build, plan, general}`; requires Node ≥ 20. |\n| `codex_sdk` | stable | `gpt-4o`, `gpt-4.1` (any model the Codex SDK accepts) | In-proc Node provider via `@openai/codex-sdk` (CJS); reserves a per-session HOME and a per-case `git init`-ed workspace before invoking `Codex.startThread`; multi-turn via a single Thread; defaults `sandbox_mode=workspace-write` and `model_reasoning_effort=medium` (overridable via `extra`); requires Node ≥ 20. |\n\n### Plugin Glue Providers\n\nPlugin glue providers wrap or augment the bridge-routed kinds above with\nruntime metadata, auto-reply gates, plugin discovery, and run telemetry.\nThey are referenced from tier scenarios via the `framework://` URL scheme\nand consume the new optional `[runtime]` fields documented in\n[`docs/setup.md`](docs/setup.md).\n\n| Wrapper | Wraps | Reads (optional `[runtime]` fields) |\n| --- | --- | --- |\n| `framework://codex-sdk-provider.js` | `codex_sdk` (via bridge) | `bootstrap_prompt`, `agent_id`, `agent_entrypoint_file`, `opencode_config`, `model`, `project_dir` |\n| `framework://claude-agent-sdk-provider.js` | `claude_agent_sdk_node` (Node-side, NOT the Python `claude_agent_sdk` bridge) | `bootstrap_prompt`, `auto_reply_text`, `max_auto_replies`, `idle_turn_stop`, `plugin_subdirs`, `agent_id`, `agent_entrypoint_file`, `opencode_config`, `model`, `empty_output_retries` |\n| `framework://opencode-cli-plugin-provider.js` | `opencode_cli` (sibling of `framework://opencode-cli-provider.js`) | `bootstrap_prompt`, `opencode_plugin_link_path`, `capture_on_failure`, `write_run_metadata`, `load_local_env`, `opencode_parser_module`, `opencode_runner_command`, `agent_id`, `agent_entrypoint_file` |\n\nThe base `framework://opencode-cli-provider.js` (the locked §7.4 contract)\nis byte-identical and stays the default for non-plugin scenarios; the\nplugin sibling composes the base without subclassing it. The Claude\nwrapper introduces a **second** transport (`claude_agent_sdk_node`)\nalongside the existing Python-bridge `claude_agent_sdk` kind — see\n[`docs/design.md`](docs/design.md) for when to pick which.\n\n## Scenarios\n\nFramework-owned mock-mode scenarios under\n[`tests/harness-scenarios/packages/`](tests/harness-scenarios/packages/).\nAll run without live API keys and are exercised by the nightly CI\nworkflow on `main`.\n\n| Scenario | `provider_kind` | Description |\n| --- | --- | --- |\n| [`minimal-smoke`](tests/harness-scenarios/packages/minimal-smoke/README.md) | `opencode_cli` | Single-turn smoke test that exercises the bridge end-to-end. |\n| [`opencode-cli-compatibility`](tests/harness-scenarios/packages/opencode-cli-compatibility/README.md) | `opencode_cli` | Layer 4 regression locking all five §7.4 `opencode_cli` behaviors; 3 cases. |\n| [`openhands-mock-multi-turn`](tests/harness-scenarios/packages/openhands-mock-multi-turn/README.md) | `openhands_sdk` | 3-turn conversation via mock SDK; validates NDJSON IPC multi-turn path. |\n| [`claude-mock-multi-turn`](tests/harness-scenarios/packages/claude-mock-multi-turn/README.md) | `claude_agent_sdk` | Multi-turn through the Python `claude_agent_sdk` kind with deterministic mock responses. |\n| [`codex-sdk-mock-multi-turn`](tests/harness-scenarios/packages/codex-sdk-mock-multi-turn/README.md) | `codex_sdk` | Multi-turn through the in-proc `codex_sdk` kind. |\n| [`opencode-sdk-mock-multi-turn`](tests/harness-scenarios/packages/opencode-sdk-mock-multi-turn/README.md) | `opencode_sdk` | Multi-turn through the in-proc `opencode_sdk` kind (ephemeral server). |\n\nSee [Setup Guide → Scenarios](docs/setup.md#scenarios) for the\n`vars.turns` multi-turn pattern and consumer-package authoring rules.\n\n## Quick Start\n\nBootstrap a new repo:\n\n```bash\nnpx --package @accelerate-data/promptfoo-eval-harness eval-harness-init\n```\n\nThis scaffolds `tests/evals/` with `package.json`, `opencode.json`,\n`config/eval-tiers.toml`, and a `harness-smoke` package, installs\ndependencies, and adds a Dependabot entry to `.github/dependabot.yml`\nso the repo receives PRs when a new version is released.\n\nSmoke-test the install immediately (no live API key needed):\n\n```bash\nOPENCODE_MOCK_MODE=1 npx ad-evals run tests/harness-scenarios/packages/minimal-smoke\n```\n\nVerify the full install:\n\n```bash\ncd tests/evals\nnpm test                     # contract tests\nnpm run doctor               # print resolved paths\nnpm run eval:harness-smoke   # one live execution\nnpm run eval:smoke           # smoke across all packages\n```\n\nDependencies are installed automatically on the first `ad-evals` run\nand re-installed whenever `package-lock.json` changes.\n\n### Migration from v0\n\nv0 tier configs (no `provider_kind` field) are accepted at runtime but deprecated.\nRun the in-place migration to avoid a v1.1.0 breaking change:\n\n```bash\nnpx --package @accelerate-data/promptfoo-eval-harness eval-harness-init --upgrade --migrate-from-v0\n```\n\nFull guide: [`docs/migration-v0-to-v1.md`](docs/migration-v0-to-v1.md)\n\n## Usage\n\n```bash\n# Run the smoke filter across all packages\nnpm run eval:smoke\n\n# Run all tests in all packages\nnpm run eval:regression\n\n# Run one package\nad-evals run packages/my-feature/promptfooconfig.json\n\n# Open the Promptfoo UI\nnpm run view\n\n# Print resolved state paths\nnpm run doctor\n```\n\n## Documentation\n\n| Doc | What it covers |\n| --- | --- |\n| [Setup Guide](docs/setup.md) | Bootstrap, verify, write a package, run evals, wire CI; also covers [scenarios](docs/setup.md#scenarios), [provider key matrix](docs/setup.md#provider-key-matrix), [parallelism](docs/setup.md#parallelism), [multi-turn evals](docs/setup.md#multi-turn-conversational-evals), [LLM judge & custom assertions](docs/setup.md#llm-judge--custom-assertions), [workspace isolation & git ops](docs/setup.md#workspace-isolation--git-operations) — give this to a coding agent |\n| [Design](docs/design.md) | Framework architecture and ownership boundary |\n| [Changelog](CHANGELOG.md) | Release notes per version |\n\n## What the Framework Owns\n\n- CLI entrypoint (`ad-evals`)\n- Bootstrap (`eval-harness-init`)\n- Path resolution across worktrees\n- Promptfoo and OpenCode environment export\n- Package discovery rules\n- Provider wiring\n- Resolved config materialization\n- Artifact cleanup guard\n- Default tier → agent mapping\n\n## What Your Repo Owns\n\n- `opencode.json` agent definitions (model, steps, permissions)\n- Package configs under `packages/<name>/`\n- Prompts, fixtures, vars\n- Domain assertions\n- Scenario inventory and per-package documentation\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md).\n","readmeFilename":"README.md"}