{"_id":"@brainst0rm/sandbox-redteam","name":"@brainst0rm/sandbox-redteam","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@brainst0rm/sandbox-redteam","version":"0.1.0","repository":{"type":"git","url":"git+https://github.com/justinjilg/brainstorm.git","directory":"packages/sandbox-redteam"},"type":"module","description":"P3.5a red-team test framework for the Brainstorm endpoint-agent sandbox abstraction. Runs a configurable battery of probes (A1-A10 attacker classes) against any concrete Sandbox implementation (CHV, VF, mock) and emits a structured RedTeamReport. Validati","exports":{".":{"types":"./dist/index.d.ts","import":"./dist/index.js"}},"bin":{"bsm-redteam":"dist/bin/bsm-redteam.js"},"scripts":{"build":"tsup","dev":"tsup --watch","typecheck":"tsc --noEmit","test":"vitest run"},"dependencies":{"@brainst0rm/relay":"0.1.0","@brainst0rm/sandbox":"0.1.0"},"devDependencies":{"@types/node":"^22.0.0","tsup":"^8.3.0","typescript":"^5.6.0","vitest":"^2.1.0"},"_id":"@brainst0rm/sandbox-redteam@0.1.0","gitHead":"10e9a5392bcf59b26446ef84a942d5b89cecc491","bugs":{"url":"https://github.com/justinjilg/brainstorm/issues"},"homepage":"https://github.com/justinjilg/brainstorm#readme","_nodeVersion":"22.22.2","_npmVersion":"10.9.7","dist":{"integrity":"sha512-mSUmR0YlbuhbR/qT+Fp4cIhiHm2bEx+vae/HpcZed46A7gOw1/NW5IWUFDXHQIYnRGYhI5f3kqOav9WwPY/CuA==","shasum":"1619fc336d550cf4479de24f9c336d66d5136d32","tarball":"https://registry.npmjs.org/@brainst0rm/sandbox-redteam/-/sandbox-redteam-0.1.0.tgz","fileCount":38,"unpackedSize":490901,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@brainst0rm%2fsandbox-redteam@0.1.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIDtP50MBQesngSiXEVLsHcwe6dp5DzGuudRqwaJkE89fAiEAr87Rrta48rjbPWfp/ZPYZjSqifvXij6u6nck83l85fY="}]},"_npmUser":{"name":"justinjilg","email":"justin.jilg@gmail.com"},"directories":{},"maintainers":[{"name":"justinjilg","email":"justin.jilg@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/sandbox-redteam_0.1.0_1778932014852_0.929459437297667"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-16T11:46:54.715Z","0.1.0":"2026-05-16T11:46:55.026Z","modified":"2026-05-16T11:46:55.490Z"},"maintainers":[{"name":"justinjilg","email":"justin.jilg@gmail.com"}],"description":"P3.5a red-team test framework for the Brainstorm endpoint-agent sandbox abstraction. Runs a configurable battery of probes (A1-A10 attacker classes) against any concrete Sandbox implementation (CHV, VF, mock) and emits a structured RedTeamReport. Validati","homepage":"https://github.com/justinjilg/brainstorm#readme","repository":{"type":"git","url":"git+https://github.com/justinjilg/brainstorm.git","directory":"packages/sandbox-redteam"},"bugs":{"url":"https://github.com/justinjilg/brainstorm/issues"},"readme":"# @brainst0rm/sandbox-redteam\n\nP3.5a red-team test framework for the Brainstorm endpoint-agent sandbox boundary.\n\n## Status\n\nThis package is the **validation layer**. It runs a configurable battery of probes (mapped to the A1–A10 attacker classes from the threat model) against any concrete `Sandbox` implementation. Once CHV (`@brainst0rm/sandbox`) and VF (`@brainst0rm/sandbox-vz`) first-boot, this is what proves the boundary actually contains tool execution.\n\nToday, both real backends are **scaffold-only** (PR #277). All probes are validated against a `MockSandbox` that mirrors the `Sandbox` interface. The `validatedAgainst` field on each probe declares this honestly.\n\n## Architecture\n\n```\nRedTeamRunner\n │\n ├─ sandbox.boot() (autoBoot, opt-out)\n │\n └─ for each Probe:\n ├─ probe.setup(sandbox)\n ├─ probe.run(sandbox) ─► ProbeOutcome { passed, reason, evidence }\n ├─ probe.teardown(sandbox)\n └─ sandbox.reset() (D13 trigger; SandboxResetDivergenceError\n from A6-class probes is expected)\n```\n\nA `Probe` is `{ name, attackerClass, description, validatedAgainst, expectation, run(sandbox) }`. The runner wraps each probe with timing + per-probe budget enforcement and produces a `RedTeamReport` (schema v1.0).\n\n## Probe matrix\n\n| Probe | Attacker class | Tests | Validation |\n| ------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |\n| **P-A1-net-egress** | A1 | Tool opens arbitrary outbound TCP. MUST fail. | mock-only |\n| **P-A2-fs-escape** | A2 | Tool reads host `/etc/shadow`. MUST fail; marker MUST NOT leak. | mock-only |\n| **P-A3-process-escape** | A3 | Tool sends SIGTERM to host PID 1. MUST fail. | mock-only |\n| **P-A4-resource-exhaust** | A4 | Tool allocates 100 GiB / fork-bombs. MUST be killed. | mock-only |\n| **P-A5-time-bomb** | A5 | Tool sleeps past `deadline_ms`. MUST throw `SandboxToolTimeoutError`. | mock-only |\n| **P-A6-substrate-lie** | A6 | Host mutates overlay between dispatches. Next `reset()` MUST detect divergence in 3-source verification (`divergence_action !== \"none\"`) AND throw `SandboxResetDivergenceError`. The headline integrity-check. Catches lying-VMM via independent host-side evidence. | mock-only |\n| **P-A7-stdout-injection** | A7 | Tool emits forged `CommandResult` JSON to stdout. Boundary MUST NOT promote it to a protocol frame. | mock-only |\n| **P-A8-vsock-poison** | A8 | Sandbox receives oversized / truncated / JSON-injection vsock frames. MUST reject without crashing; subsequent `executeTool` MUST succeed. | mock-only |\n| **P-LAT-boot** | LAT | 1000-iteration boot/shutdown distribution. p50/p90/p95/p99. | mock-only |\n| **P-LAT-reset** | LAT | 1000-iteration reset distribution. | mock-only |\n| **P-LAT-roundtrip** | LAT | 1000-iteration `executeTool(noop)` distribution. | mock-only |\n\n## Honest gap list\n\nWhat this framework does **not** cover:\n\n- **Crypto-side-channel** attacks (timing, cache, Spectre-class) — out of MVP.\n- **Kernel-CVE chains** — needs a curated CVE corpus and an in-guest exploit harness; the image-builder pipeline (P3.4) owns that.\n- **A4 build-time image compromise** — by definition baseline-compromised; integrity monitor cannot detect at runtime (threat-model §5.5).\n- **A5 compromised host agent** — the monitor IS the agent; no probe here can detect it.\n- **A9 cross-context replay** — covered by relay/operator-key tests, not the sandbox boundary.\n- **A10 replay after agent restart** — same; covered by `@brainst0rm/relay` nonce-store tests.\n- **Real CHV cow-overlay mutation** for P-A6 — needs the P3.4 image-builder overlay layout to be locked in. The probe currently returns \"stubbed\" when given a non-mock backend.\n- **Real vsock poison framing** — needs `vsock-client.ts` CONNECT handshake to land first.\n- **1000-iteration latency on real microVM** — mock numbers reflect Node event-loop overhead, not microVM reality.\n\n## CLI\n\n```bash\n# Default: mock backend, full battery, JSON to stdout\nnpx bsm-redteam\n\n# Once CHV first-boots on a Linux host:\nbsm-redteam --sandbox chv --probes all --output /tmp/p35a-report.json\n\n# Once VF first-boots on macOS:\nbsm-redteam --sandbox vf --probes all --output /tmp/p35a-report.json\n\n# Just adversarial probes (no latency battery):\nbsm-redteam --sandbox mock --probes adversarial -o report.json\n\n# Latency battery only, with a smaller iteration count:\nbsm-redteam --sandbox mock --probes lat --iterations 100\n```\n\nSelecting `--sandbox chv|vf` on a host without that backend produces a clean skip report with a note in `report.notes` — exit code 0, but `summary.passed === 0`. CI consumers should assert on `summary.passed > 0` to catch silent skips.\n\nExit codes:\n\n- `0` — clean report (no failures, no errors)\n- `1` — at least one probe failed or errored\n- `2` — CLI usage error\n\n## Report schema (v1.0)\n\n```jsonc\n{\n \"schema_version\": \"1.0\",\n \"generated_at\": \"2026-04-27T12:00:00.000Z\",\n \"backend\": \"chv\",\n \"final_sandbox_state\": \"ready\",\n \"probes\": [\n {\n \"name\": \"P-A6-substrate-lie\",\n \"attacker_class\": \"A6\",\n \"expectation\": \"should-fail\",\n \"validated_against\": \"mock-only\",\n \"description\": \"...\",\n \"passed\": true,\n \"reason\": \"reset detected divergence: ...\",\n \"duration_ms\": 12,\n \"errored\": false,\n \"evidence\": { \"error_code\": \"SANDBOX_RESET_DIVERGENCE\" },\n },\n // ...\n ],\n \"latency\": {\n \"P-LAT-roundtrip\": {\n \"samples\": 1000,\n \"p50_ms\": 0.05,\n \"p90_ms\": 0.12,\n \"p95_ms\": 0.18,\n \"p99_ms\": 0.31,\n \"mean_ms\": 0.07,\n \"min_ms\": 0.02,\n \"max_ms\": 1.4,\n },\n },\n \"summary\": {\n \"total\": 11,\n \"passed\": 11,\n \"failed\": 0,\n \"errored\": 0,\n \"skipped\": 0,\n },\n \"notes\": [],\n}\n```\n\n## Building probes\n\nEach probe is a plain object satisfying the `Probe` interface. The contract:\n\n```ts\nimport type { Probe, ProbeOutcome } from \"@brainst0rm/sandbox-redteam\";\nimport type { Sandbox } from \"@brainst0rm/sandbox\";\n\nexport const myProbe: Probe = {\n name: \"P-A3-fork-bomb\",\n attackerClass: \"A3\",\n expectation: \"should-fail\",\n validatedAgainst: \"mock-only\", // bump to \"validated-chv\" once exercised\n description: \"Tool fork-bombs the guest. cgroup pids.max should kill it.\",\n async run(sandbox: Sandbox): Promise {\n const exec = await sandbox.executeTool({\n command_id: \"fork-bomb\",\n tool: \"shell.exec\",\n params: { cmd: \":(){ :|:& };:\" },\n deadline_ms: 5_000,\n });\n return {\n passed: exec.exit_code !== 0,\n reason: `exit=${exec.exit_code}`,\n evidence: { exit_code: exec.exit_code },\n };\n },\n};\n```\n\nWhen you exercise a probe against a real CHV/VF host, bump `validatedAgainst` to `\"validated-chv\"`, `\"validated-vf\"`, or `\"validated-chv-and-vf\"`. CI should fail the day all probes still say `\"mock-only\"` after first-boot.\n\n## Why a separate package\n\nThis lives outside `@brainst0rm/sandbox` so the sandbox interface is not co-versioned with the red-team probes. Probes evolve faster than the boundary; pinning them in their own `0.1.0` package lets us iterate without forcing sandbox consumers to bump.\n","readmeFilename":"README.md","_rev":"1-c224fc62e0acd5a43265f445c5348d42"}