# Key Constraints — Deep Reference

Architecture rules that govern this codebase. Read the matching section before working in that area. For risk-tier and gate-order details, see [agent-security-model.md](agent-security-model.md).

## Process / runtime

- **ESM throughout** — `"type": "module"` in `package.json`. Use `.mjs` for scripts or standard `.js` with ESM syntax.
- **Node ≥ 22.5** required for `node:sqlite` (`DatabaseSync`).
- **Claude Code dry-run only** — `claudeDryRun()` prepares a command string for human review; it never shells out. External actions (email, social posts, deployments) are always approval-gated.
- **Fresh-start by default** — `lib/seeds.js` is mostly a no-op for demo data. The one exception is the canonical **Caretaker** agent (id `ops-agent` — ID retained for back-compat; display name renamed from "Ops Agent" to "Caretaker"). `seedOpsAgentIfMissing` auto-creates it on first launch via `ensureOpsAgent({ silent: true })`. Caretaker runs `daily 08:00`, has a fixed safe toolset (read/search/web). Idempotent — only fires when no `ops-agent` row exists; also performs a one-shot in-place rename of legacy "Ops Agent" rows.
- **Local tools are bounded** — file and terminal/process tools must use `lib/tools/safety.js`, stay inside allowed roots, block secret/system paths, cap/redact output, and write audit rows.
- **Approvals are one-time** — risky local actions create `approval_requests`; approved requests are consumed exactly once by the matching action type.
- **Heartbeat is durable** — cron queues work into SQLite; the heartbeat worker owns execution, leases, retries, approval pauses, and stale recovery.

## Secrets vault

Integration credentials (`GITHUB_TOKEN`, `RESEND_API_KEY`, `SUPABASE_*`, `VERCEL_TOKEN`, etc.) are stored encrypted in the `secrets` SQLite table, not in `.env.local`. Encryption: AES-256-GCM with a 12-byte random IV and 16-byte auth tag per row. Master key is read from `MADARX_MASTER_KEY` env var; on first install with an empty table, a random key is generated and written to `.env.local` with a warning. Set `MADARX_MASTER_KEY` before running in any environment where secrets will be used. `lib/secrets.js` caches decrypted values in-process. During a migration window, `process.env` is read as a fallback if a secret is not in the vault (with a console warning). `.env.local` is for non-credential config only. Migrate existing keys with `node scripts/migrate-env-to-secrets.mjs`.

## Tool allowlists, rate limits, egress

**Per-tool destination allowlists** — `lib/tool-allowlists.js` and the `tool_allowlists` table enforce which destinations a tool may contact. Each tool that writes externally defines a `scope_key` (e.g. `recipient_domain` for email, `repo_full_name` for GitHub, `table` for Supabase writes). Default-deny tools (most write tools) block all calls until at least one allowlist entry is added. Default-allow tools (`web_search`, `web_fetch`) permit all destinations unless scoped. The allowlist gate runs in `executeRegistryTool` after the irreversible/trust-mode gates and before the rate limit and adapter.

**Per-tool rate limits** — `lib/tool-rate-limits.js` and the `tool_rate_limit_events` table enforce sliding-window call quotas per tool. Windows and maximums defined in `MARKETPLACE_TOOLS` per tool (e.g. `email_send`: 20/hour, `gh_pr_merge`: 5/hour). Main-agent and per-agent counters are independent. `recordCall` is only called after successful adapter execution — failed or gated calls do not consume quota. A daily heartbeat GC task removes expired rows.

**Tool egress logging** — every external API call made by any adapter that uses `wrapFetch` from `lib/egress.js` is recorded in the `tool_egress` table. Fields: host, method, status code, bytes in/out, duration, error. No request or response bodies are stored. DB write failures are silent (observation-only). Query example: `SELECT * FROM tool_egress WHERE host = 'api.resend.com' AND at >= datetime('now', '-1 hour')`.

## Marketplace

**Tools across categories** — canonical definitions live in `MARKETPLACE_TOOLS` in `lib/tools/registry.js` (local machine, web/browser, Supabase, Vercel, GitHub, email, travel, voice, etc.). All new integration tools are disabled by default (`enabled_default: false`) and require credentials via the secrets vault when they call paid/external APIs.

**Marketplace foundation (Phase 1)** — `/tools` is now a redirect to `/marketplace?kind=tool`. The unified marketplace at `/marketplace` catalogs three kinds: **tool** (47 API adapters from `tool_installations`), **skill** (4 core packs plus curated Open Design packs), and **bundle** (gstack plus `bundle:open-design`). The domain layer `lib/marketplace.js` is the single write path — `setItemInstalled(id, installed)` dispatches on kind: tool → `setToolEnabled`, skill → `marketplace_installations`, bundle → `{ status: 'deferred' }`. **Agent personas are out of scope.** A future Agent Marketplace product (separate route, separate spec) catalogs personas like Hermes / OpenClaw / gstack-ceo. The brand tokens `--mkt-agent-*` are declared but unused in Phase 1 so the two surfaces stay siblings. See `ai-docs/marketplace-brand.md` and `specs/active/tools-marketplace-foundation.md`. Phase 2 wires the GitHub URL importer; Phase 3 the AI discovery agent; Phase 4 publishing.

**Open Design corpus** — vendored, read-only resources from `nexu-io/open-design` live at `workspace/skills/open-design/` and are refreshed only via `npm run vendor:open-design` after bumping `scripts/vendor-open-design.config.json`. Preserve `NOTICE.md`, nested LICENSE files, and `INDEX.md`; do not edit vendored files by hand. Claude-Code agents read the tree through `--add-dir workspace`; non-CC engines use `lib/agent-open-design-skills.js` to inject a catalog/reference block.

## Trust mode and risk tiers

**Main agent trust mode** — `executeRegistryTool()` in `lib/tools/registry.js` gates risky tool calls (write/exec adapters and any `terminal_run`/`process_start` that `classifyCommand` flags as `approval_required`) when no `agent_id` is set. Default mode is `approval_required`: the call returns `{ status: 'approval_required', approval_request_id }` and creates an `approval_requests` row with `action_type: 'main_agent_tool:<toolName>'` bound by an input signature. The owner approves from the dashboard or `PATCH /api/approvals/[id]`; the next call with `approval_id` consumes the approval and bypasses the adapter's own gate via the internal `owner_trusted` flag. Trust mode (`auto`) skips the registry gate for `high` and below — destructive commands flagged `blocked: true` and `irreversible_external` tools are never auto-approved. Toggle from the dashboard Tools tab (with a confirm step) or via `PATCH /api/settings/trust-mode { mode: 'auto', confirm: true }`. Every flip writes an `audit_logs` row.

**`irreversible_external` risk tier** — a fourth risk level above `high`, implemented in `lib/tools/registry.js` (`isIrreversibleExternal`) and `lib/tools/safety.js` (`classifyCommand`). Tools at this tier (e.g. `gh_pr_merge`, `stripe_refund`, `email_send`, raw destructive SQL) always create an `approval_requests` row with `action_type: 'main_agent_tool_irreversible:<toolName>'` and audit type `main_agent.tool.irreversible_external_approval_required`. **Trust mode `auto` does NOT bypass this gate.** This is the load-bearing guarantee that an LLM on auto-trust cannot make irreversible external writes without a human in the loop. `classifyCommand` also produces `risk: 'irreversible_external'` for `git push --force/-f`, `kubectl delete`, and destructive SQL patterns (DROP TABLE/DATABASE, TRUNCATE, DELETE FROM without WHERE) via SQL CLI tools. Risk tiers in order: `blocked` → refuse always | `irreversible_external` → always gate | `high`/`medium` → gate in approval_required mode only | `low` → always execute.

## Tool adapters

**Resend email** — `email_send` (irreversible_external, 20/hour) and `email_status_lookup` (low risk, 60/min) in `lib/tools/adapters/resend.js`. Credentials: `RESEND_API_KEY` and `RESEND_FROM_EMAIL` via secrets vault. `email_send` requires an explicit `recipient_domain` matching the domain in `to` (two-step validation inside the adapter) and an allowlist entry (`tool_allowlists` table, scope_key=`recipient_domain`) — seeded on first install from `git config user.email`. Both tools use `wrapFetch` for egress logging.

**GitHub (A1)** — 11 tools in `lib/tools/adapters/github.js`. All disabled by default (`enabled_default: false`), credential: `GITHUB_TOKEN` via secrets vault. Read tools (`gh_repo_read`, `gh_repo_list`, `gh_issue_list`, `gh_pr_list`) are `low` risk, rate-limited 60/min, no allowlist. Write tools (`gh_issue_create`, `gh_issue_comment`, `gh_pr_comment`) are `medium` risk, require a `repo_full_name` allowlist entry. High-risk tools (`gh_pr_create`, `gh_actions_dispatch`, `gh_release_create`) require allowlist + trust-mode gate. `gh_pr_merge` is `irreversible_external` — always requires human approval regardless of trust mode. All responses return curated shapes (never raw API blobs). All calls use `wrapFetch` for egress logging. Auth: `Authorization: Bearer <GITHUB_TOKEN>`, `Accept: application/vnd.github+json`, `X-GitHub-Api-Version: 2022-11-28`, `User-Agent: madarx-os`.

**Skill bridge (`skill_bridge_run`)** — Phase 4 of voice/main-agent unification. `lib/tools/adapters/skill-bridge.js` exposes a 3-entry allowlist of first-party gstack CLI skills (`connect-chrome`, `browse`, `qa-only`) as ONE tool so the main agent — and voice, which shares the full catalog — can drive the **visible** GStack browser the operator watches live. Risk `irreversible_external`, `enabled_default:false`, rate-limited 10/5min. It crosses the Claude Code CLI trust boundary, so it is the most sharply gated tool: ALWAYS approval-gated on the main path (even trust-mode `auto`), force-gated on the agent path (added next to `browser_harness_run` in `registry.js`'s `agentIrreversibleGate`), hard kill-switch `MADARX_SKILL_BRIDGE_DISABLED=1`, and only allowlisted skill slugs are runnable (the LLM supplies the skill choice + a plain-language instruction as data, never a shell line). It runs via `runClaudeCodeAgent` under `bypassPermissions` bounded by the load-bearing `.claude/settings.json` deny list. These are first-party skills, NOT agent-authored code, so the "never execute LLM-authored code" invariant holds. Output is capped + redacted. Tested in `tests/skill-bridge.test.js` (allowlist, kill-switch, always-gate on both paths, approved one-shot replay via `MADARX_MOCK_CLAUDE_CODE_RESPONSE`).

**Supabase writes** — six new tools in `lib/tools/adapters/supabase.js`. All disabled by default, all use `wrapFetch` for egress logging, all credentials via secrets vault (`SUPABASE_URL` + `SUPABASE_SERVICE_ROLE_KEY` or `SUPABASE_ANON_KEY`). Table and function names validated against `/^[a-zA-Z_][a-zA-Z0-9_]*(\.[a-zA-Z_][a-zA-Z0-9_]*)?$/` (schema-qualified names like `auth.users` accepted). `supabase_rest_insert` (medium, 30/min, table allowlist) — POST rows; `supabase_rest_upsert` (medium, 30/min, table allowlist) — merge-on-conflict POST; `supabase_rest_update` (high, 30/min, table allowlist) — PATCH with required filters; `supabase_rest_delete` (irreversible_external, 10/hour, table allowlist) — DELETE with required filters, always approval-gated; `supabase_rpc` (high, 30/min, function_name allowlist) — RPC call; `supabase_sql` (irreversible_external, 5/hour, default_allow) — raw SQL via `madarx_exec_sql()` Postgres function, uses service-role key and bypasses RLS, approval payload always includes RLS-bypass warning and full query text. Both `supabase_rest_update` and `supabase_rest_delete` throw synchronously on empty/missing filters before any network call.

**`flight_search` dual-provider routing** — `flight_search` is always available regardless of credential state (`required_env: []`). `lib/tools/adapters/travel.js::flightSearch` is a router: when both `AMADEUS_CLIENT_ID` and `AMADEUS_CLIENT_SECRET` resolve, it calls `_flightSearchAmadeus`; otherwise falls back to `googleFlightsSearch` in `lib/tools/adapters/google-flights.js`, a key-less reverse-engineered call to Google's batchexecute RPC (`https://www.google.com/_/TravelFrontendUi/data/batchexecute`, RPC id `H4cFwe`). Both providers return the same curated shape; callers branch on `source.provider` (`'amadeus' | 'google_flights_web'`) only when they want to disclose origin. `lib/flight-cache.js` is a 15-minute in-process LRU (cap 200) keyed by sha1 of the normalized query — cache hits skip `wrapFetch` and rate-limit accounting and set `source.cache_hit=true`. Set `MADARX_FLIGHT_SEARCH_NO_FALLBACK=1` to disable the Google fallback and surface the Amadeus error instead. Pure helpers `encodeFreq`, `parseBatchexecute`, `curateGoogleOffer` are exported for golden testing; the parser is defensive (strips `)]}'` prefix, balance-scans frames, walks the inner payload heuristically) and degrades gracefully to `{ count: 0, offers: [], source.parse_warning: ... }` if Google's RPC shape shifts.

## Voice tools (Jarvis console)

Provider abstraction lives in `lib/voice/providers.js`. Lanes: `realtime` (primary — OpenAI Realtime via WebRTC; vault keys `OPENAI_API_KEY`, optional `OPENAI_REALTIME_MODEL`, `OPENAI_REALTIME_VOICE`), `tts` (fallback — existing typed chat speaks final reply via OpenAI TTS, ElevenLabs, or the env-driven local HTTP TTS slot; vault keys `OPENAI_API_KEY`/`ELEVENLABS_API_KEY` + optional `ELEVENLABS_VOICE_ID`; local uses `MADARX_LOCAL_TTS_URL`, optional `MADARX_LOCAL_TTS_MODEL`/`MADARX_LOCAL_TTS_VOICE`), and `local` (open-source STT/TTS slots — `MADARX_LOCAL_STT_URL`/`MADARX_LOCAL_TTS_URL`; `scripts/misotts-local-server.py` provides a minimal MisoLabsAI/MisoTTS bridge). Default mode resolves at runtime (`realtime` → `tts` → `local` → `off`) and respects `MADARX_VOICE_DEFAULT_PROVIDER`. Every voice session is minted with a durable Madarx context (`lib/voice/context.js`): today's ISO date, active project, brand voice, open tasks, recent artifact titles, pending approvals, and non-`local_only` agent memory — never API keys, ephemeral tokens, or `local_only` memory. **Voice IS the main agent — transport-only realtime brain (unified path).** The `realtime` lane is, by default, STT/TTS transport only: OpenAI Realtime captures speech and emits the user's final transcript (`mintOpenAIRealtimeSession({ mode: 'transport' })` forces `tools: []`, a transcribe-only instruction block, `audio.input.transcription`, and `turn_detection.create_response:false` so the Realtime model never reasons or speaks). The browser forwards each final utterance to `mainAgent.command(text)` — the SAME `/api/orchestrator/stream` path typed chat uses — so cognition, the live `getTools()` catalog, trust-mode, approval gates (including the never-bypassable `irreversible_external` tier), allowlists, rate limits, egress logging, and audit rows are literally identical to typed chat. The orchestrator's reply is voiced through the TTS lane (`speakText` → `/api/voice/tts`). Because there is no per-session tool snapshot, a tool enabled mid-conversation is usable on the next voice turn. The legacy in-browser-brain path (Realtime reasons + calls `getVoiceToolsForRealtime()` tools + speaks, via `lib/voice/tools.js` → `relayRealtimeToolCall`) still exists for rollback behind `localStorage madarx:voice-brain='realtime'` (`mode:'realtime'` on mint). Voice approvals surface in the chat panel; the voice orb says "needs your approval" and pauses. Realtime ephemeral client secrets are returned to the browser but never persisted; the long-lived OpenAI key never crosses to the browser. UI: `app/components/dashboard/voice-panel.jsx` + `voice-orb.jsx` + `use-realtime-voice.js` (the `transport` flag + `onUserUtterance` callback drive the unified loop). Brand: Onyx surface, Bone/Graphite text, Mars Red only on active speech peaks + approval banners. Operator guide: `docs/voice/jarvis.md`. `cli-printing-press` documented as a deferred external companion in `docs/integrations/cli-printing-press.md`.

## WhatsApp gateway

`scripts/whatsapp-bridge/` is a Baileys-based sub-package with its own `node_modules`. The daemon at `scripts/whatsapp-daemon.mjs` spawns the bridge on `127.0.0.1:3210` and runs the inbound poller from `lib/whatsapp-router.js`. Pair via `npm run whatsapp:bridge` → open `/whatsapp` → scan QR with WhatsApp → Linked Devices. **Pair only a dedicated WhatsApp number**, not your personal account — Baileys is an unofficial Web client and WhatsApp can ban the paired number. The router enforces an allowlist (`whatsapp_allowlist` table) and a 5-msg/minute per-sender rate limit. Inbound WhatsApp messages route to `lib/whatsapp-agent.js::runWhatsAppOrchestrator()` — a Claude tool loop (max 6 turns, 1536 tokens) with a WhatsApp-specific system prompt (plain text only, artifacts land in the dashboard not in the reply). Replies are sent through `POST /send` on the bridge; if the orchestrator takes >5s, an interim "Working on it…" reply goes out automatically. The main agent can also message proactively via the `whatsapp_send` tool. Session lives at `data/whatsapp/session/`; wipe with `npm run whatsapp:reset` to force re-pairing.

## Process / Workflow engine

`lib/process/engine.js` is the **sole writer** of `process_cases.current_stage_id`, `current_stage_slug`, `status`, and `context_json`. Stage handlers in `lib/process/stages/*.js` receive a read-only case snapshot and return an outcome envelope; they never mutate case state. Every transition flows through `transitionTo({ case_id, from_stage_id, next_stage|terminal_status })` which is wrapped in a SQLite transaction and asserts the case is currently in the expected `from_stage_id` — stale transitions throw `PROCESS_TRANSITION_CONFLICT`. Stage execution rows (`process_stage_executions`) are append-only once terminal (`pass`/`fail`/`skipped`/`cancelled`); retries create new rows with `attempt = previous + 1`. Engine-driven stages (`agent_run`, `policy_gate`, `human_approval`, `package`) execute as `process_stage` heartbeat tasks; `human_approval` returns `approval_required` on first run and pauses via the existing heartbeat-queue approval machinery. `lib/approvals.js::registerApprovalDecisionHandler` is the engine's synchronous hook for approve/deny routing. Full reference: [`ai-docs/process-engine.md`](process-engine.md).

## Sub-agent engines

`lib/agent-engines.js` defines five engines: `claude-code` (**default**), `anthropic` (madarx tool marketplace via the Anthropic Messages API — required for `create_artifact`, `generate_ui`, `whatsapp_send`, `competitor_report`, `marketing_plan` which do NOT exist on the CC path), `ollama`, `codex`, `hermes`. `claude-code` agents spawn the local CC CLI per run via `runClaudeCodeAgent` with `--permission-mode bypassPermissions --setting-sources project,local,user --max-turns 30 --add-dir workspace --add-dir data` (matches the pattern OpenClaw's `extensions/anthropic/cli-backend.ts` uses). `bypassPermissions` skips per-call confirmation prompts so multi-step agentic work runs uninterrupted; **the `deny` list in `.claude/settings.json` is the load-bearing safety boundary** — it hard-blocks `sudo`, `rm -rf/-r/-f`, `git reset --hard`, force-push, `git clean -fd`, package-manager installs/publishes, outbound network from shell (`curl`/`wget`/`nc`/`ssh`/`scp`), database CLIs (`psql`/`mysql`/`redis-cli`), filesystem ops (`dd`/`mkfs`/`fdisk`), service control (`systemctl`/`launchctl`), reads of `.env*` / `data/whatsapp/session/` / `~/.ssh` / `~/.aws` / `/etc`, and writes to `.git/`, `node_modules/`, `data/madarx.sqlite`, `.claude/`. The allow list pre-approves read-only tools (Read/Glob/Grep/LS), Web (WebSearch/WebFetch), edits inside `workspace/`, and a curated Bash safelist (git status/log/diff, ls/cat/head/tail/wc, jq, mkdir/touch/cp/mv inside `workspace/`). Extend in `.claude/settings.json` (shared) or `.claude/settings.local.json` (personal, gitignored). Override the spawn timeout via `CLAUDE_CODE_TICK_TIMEOUT_MS` (default 600000) and max turns via `CLAUDE_CODE_MAX_TURNS` (default 30).

**Live verbose streaming (engine-agnostic).** When the main agent answers in chat, madarX surfaces the engine's work in real time — token-by-token text, the model's reasoning ("thinking"), and rich tool detail (full input + output preview + elapsed ms). One knob governs every engine: `MADARX_CC_STREAM_VERBOSITY` = `quiet` | `thinking` (default) | `full` (`lib/engine-verbosity.js`). Mechanics by engine:
- **`claude-code`** — `runClaudeCodeAgent` adds `--include-partial-messages` (capability-probed once via `claude --help`; older CLIs degrade to chunked complete-message streaming) so the stream-json parser (`makeStreamJsonParser`) emits `stream_event` text/thinking deltas. Best-effort thinking via the `MAX_THINKING_TOKENS` subprocess env.
- **`anthropic`** — the orchestrator loop enables extended thinking on thinking-capable models (`modelSupportsThinking` allowlist: sonnet-4/opus-4/haiku-4/3.7-sonnet) with budget `MADARX_THINKING_BUDGET_TOKENS` (default 4000); `thinking`/`redacted_thinking` blocks (with signatures) are preserved in the assistant turn so tool-use follow-ups validate. Thinking is streamed but never persisted to chat history or saved artifacts.
- **`ollama`** — streams NDJSON tokens; reasoning when `MADARX_OLLAMA_THINK=1` (opt-in; `think:true` errors on some models). The fabrication guard still runs on the full output before persistence.
- **SSE events** — `cli_thinking` (accumulated reasoning), `cli_status` (`starting`/`working`+`elapsedMs`/`thinking`/`generating`/`done` cold-start + heartbeat), enriched `tool_result` (`durationMs`, output `preview`/`lines`). Client: `use-stream-command.js` → `ThinkingDisclosure` collapsible panel + per-step duration/preview in `chat-panel.jsx`. Set `MADARX_CC_FORCE_PARTIAL=0/1` to override the CC capability probe.

**Tool-loop capability + no-fabrication guarantee.** Only `anthropic` runs the GOVERNED madarX tool loop (`generateTickOutput`: `chatWithClaude` + `executeRegistryTool` with gates/allowlists/audit). `claude-code`/`codex` run their own native CLI tool runtimes (real, but not the madarX toolset). `ollama`/`hermes` complete text single-shot with **no tool dispatch at all** — a Hermes/Nous-template model "satisfies" a tool task by role-playing the whole loop, emitting `<tool_call>`/`<tool_response>` with **invented results** in plain text. Two layers stop that fiction ever reaching the user as a real result:
- **Layer A (`lib/engine-capabilities.js::assertRunnable`)** — runs in `agent-runner.js` before the engine ternary. A tool-needing agent (≥1 enabled tool) on a non-tool-executing engine (`ollama`/`hermes`) throws `engine_cannot_run_tools` → recorded as a FAILED run with remediation, never a fabricated "complete" deliverable. Tool-less agents run single-shot on any engine.
- **Layer B (`lib/fabricated-tool-output.js::looksFabricatedToolTranscript`)** — backstop after output. When `toolEvents.length === 0` and the output is a fabricated tool transcript, the run throws `fabricated_tool_output` (no artifact written). The detector strips fenced/inline code first so genuine reports *quoting* `<tool_call>` are not false-positives.
- **Orchestrator chat (`app/api/orchestrator/stream/route.js`)** — the claude-code/codex branch detects when the CLI role-plays a sub-agent run (fabricated transcript instead of a `run_agent` `madarx-action` block): the fabricated text no longer counts as "substantive" (so it can't mask itself and block fallback), the retry nudge explicitly forbids simulating tools, and if it persists with no Anthropic fallback the user gets an honest "did not run" error — the fiction is never delivered or auto-artifacted. The `ollama` chat branch has the same detector.
