# Composio integration — live test playbook

This is the runbook for testing the `composio` tool end-to-end against
real Composio infrastructure. Unit tests (`src/tools/__tests__/composioTool.test.js`)
cover the wiring with a mocked SDK; this playbook covers the real thing.

## Prerequisites

1. **Composio account** — sign up at https://composio.dev (free tier is
   20K tool calls/month, sufficient for testing).

2. **Org API key** — dashboard.composio.dev → API Keys → generate.

3. **Install the SDK** — listed in `optionalDependencies` so it's not
   pulled by default:
   ```bash
   npm install @composio/core
   ```

4. **Set the env var** before starting Loxia:
   - PowerShell: `$env:COMPOSIO_API_KEY = "your-key"`
   - cmd: `set COMPOSIO_API_KEY=your-key`
   - bash: `export COMPOSIO_API_KEY=your-key`

5. **Add `composio` to a test agent's capabilities** in your agent
   config. Agents without this capability never see the tool.

## Smoke tests (read-only — no side effects)

Run these first to confirm wiring without sending real messages or
creating real issues.

### 1. List enabled toolkits

```json
{
  "toolId": "composio",
  "actions": [{"type": "list-toolkits"}]
}
```

Expected: `{ success: true, toolkits: [...] }` with at least one entry
if you've enabled any integration in the dashboard.

### 2. Connection status (no-op if not connected)

```json
{
  "toolId": "composio",
  "actions": [{"type": "connection-status", "toolkitSlug": "github"}]
}
```

Expected: `{ connected: false, status: "NOT_CONNECTED" }` initially.

### 3. Connect a test integration (one-time setup, opens browser)

```json
{
  "toolId": "composio",
  "actions": [{"type": "connect", "toolkitSlug": "github"}]
}
```

Expected: `{ connectLink: "https://connect.composio.dev/link/..." }`.
Open the URL in a browser, complete the OAuth flow against a
**dedicated test GitHub account** (not your personal one).

Re-run #2; should now report `connected: true, status: "ACTIVE"`.

### 4. List actions for a connected toolkit

```json
{
  "toolId": "composio",
  "actions": [{"type": "list-tools", "toolkitSlug": "github"}]
}
```

Expected: `{ tools: [...] }` with available action slugs and their schemas.

## End-to-end test (read-only action)

The least-risky live action: fetch a single repo from the connected user.

```json
{
  "toolId": "composio",
  "actions": [{
    "type": "execute",
    "toolkitSlug": "github",
    "actionSlug": "GITHUB_LIST_REPOSITORIES_FOR_AUTHENTICATED_USER",
    "arguments": {"per_page": 1}
  }]
}
```

Expected: `{ success: true, output: [{ id: ..., name: ..., ... }] }`.

If this succeeds, the wiring is good: Loxia → SDK → Composio → upstream
API → back, with per-user OAuth scoping working.

## Test recipes per integration family

Pick one read-only action from each toolkit you plan to use:

| Toolkit | Read-only smoke action |
|---|---|
| GitHub | `GITHUB_LIST_REPOSITORIES_FOR_AUTHENTICATED_USER` |
| Gmail | `GMAIL_FETCH_EMAILS` with `limit: 1` |
| Slack | `SLACK_LIST_CHANNELS` |
| Linear | `LINEAR_LIST_PROJECTS` |
| Notion | `NOTION_SEARCH` with empty query |
| Drive | `GOOGLE_DRIVE_LIST_FILES` with `pageSize: 1` |
| Sheets | `GOOGLE_SHEETS_LIST_SPREADSHEETS` |
| Calendar | `GOOGLE_CALENDAR_LIST_EVENTS` with `maxResults: 1` |

## Failure modes to verify gracefully

| Condition | Expected behavior |
|---|---|
| No `COMPOSIO_API_KEY` env var | Tool returns `{ success: false, error: "COMPOSIO_API_KEY env var is not set..." }` |
| `@composio/core` not installed | Tool returns `{ success: false, error: "@composio/core SDK is not installed..." }` |
| Toolkit not connected for user | `execute` returns upstream error; agent can fall back to `connect` |
| Composio rate limit (429) | SDK handles transient 429s; surfaces as structured `{ success: false, error: "rate limited" }` if it bubbles |
| Upstream API down | Same as rate limit — error surfaced in `result.error` |

## Security checklist

The Composio May 2026 incident leaked ~5K GitHub OAuth tokens. Before
production:

- [ ] API key in env var, NOT committed
- [ ] Key rotation playbook documented (env-var refresh + Loxia restart)
- [ ] Composio dashboard IP allowlist enabled (if available on your tier)
- [ ] Per-user `userId` enforced (Loxia maps agentId → Composio userId)
- [ ] `composio` capability is opt-in per agent — NEVER default-on
- [ ] Audit log of action invocations (Loxia logs all tool calls already)

## Optional: build a bench harness

If you want repeatable A/B testing (e.g. "does Composio integration
change agent task-completion rate?"), the pattern is:

1. Add scenarios to `scripts/composio-bench/scenarios.mjs` (analogous
   to `scripts/memory-bench/scenarios-v2.mjs`)
2. Live probe sends scripted user turns to a real Loxia agent with
   `composio` capability
3. Score by whether the agent called the right Composio actions

Not built yet — request when needed.
