# RFC: Browser Connectors

> **Status**: Draft — design only, no implementation.
> **Owner**: aion0
> **Date**: 2026-05-16
> **Related**:
> - `PRD.md` (Forge product PRD)
> - `roadmap-multi-agent-workflow.md`
> - `../temper` (memory service used by enterprise agents)

## Problem

Enterprise agents need access to dozens of internal web systems (JIRA,
Mantis, Confluence, Teams, internal CRM, monitoring dashboards). Two
existing approaches both fail:

1. **OAuth / API tokens per system** — IT申请慢、credentials 难管、
   refresh tokens 复杂、有些遗留系统压根没 API。
2. **Anthropic Computer Use 视觉路径** — 截图 + vision LLM 推坐标，慢、
   不稳、token 消耗大、出错难调试。

We propose a **third path**: a Chrome browser extension that runs as the
user's already-authenticated session, executes semantic actions via
per-site **connectors** that extract data from the rendered DOM (using
the extension's Chrome MCP capabilities), and exposes them as MCP tools
to Forge / any MCP client.

**Connectors do not call REST APIs.** They navigate tabs and parse the
rendered HTML, inheriting the user's UI session implicitly through
cookies. Token-based API access is an opt-in escape hatch, not the
default — see "Execution Model" below for the rationale.

## Goals

- **Zero credential setup** — re-use the user's existing browser sessions.
- **Reliable per-site integration** — structured tool calls (not vision).
- **Forge-orchestratable** — agents in Forge pipelines/crafts can call
  browser connectors like any other MCP tool.
- **Composable** — connectors are independent packages, easy to publish
  and version.

## Non-Goals

- Building a general-purpose web automation framework (not Selenium).
- Replacing Computer Use entirely — vision-based fallback is a separate
  Tier 3 below.
- Synchronous chat UI in the browser — that's a separate component
  layered on top (see "Out of Scope" below).

## High-level Architecture

```
┌──────────────────────────────────────────────┐
│ Browser (Chrome / Edge with Chromium)        │
│ ┌────────────────────────────────────────┐   │
│ │ Forge Browser Extension (MV3)          │   │
│ │  • background service worker           │   │
│ │  • content scripts (per-domain)        │   │
│ │  • connectors (built-in + downloaded)  │   │
│ └─────────────────┬──────────────────────┘   │
└───────────────────┼──────────────────────────┘
                    │ WebSocket (loopback, paired)
                    ▼
┌──────────────────────────────────────────────┐
│ Forge installation                           │
│ ┌────────────────────────────────────────┐   │
│ │ browser-bridge-standalone (NEW)        │   │
│ │ port 8407                              │   │
│ │  • WebSocket server → extension        │   │
│ │  • MCP server (exposes connector tools)│   │
│ │  • pairing token store                 │   │
│ │  • health monitor                      │   │
│ └────────────────────────────────────────┘   │
└──────────────────────────────────────────────┘
                    ▲
                    │ MCP / SSE
                    │
      Forge agents / pipelines / crafts
      (and Claude Desktop, Cursor, etc.)
```

### Why a bridge process

Chrome MV3 service workers can't keep long-lived TCP servers or expose
SSE endpoints. The bridge is a small Node process inside Forge that:

1. Holds the persistent MCP server (port 8407)
2. Maintains the WebSocket connection to the extension
3. Routes MCP tool calls into extension actions
4. Tracks connector health + paired devices

The bridge is one of Forge's standalone services (alongside
terminal/workspace/telegram). Stops when Forge stops; auto-detaches
in background mode (same SIGHUP fix already in `forge-server.mjs`).

## Connector Spec

A **connector** is a JavaScript module that:
- Declares which URLs it can handle (`matches`)
- Exposes a set of MCP tools (`tools`)
- Each tool is an async function that runs in the extension's content
  script context

### Manifest

```ts
import { defineBrowserConnector } from '@forge/browser-connector-sdk';

export default defineBrowserConnector({
  id: 'mantis',
  name: 'MantisBT',
  version: '0.1.0',
  matches: ['https://mantis.mycompany.com/*'],
  description: 'Search + create + update Mantis bug tickets',

  // Hint to the orchestrator whether opening a tab is required when
  // none of the user's tabs match `matches`.
  requiresActiveTab: true,

  // Optional: health probe — bridge calls this periodically; failure
  // raises an alert. Should return quickly + use minimal quota.
  healthCheck: async (ctx) => {
    const r = await ctx.fetch('/api/rest/users/me');
    return r.ok;
  },

  tools: {
    'mantis.list_my_bugs': {
      description: 'List bugs assigned to the current user',
      parameters: {
        status: { type: 'string', enum: ['open', 'closed', 'all'], default: 'open' },
        limit: { type: 'number', default: 50 },
      },
      handler: async (args, ctx) => {
        // ctx.fetch attaches cookies automatically (credentials: include)
        const r = await ctx.fetch(
          `/api/rest/issues?handler_id=me&status_id=${args.status === 'open' ? '!=resolved' : ''}&page_size=${args.limit}`,
        );
        const data = await r.json();
        return { bugs: data.issues, total: data.total };
      },
    },
    'mantis.add_comment': {
      description: 'Add a comment to a bug',
      parameters: {
        bug_id: { type: 'integer', required: true },
        text: { type: 'string', required: true },
      },
      // destructive: true → bridge requires user confirmation before exec
      destructive: true,
      handler: async (args, ctx) => {
        const r = await ctx.fetch(`/api/rest/issues/${args.bug_id}/notes`, {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ text: args.text }),
        });
        return { ok: r.ok };
      },
    },
  },
});
```

### Packaging in Forge

The `defineBrowserConnector(...)` module above is the **runtime spec** for a
single connector. It's shipped inside a **Forge plugin** — the same plugin
system used for `gmail`, `jenkins`, etc. Two rules:

**1. Default 1:1 — one connector per plugin.**

Marketplace UX is cleanest when "install Mantis" = one entry. Each plugin
owns its `host_permissions`, version, and auth lifecycle independently.
A connector outage doesn't drag down unrelated plugins.

**2. Escape hatch: 1:N for same-vendor suites with shared auth.**

Atlassian (Jira + Confluence + Bitbucket), Google Workspace (Gmail + Drive +
Calendar), and Microsoft 365 (Teams + Outlook + SharePoint) share OAuth /
SSO across products. Forcing them into separate plugins triples the auth
dance. The plugin manifest therefore allows multiple `connectors:` entries:

```yaml
# plugin.yaml
name: atlassian-suite
category: connector
mode: hybrid
auth: oauth2-atlassian       # shared across all connectors below
connectors:
  - id: jira
    host_permissions: ["*://*.atlassian.net/*"]
    tools: [...]
  - id: confluence
    host_permissions: ["*://*.atlassian.net/wiki/*"]
    tools: [...]
```

This is the **minority case**. All 5 MVP defaults (gmail / teams / jira /
github / gitlab) are 1 plugin = 1 connector.

**3. `category: connector` marks the plugin as a connector in marketplace.**

Forge plugins serve different purposes (pipeline nodes, MCP servers, tools,
connectors). The plugin manifest gets a top-level `category` field so the
marketplace and extension can filter for connectors specifically:

```yaml
category: connector   # | pipeline-node | mcp-server | tool | other
```

Only `category: connector` plugins appear in the extension's "Connectors"
panel. See the plugin-mode ADR in `Implementation-Plan-Browser-Agent.md`
for the full plugin schema changes.

**Three-layer concept (don't conflate):**

| Layer | Unit | Example |
|---|---|---|
| Plugin | Install unit (marketplace entry) | `mantis`, `atlassian-suite` |
| Connector | Per-system adapter | `mantis`, `jira`, `confluence` |
| Capability | A method on a connector | `mantis.list_my_bugs`, `jira.get_issue` |

GitLab exposes issues, MRs, wiki, and CI — that's **one connector with many
capabilities**, not multiple connectors.

### `ctx` API (passed to handlers)

DOM-centric. Handlers navigate, wait, query, and parse. A `fetch` is
still provided for the rare cases where an XHR endpoint of the rendered
app is the cleanest path (still inside the tab's origin, still using
cookies — but **not** for documented REST APIs that demand tokens).

```ts
interface ConnectorContext {
  /** Navigate the active tab to a path within the connector's origin. */
  navigate(path: string, opts?: { waitFor?: string; timeout?: number }): Promise<void>;

  /** Query the DOM. Returns null if not found. */
  $(selector: string): Element | null;
  $$(selector: string): Element[];

  /** Wait for selector with timeout (most pages aren't ready synchronously). */
  waitFor(selector: string, opts?: { timeout?: number }): Promise<Element>;

  /**
   * Same-origin fetch with cookies attached. Use sparingly — only for XHR
   * endpoints the rendered page itself calls (i.e. cookie-authenticated).
   * Do NOT use for documented REST APIs that require Authorization headers;
   * those need a Tier 3 escape hatch (user-provided token).
   */
  fetch(input: string | URL, init?: RequestInit): Promise<Response>;

  /** Take a screenshot of the current viewport (only for Tier 2 vision fallback). */
  screenshot(): Promise<string>;  // base64 PNG

  /** Structured logging visible in the bridge dashboard. */
  log(...args: any[]): void;

  /** Throw this to signal "auth needed" — bridge surfaces a toast + pauses pipeline. */
  AuthExpiredError: typeof Error;
}
```

## Execution Model: DOM Extraction First (Decision 2026-05-16)

**Browser-side connectors extract data from the rendered DOM of the user's
active tab, via the extension's Chrome MCP / scripting capabilities. They
do NOT call site REST APIs.**

This is a deliberate design choice, not a fallback.

### Why DOM, not API?

The whole point of routing through a browser extension is to **reuse the
user's existing logged-in browser session for free**. Many enterprise
systems make this trade-off-laden:

| System | REST API auth | DOM auth |
|---|---|---|
| **MantisBT** | Requires API token (`Authorization` header). Cookies don't work on `/api/rest/*`. | PHPSESSID cookie authenticates rendered pages. ✅ |
| **GitLab** | Requires PAT (cookies don't work on `/api/v4/*` for browsers due to CSRF). | Session cookie authenticates UI pages. ✅ |
| **JIRA Server** | Requires basic auth or PAT. | Session cookie OK on rendered pages. ✅ |
| **Teams / Outlook** | Requires MSAL bearer token. | Session OK on rendered pages. ✅ |

If we used REST APIs, every connector would need the user to mint a token,
copy it into a settings page, rotate it on expiry, and deal with scope-
limited tokens that don't grant the same permissions as their UI session.
That defeats the value proposition of the extension entirely — the user
might as well configure server-side plugins with PATs.

**DOM extraction inherits the user's UI session, no token management,
ever.** The cost is paid in robustness against site redesigns — accepted.

### Execution path

```
LLM in extension → tool call (e.g. mantis.list_my_bugs)
                  │
                  ▼
extension's connector runtime:
  1. find tab matching connector's host_permissions (or open one)
  2. navigate to the appropriate URL inside that tab
  3. use Chrome MCP / scripting / page DOM to extract structured data
  4. return the data to the LLM
```

No `fetch('/api/rest/...')`. No `Authorization` headers. Cookies are
implicit — the tab is already authenticated because the user logged in
through normal browsing.

### Fallback tiers (in declining preference)

| Tier | Mechanism | When |
|---|---|---|
| **1 (primary)** | DOM extraction via Chrome MCP — selectors, structure parsing | Always try first |
| **2 (fallback)** | Screenshot → vision LLM | Site redesigned / selector broken |
| **3 (escape)** | Optional REST API + user-provided token | Only if user explicitly enables it in connector settings |

Tier 3 is **off by default**. We don't ship token-based fallbacks because
the friction of token management is exactly what the extension model
exists to avoid. A connector author MAY surface an optional `api_token`
setting if their site simply cannot be scraped reliably; users who want
to use it accept the trade-off explicitly.

### What this means for connector authors

- Tools execute via DOM, not HTTP. Implementation goes in the extension,
  not in REST clients.
- Tool descriptions in the manifest should describe **observable
  behavior** ("list bugs assigned to me"), not the URL path or API
  surface used to obtain them.
- The connector should declare which page(s) it needs open
  (`host_permissions`) so the extension can route or open tabs accordingly.
- Selector breakage = expected maintenance burden. Bake stable selectors
  into the connector code (e.g., `data-testid`, schema.org microdata)
  and version-pin them.

## Connector Lifecycle

### Discovery
- Built-in connectors ship in the extension package
- Remote connectors fetched from a GitHub registry
  (`forge-browser-connectors` — mirrors the `forge-crafts` model)
- Extension popup → "Marketplace" → install / update / remove

### Activation
- When a tab navigates to a URL matching some connector's `matches`,
  the extension auto-injects that connector's content script
- Bridge is notified, MCP tools are advertised dynamically
- "Connector X became available in tab Y" → Forge sees the new tools

### Health monitoring
- Bridge runs `healthCheck()` on each active connector every 10 min
- Failures show in `Settings → Browser → Connectors` (red badge)
- Optional: notify the user via Forge UI / Telegram bot

### Auth expiry
- Handler throws `AuthExpiredError` on 401 / login-redirect
- Bridge pauses any in-flight pipeline using that tool
- Notifies the user ("Please re-login to Mantis to resume")
- Resumes automatically when the next health check passes

## Connector Examples (DOM Extraction)

### MantisBT
- **Approach**: navigate the active tab to `my_view_page.php` /
  `view_all_bug_page.php` / `view.php?id=<n>` and parse the rendered
  HTML via Chrome MCP. PHP-rendered, structure hasn't changed in years.
- **Why not REST?** `/api/rest/*` requires a Mantis API token; cookies
  don't authenticate against it. Defeats the "logged-in session for free"
  model.
- **Estimated uptime**: 90%+. Self-hosted instances often theme the UI,
  but core selectors (`#buglist`, `.column-id`, `.bug-status`) are stable.

### GitLab (browser)
- **Approach**: navigate to `/-/issues/?assignee_username=me` /
  `/-/merge_requests/` / `/-/profile` pages and parse the rendered DOM.
- **Why not REST?** `/api/v4/*` requires a PAT — same problem as Mantis.
- **Selectors**: GitLab uses `data-testid` and `data-qa-*` attributes
  heavily — relatively stable across versions.

### Microsoft Teams Web
- **Approach**: DOM scrape `[data-tid='message-content']` from rendered
  messages. Microsoft's internal test-id attributes are surprisingly
  stable across releases.
- Avoid extracting MSAL tokens — fragile, key format changes 1-2× per
  year, and surfacing them ourselves is a security smell.

### Confluence / JIRA Server (browser)
- **Approach**: DOM scrape `#main-content` and structured fields. The
  rendered UI carries everything an agent needs (title, body, comments,
  status), and the session cookie keeps working.

### Internal homemade dashboards
- **Approach**: stable selectors with `data-testid` if available, else
  semantic class names. Pin selector versions; expect maintenance when
  the dashboard ships a redesign.

### When DOM extraction genuinely cannot work

Some sites are pure SPAs that render via canvas, virtualize lists
aggressively, or require many sequential interactions to expose data.
For these, the connector author may surface an optional `api_token`
setting and fall back to REST. This is **opt-in per user**, not a
default code path. See "Tier 3 (escape)" in the execution model above.

## Security

### Trust boundaries
- Extension runs with `host_permissions` for each connector's `matches`
- Bridge ↔ extension uses a **per-installation pairing token** (random
  256-bit, generated by Forge, copied into extension on first run)
- Connector code is **not arbitrary** — must come from the marketplace,
  signed by maintainer's GPG (sketched, full design TBD)

### Destructive actions
- Connector spec marks tools as `destructive: true` (writes, deletes,
  sends messages)
- Bridge presents an interactive confirmation to the user before
  executing them — at least the first time per session per tool
- Audit log retains request + response for every destructive call

### Data flow
- All connector responses pass through the bridge → Forge agent context
- **No connector data is ever sent to external LLMs without explicit
  agent invocation** (i.e., the connector doesn't auto-train the agent;
  the agent decides what to do with the result)

### Compliance
- Some sites' ToS prohibit automated access. For internal company
  systems and CRMs, this is a non-issue (you have a license). For
  SaaS like Teams, we operate as "the user" — gray area; document
  expectations in the extension's first-run consent.

## Open Questions

1. **Pairing token rotation** — should it be revocable per-device?
   How is it rotated? Web UI?
2. **Remote Forge** — extension currently assumes loopback bridge.
   Should it support connecting to a Forge behind Cloudflare tunnel
   so a phone can drive desktop's browser? Probably not v1.
3. **Off-screen tab handling** — if a connector needs an active tab
   but the user closed it, do we auto-reopen + wait, or error out?
   Default: error, with optional auto-reopen flag.
4. **Vision tier provider** — Anthropic Computer Use API vs OpenAI
   gpt-4o-vision vs local model? Probably configurable.
5. **Connector marketplace governance** — review process for accepting
   new connectors? Mirrors forge-crafts but stricter (these run inside
   the user's session).
6. **Smith folding** — should `@aion0/smith-engine` (pi-coding-agent
   loop + tempr memory client) be embedded in the extension as the
   default chat backend? Or stay as a separate optional process?
   See "Mutual-enhancement boundaries" in the agent strategy doc.

## Implementation Phases

### Phase 0 — Pairing + bridge skeleton (week 1)
- `lib/browser-bridge-standalone.ts` in Forge, port 8407
- Pairing UI in Forge Settings → Browser
- Minimal MV3 extension shell, popup with "Connect to Forge" button
- WebSocket handshake + ping/pong

### Phase 1 — First connector + MCP wiring (week 2)
- Connector SDK package (`@forge/browser-connector-sdk`)
- Single connector: Mantis (assume internal install)
- MCP tools auto-registered to Forge from bridge
- End-to-end smoke: Forge agent calls `mantis.list_my_bugs` → result

### Phase 2 — Tier 2 + monitoring (week 3)
- DOM scraping fallback in Mantis connector
- Health checks every 10 min, dashboard in Forge
- Auth expiry detection + user notification

### Phase 3 — Multi-connector + marketplace (week 4-5)
- 2-3 more connectors (pick highest-value internal systems)
- `forge-browser-connectors` GitHub registry
- One-click install from extension marketplace
- Versioning + update detection

### Phase 4 — Vision fallback + Computer Use bridge (later)
- Tier 3 vision LLM integration
- Configurable provider
- Reserved for known-bad sites only

### Phase 5 — Smith integration / chat UI (separate RFC)
- Decide whether extension hosts chat UI directly
- If yes: temper-as-memory + pi-coding-agent loop in extension
- Probably one quarter out

## Out of Scope (for this RFC)

- **Chat UI in extension popup/sidepanel** — separate design needed;
  this RFC is purely about connector mechanics.
- **Mobile** — same design works in principle but each browser's
  extension API differs; Phase ∞.
- **Cross-browser** (Safari, Firefox) — eventually yes, but Chrome
  + Edge (Chromium) cover ~95% of enterprise.

## References

- `RFC-…` (placeholder for future chat-UI / agent-loop RFC)
- MCP spec: <https://spec.modelcontextprotocol.io>
- pi-coding-agent: <https://github.com/earendil-works/pi>
- Anthropic Computer Use: <https://docs.anthropic.com/en/docs/build-with-claude/computer-use>
- Chrome MV3 extension docs: <https://developer.chrome.com/docs/extensions/mv3/intro>

## Changelog

- **2026-05-16**: initial draft.
