var e=[{file:`SKILL.md`,content:`---
name: browser-use
description: "Browser automation for AI agents using AI Kit's owned \`browser\` MCP tool. Triggered when: (1) repo-access exhausts its Strategy Ladder and auth requires browser interaction, (2) \`web_fetch\` returns login page HTML, SAML redirect, or CAPTCHA instead of content, (3) user needs to interact with web applications (fill forms, click buttons, extract data), (4) a site requires JavaScript rendering that \`web_fetch\` cannot handle, (5) user asks to browse, scrape, test, or automate a website, or (6) another skill needs a standard recipe format for browser-driven workflows. Uses AI Kit's owned Chromium runtime and recipe patterns for domain-specific automation skills — no external MCP server dependency."
metadata:
  category: cross-cutting
  domain: general
  applicability: on-demand
  inputs: [url, auth-error, browser-task, login-wall]
  outputs: [page-content, screenshots, extracted-data, authenticated-session, network-captures]
  requires: []
  relatedSkills: [repo-access, present, aikit]
argument-hint: "URL or browser task description"
---

# Browser Automation for AI Agents

Use AI Kit's \`browser\` MCP tool for authentication barriers, data extraction, form interactions, network capture, and web automation. Single tool, action-based dispatch, owned Chromium runtime.

## Runtime Preference

- Always use AI Kit's controlled Chromium when the agent needs to inspect, verify, or interact with a page.
- Use \`mode: 'headless'\` by default. Switch to \`mode: 'ui'\` only for user-visible auth or visual debugging.
- Never use system browser commands for agent work. They provide no programmatic feedback.

## Quick Reference

| I need to... | Action | Key params |
|---|---|---|
| Open a page | \`open\` | url, mode (ui/headless) |
| Read page content | \`read\` | readMode: snapshot/dom/markdown/text |
| Click/type/interact | \`act\` | kind: click/type/press/hover/select |
| Wait for something | \`navigate\` | type: waitFor, selector |
| Check network calls | \`network\` | subAction: enable → get |
| Get cookies/storage | \`session\` | sessionAction: cookies/get-storage |
| Take screenshot | \`screenshot\` | fullPage, selector |
| Compare changes | \`diff\` | (compares to previous snapshot) |

For full parameter details: \`describe_tool('browser')\`

## Principles

- **Prefer \`read\` over \`screenshot\`** — snapshots are structured (ARIA tree), searchable, and token-efficient. Screenshots are opaque blobs. Use screenshots only for visual verification.
- **Prefer \`headless\` over \`ui\`** — faster, no window management. Use \`ui\` only when: user needs to see the browser, auth requires manual interaction, or debugging visual issues.
- **Always \`read\` after \`act\`** — actions don't return page state. You need to verify the result.
- **Use \`diff\` instead of re-reading** — after an action, \`diff\` shows only what changed. Much more efficient than full \`read\`.
- **Network capture BEFORE navigation** — enable network capture THEN navigate. Captures start from enable time, not retroactively.
- **One page = one task** — don't reuse pages across unrelated tasks. Fresh pages avoid state contamination.
- **Read snapshot before targeting** — ARIA refs are more stable than guessing CSS selectors or text matches.
- **Use page-context fetch for authenticated APIs** — if the browser session already has cookies and CSRF state, \`fetch\` is usually simpler than exporting cookies.

## NEVER

- **NEVER use \`file:///\` URLs** — the browser blocks local file access for security. Serve locally instead: \`npx -y serve <dir>\` then open \`http://localhost:<port>\`.
- **NEVER interact without reading first** — you need the ARIA tree to know what elements exist. Blind clicks fail.
- **NEVER send passwords via \`act({ kind: 'type' })\`** — tell the user to type credentials manually. Agent should never handle secrets.
- **NEVER use \`screenshot\` as primary information source** — screenshots waste tokens and can't be searched. Use \`read\` with appropriate readMode.
- **NEVER open system browser (\`Start-Process\`, \`open\`, \`xdg-open\`)** — provides zero feedback to the agent. Always use the owned browser.
- **NEVER leave pages open** — close pages when done: \`session({ sessionAction: 'close', pageId })\`. Leaked pages consume resources.
- **NEVER scrape without rate limiting** — rapid page loads trigger bot detection. Add reasonable delays between navigations.
- **NEVER enable network capture after the event you care about** — you can't recover missed requests.

## Activation Signals

- Activate when \`web_fetch\` returns login HTML, SAML redirects, CAPTCHA pages, or JS-heavy shells with no readable content.
- Activate when \`http\` returns 401/403/407 and browser auth is a plausible recovery path.
- Activate when the task requires interaction, screenshots, network capture, authenticated browser-session fetches, or previewing a locally served HTML file.
- Skip it when \`web_fetch\` or \`http\` already gives the answer.
- The \`browser\` tool is always callable directly. This skill exists for recipes and operating discipline, not for basic availability.

## Workflows

**Two modes:**
- **Script Mode** — direct sequential \`browser()\` calls for one-off tasks, debugging, and authenticated API capture.
- **Recipe Mode** — reusable labeled step sequences for domain-specific automation.

### Script Mode (Default — Imperative)

Direct sequential \`browser()\` calls. Best for one-off tasks, testing, API capture.

~~~text
// Open → Read → Act → Read loop
browser({ action: 'open', url: 'https://app.example.com', mode: 'ui' })
browser({ action: 'read', pageId })
browser({ action: 'act', pageId, kind: 'click', ref: '@login-button' })
browser({ action: 'read', pageId })  // verify state changed
~~~

**Network Intelligence pattern:**

~~~text
browser({ action: 'network', pageId, subAction: 'enable', filter: { resourceTypes: ['xhr', 'fetch'] } })
// ... navigate/interact to trigger API calls ...
browser({ action: 'network', pageId, subAction: 'get' })
browser({ action: 'network', pageId, subAction: 'export-har' })
~~~

**Authenticated API calls (using page cookies/session):**

~~~text
browser({ action: 'fetch', pageId, fetchUrl: 'https://app.example.com/api/data', fetchMethod: 'GET' })
~~~

Executes \`fetch()\` in the page, so cookies, session state, and CSRF tokens are reused automatically.

**Console capture:**

~~~text
browser({ action: 'console', pageId, consoleSubAction: 'enable' })
// ... trigger page actions ...
browser({ action: 'console', pageId, consoleSubAction: 'get', level: 'error' })
~~~

### Recipe Mode (Declarative)

Structured step-by-step format for reusable workflows and domain skills. Each step declares Action, Verify, On Failure, and Extract fields.

Load [references/recipes.md](references/recipes.md) for full recipe templates and the recipe format specification.

Brief recipe format:

~~~text
Step N: <description>
  Action: browser({ ... })
  Verify: <condition to check after action>
  On Failure: <recovery strategy>
  Extract: <data to capture for next steps>
~~~

### Element Targeting Priority

1. **\`ref\`** (e.g., \`@F12\`) — From \`read(snapshot)\` ARIA tree. Most reliable.
2. **\`selector\`** (e.g., \`input[name='q']\`) — Playwright CSS/attribute selector. Precise.
3. **\`element\`** (e.g., \`'Submit'\`) — Text matching via \`text=\` locator. **Picks first DOM match regardless of visibility.** Fragile for complex widgets (comboboxes, ARIA roles). Last resort.

**Always \`read(snapshot)\` first** to get refs before interacting.

If a selector times out, assume visibility ambiguity first: narrow the selector, add \`:visible\`, or switch to a snapshot ref.

## Network Intelligence

Use browser-native capture when you need to learn how a web app really talks to its backend:

- **\`network\`** for passive capture of XHR/fetch traffic, timing, and HAR export.
- **\`console\`** for browser-side errors after UI actions.
- **\`fetch\`** for replaying authenticated requests from page context without manually exporting cookies.

Headers are redacted by default. Use sensitive output only when the task explicitly requires it and never echo secrets back to the user.

**Workflow — Reverse-engineer API:**

~~~text
1. open target page
2. network enable (filter: xhr, fetch)
3. interact with the page (click buttons, submit forms)
4. network get → see API endpoints, methods, headers
5. fetch → replay API calls using page session
~~~

## Session Management

- Cookie read/write actions require explicit confirmation.
- Prefer \`fetch\` over cookie export when the goal is an authenticated API call.
- Use labels for long flows, but close the page when the task ends.

## Security Model

**Hard gates — NEVER bypass:**
- Credentials go via terminal input (NEVER through tool params or chat)
- CAPTCHA/MFA: pause and ask user
- Never store tokens in conversation
- Close pages containing sensitive data when done
- Verify page URL before entering credentials (phishing prevention)
- Use \`headless\` mode for automated non-interactive tasks; \`ui\` for user-supervised auth

**Cookie safety gate:** All cookie read/write session actions (\`cookies\`, \`set-cookie\`, \`delete-cookie\`, \`clear-cookies\`) require \`confirm: true\` as an explicit acknowledgment. Without it, the tool returns an error.

## Local File Preview

The browser tool blocks \`file:///\` URLs for security. To preview local HTML files, serve them via a local HTTP server first.

**Pattern:**

~~~text
// 1. Start local server (pick an unused port)
//    Terminal: npx -y serve <directory> -l <port>
//    Example: npx -y serve ./dist -l 3847

// 2. Open in browser
browser({ action: 'open', url: 'http://localhost:3847/my-file.html', mode: 'ui' })

// 3. Read content or take screenshot
browser({ action: 'read', pageId, readMode: 'markdown' })
browser({ action: 'screenshot', pageId, fullPage: true })

// 4. Clean up — kill the server terminal when done
~~~

**Use cases:**
- Preview generated HTML (viewers, reports, docs)
- Visual regression testing of local builds
- Inspect single-file HTML applications
- Screenshot local pages for review

**Important:** Always use \`mode: 'ui'\` for visual preview so the user can also see and interact with the page.

## High-Value Patterns

- **Dialogs:** register \`dialog\` before the action that triggers it. Prompt dialogs also need \`promptText\`.
- **Labels:** assign a \`label\` on open for long-running flows so later calls stay readable.
- **Batch:** use \`batch\` to reduce round-trips, but only when the needed \`pageId\` is already known.
- **Diff:** first call establishes baseline, second call shows the delta.
- **Preview local HTML:** serve the directory first, then open the localhost URL in the owned browser.
`},{file:`references/recipes.md`,content:`# Browser Recipes & Domain Skills

Reference file for reusable browser automation patterns. Load this when building domain-specific browser workflows.

## Recipe Format

Each recipe step declares:
- **Action**: The browser() call to execute
- **Verify**: Condition to check (read page, check element, verify URL)
- **On Failure**: Recovery strategy (retry, alternative selector, escalate)
- **Extract**: Data to capture for subsequent steps

## Recipe Templates

### 1. Form Login

~~~text
Step 1: Open login page
  Action: browser({ action: 'open', url: '<login-url>', mode: 'ui' })
  Verify: Page contains login form (read snapshot, check for username/password fields)
  On Failure: URL may have changed — check redirects
  Extract: pageId

Step 2: Read form structure
  Action: browser({ action: 'read', pageId, readMode: 'snapshot' })
  Verify: Found username field, password field, submit button
  On Failure: Try readMode 'dom' for hidden fields
  Extract: field refs (@username, @password, @submit)

Step 3: Enter credentials
  Action: browser({ action: 'act', pageId, kind: 'type', ref: '@username', text: '<from-user>' })
  Action: browser({ action: 'act', pageId, kind: 'type', ref: '@password', text: '<from-user>' })
  Verify: Fields populated (read snapshot)
  On Failure: Field may need click-to-focus first

Step 4: Submit
  Action: browser({ action: 'act', pageId, kind: 'click', ref: '@submit' })
  Verify: URL changed OR dashboard content appears (read markdown)
  On Failure: Check for error messages, CAPTCHA, MFA prompt
  Extract: authenticated session state

Step 5: Verify authentication
  Action: browser({ action: 'read', pageId, readMode: 'markdown' })
  Verify: Page shows authenticated content (username, dashboard, logout link)
  On Failure: Retry login or escalate to user
~~~

### 2. OAuth/SSO Flow

~~~text
Step 1: Open target application
  Action: browser({ action: 'open', url: '<app-url>', mode: 'ui' })
  Verify: Redirected to SSO provider (check URL domain change)
  Extract: pageId, SSO provider URL

Step 2: Read SSO login page
  Action: browser({ action: 'read', pageId, readMode: 'snapshot' })
  Verify: SSO form present (Okta, Azure AD, etc.)
  On Failure: May need to click "Sign in with SSO" button first
  Extract: form field refs

Step 3: Enter SSO credentials (pause for user)
  Action: PAUSE — ask user to enter credentials in browser window
  Verify: User confirms login complete OR URL returns to app domain
  On Failure: Check for MFA prompt, ask user to complete

Step 4: Verify return to application
  Action: browser({ action: 'read', pageId, readMode: 'markdown' })
  Verify: Back on application domain with authenticated content
  On Failure: Cookie may not have been set — check session cookies
  Extract: session state

Step 5: Export session (optional)
  Action: browser({ action: 'session', pageId, sessionAction: 'cookies', confirm: true })
  Verify: Got authentication cookies
  Extract: cookies for http tool usage
~~~

### 3. Data Extraction

~~~text
Step 1: Navigate to data page
  Action: browser({ action: 'open', url: '<data-url>', mode: 'headless' })
  Verify: Page loaded successfully
  Extract: pageId

Step 2: Wait for dynamic content
  Action: browser({ action: 'navigate', pageId, type: 'waitFor', selector: '<data-container>' })
  Verify: Target element present in DOM
  On Failure: Increase timeout, check if JS is needed

Step 3: Extract structured data
  Action: browser({ action: 'eval', pageId, code: 'document.querySelectorAll("<selector>").map(...)' })
  Verify: Got expected data structure
  On Failure: Try different selector, use read(dom) to inspect structure
  Extract: structured data

Step 4: Paginate (if needed)
  Action: browser({ action: 'act', pageId, kind: 'click', ref: '@next-page' })
  Verify: New content loaded (different from previous page)
  On Failure: End of pagination reached
  Extract: additional data, loop back to Step 3
~~~

### 4. File Upload

~~~text
Step 1: Navigate to upload page
  Action: browser({ action: 'open', url: '<upload-url>', mode: 'ui' })
  Verify: Upload form present
  Extract: pageId

Step 2: Locate upload input
  Action: browser({ action: 'read', pageId, readMode: 'snapshot' })
  Verify: Found file input element
  Extract: file input ref

Step 3: Upload file
  Action: browser({ action: 'act', pageId, kind: 'upload', ref: '@file-input', value: '<local-file-path>' })
  Verify: File name appears in UI
  On Failure: Input may be hidden — try selector instead of ref

Step 4: Submit upload
  Action: browser({ action: 'act', pageId, kind: 'click', ref: '@upload-button' })
  Verify: Success message or progress indicator
  On Failure: Check file size limits, format restrictions
  Extract: upload result (URL, ID, etc.)
~~~

### 5. API Reverse-Engineering (Network Intelligence)

~~~text
Step 1: Open application and enable capture
  Action: browser({ action: 'open', url: '<app-url>', mode: 'ui' })
  Action: browser({ action: 'network', pageId, subAction: 'enable', filter: { resourceTypes: ['xhr', 'fetch'] } })
  Verify: Network capture enabled
  Extract: pageId

Step 2: Trigger target API calls
  Action: browser({ action: 'act', pageId, kind: 'click', ref: '@action-button' })
  Verify: Page state changed (new data loaded)
  On Failure: Try different interaction to trigger API

Step 3: Review captured traffic
  Action: browser({ action: 'network', pageId, subAction: 'get' })
  Verify: Found target API endpoints in captured requests
  Extract: API URL, method, headers, response structure

Step 4: Replay with fetch (using page session)
  Action: browser({ action: 'fetch', pageId, fetchUrl: '<captured-api-url>', fetchMethod: 'GET' })
  Verify: Got same response as captured
  Extract: API response data

Step 5: Export HAR (optional)
  Action: browser({ action: 'network', pageId, subAction: 'export-har' })
  Extract: HAR file for documentation or replay
~~~

### 6. Monitoring / Health Check

~~~text
Step 1: Open target page
  Action: browser({ action: 'open', url: '<url>', mode: 'headless', waitUntil: 'networkidle' })
  Verify: Page loaded without errors
  Extract: pageId

Step 2: Check for error indicators
  Action: browser({ action: 'console', pageId, consoleSubAction: 'enable' })
  Action: browser({ action: 'read', pageId, readMode: 'markdown' })
  Verify: No error banners, expected content present
  Extract: page health status

Step 3: Check console for JS errors
  Action: browser({ action: 'console', pageId, consoleSubAction: 'get', level: 'error' })
  Verify: No critical JS errors
  Extract: error list (if any)

Step 4: Screenshot for visual regression
  Action: browser({ action: 'screenshot', pageId, fullPage: true })
  Extract: screenshot for comparison
~~~

### 7. Local File Preview

~~~text
Step 1: Start local HTTP server
  Action: Run in terminal (async): npx -y serve <directory-containing-file> -l <port>
  Verify: Server output shows "Accepting connections at http://localhost:<port>"
  On Failure: Port in use — try different port
  Extract: server URL, terminal ID

Step 2: Open file in browser
  Action: browser({ action: 'open', url: 'http://localhost:<port>/<filename>.html', mode: 'ui' })
  Verify: Page loaded (read markdown shows expected content, not 404)
  On Failure: Check filename spelling, verify server is running
  Extract: pageId

Step 3: Inspect content
  Action: browser({ action: 'read', pageId, readMode: 'markdown' })
  Verify: Got meaningful page content
  On Failure: Try readMode 'snapshot' for interactive elements, 'dom' for structure
  Extract: page content

Step 4: Screenshot (optional)
  Action: browser({ action: 'screenshot', pageId, fullPage: true })
  Verify: Screenshot captured
  Extract: visual snapshot

Step 5: Cleanup
  Action: Kill the server terminal by its terminal ID
  Action: browser({ action: 'session', pageId, sessionAction: 'close' })
  Verify: Terminal stopped, page closed
~~~

## Creating Domain Skills

Domain skills use browser-use recipes as building blocks for specific automation tasks (e.g., "jira-automation", "salesforce-extract").

### Structure

A domain skill should:
1. Import the browser-use skill as a dependency
2. Define domain-specific recipes using the recipe format above
3. Include domain knowledge (selectors, URLs, auth patterns for that service)
4. Provide error recovery specific to that domain

### Quality Checklist

- [ ] Every recipe step has Action + Verify + On Failure
- [ ] Credentials handled via user input (never hardcoded)
- [ ] Selectors are resilient (prefer aria labels, data-testid, role over CSS classes)
- [ ] Timeouts configured for slow pages
- [ ] Screenshots taken at key verification points
- [ ] Session cleanup (close page when done)
- [ ] Works in both headless and ui modes

### Composition

Recipes compose by chaining — output of one recipe feeds into another:

~~~text
Login Recipe → (authenticated pageId) → Data Extraction Recipe → (data) → Present Results
~~~

For complex flows:

~~~text
Login Recipe → Network Enable → Interaction Recipe → Network Get → Fetch API → Process Data
~~~
`},{file:`references/auth-patterns.md`,content:`# Authentication Patterns

Browser-based authentication strategies for different auth mechanisms.

## Pattern 1: Basic Form Login

**When:** Simple username/password form without SSO.

**Steps:**
1. \`open\` login page
2. \`read\` snapshot to find form fields
3. \`act\` type credentials (ask user for input via terminal)
4. \`act\` click submit
5. \`read\` to verify authenticated state

**Verification:** URL changed to dashboard, or page contains user-specific content.

**Fallback:** Check for error messages, CAPTCHA, or MFA prompts.

## Pattern 2: OAuth2 / SSO (Okta, Azure AD, Google)

**When:** Application redirects to external identity provider.

**Steps:**
1. \`open\` application URL — observe redirect to IdP
2. \`read\` SSO page snapshot
3. PAUSE — ask user to complete SSO login in the browser window
4. Detect redirect back to application (URL change)
5. \`read\` to verify authenticated

**Verification:** URL domain returned to application, authenticated content visible.

**Fallback:** User may need to complete MFA or consent screens manually.

## Pattern 3: Cookie/Session Transfer

**When:** Need to use captured session in \`http\` tool for API calls.

**Steps:**
1. Complete authentication via Pattern 1 or 2
2. \`session({ sessionAction: 'cookies', confirm: true })\` — export all cookies
3. Use cookies in \`http\` tool: \`http({ url, headers: { Cookie: '<exported>' } })\`

**Alternative (Recommended):** Use \`browser({ action: 'fetch', pageId, fetchUrl: '<api-endpoint>' })\` instead of extracting cookies. The \`fetch\` action executes HTTP requests directly in the page context, automatically using the page's cookies, session, and CSRF tokens. No cookie extraction or manual header management needed.

**Verification:** API response returns authenticated data.

## Pattern 4: Client Certificate / mTLS

**When:** Site requires client certificate authentication.

**Steps:**
1. Certificate must be installed in the system cert store
2. \`open\` with \`mode: 'ui'\` — browser will prompt for cert selection
3. PAUSE — user selects certificate
4. Verify page loads authenticated content

**Note:** Headless mode may not support cert picker — use \`ui\` mode.

## Pattern 5: Multi-Factor Authentication (MFA)

**When:** Login requires second factor (TOTP, SMS, push notification).

**Steps:**
1. Complete username/password entry
2. \`read\` snapshot — detect MFA prompt type
3. PAUSE — ask user to complete MFA (enter code, approve push, etc.)
4. \`read\` to verify MFA completed and authenticated

**Hard Rule:** NEVER attempt to automate MFA — always defer to user.

## Pattern 6: API Key Discovery

**When:** Need to find API keys or tokens from an authenticated web session.

**Steps:**
1. Authenticate using Pattern 1 or 2
2. \`network({ subAction: 'enable', filter: { resourceTypes: ['xhr', 'fetch'] } })\`
3. Navigate to pages that trigger API calls
4. \`network({ subAction: 'get' })\` — inspect Authorization headers
5. Extract bearer tokens, API keys from captured request headers

**Important:** Pass \`showSensitive: true\` to see full Authorization headers (redacted by default).

**Alternative:** Use \`eval\` to read tokens from localStorage/sessionStorage:

~~~text
browser({ action: 'eval', pageId, code: 'localStorage.getItem("authToken")' })
~~~

## Pattern 7: Session Persistence Across Tasks

**When:** Need to maintain authenticated state for multiple operations.

**Steps:**
1. Authenticate once (any pattern above)
2. Keep the page open (don't close)
3. Use \`fetch\` action for subsequent API calls — inherits page session
4. Or use \`navigate\` to move between pages — cookies persist

**Important:** The browser auto-closes after idle timeout. For long-running tasks, interact periodically to reset the idle timer.
`}];export{e as default};