![\"gui-agent](\"docs/logo.svg\"){kind=logo}
\n\n \n
\n An open-source, WebMCP-powered GUI agent that lives inside your web app.
\n Let a natural-language agent operate your UI — click, fill, navigate, call your app's own actions — driven by any LLM you bring.\n
\n ![\"npm](\"https://img.shields.io/npm/v/%40aralroca%2Fgui-agent\")
![\"npm](\"https://img.shields.io/npm/dm/%40aralroca%2Fgui-agent\")
![\"WebMCP\"](\"https://img.shields.io/badge/WebMCP-standard-6d28d9\"){kind=icon}
![\"TypeScript\"](\"https://img.shields.io/badge/TypeScript-first-3178c6?logo=typescript&logoColor=white\"){kind=icon}
![\"MIT](\"https://img.shields.io/npm/l/%40aralroca%2Fgui-agent\")
\n ![\"Demo:](\"docs/demo.gif\")
\n
The optional visualizer in action: status chips per tool call, an animated gradient glow that tours every element the agent touches, and a backdrop veil that keeps the user's focus on the action.
\n\n---\n\nInspired by [page-agent](https://github.com/alibaba/page-agent), but built on the emerging [**WebMCP**](https://webmachinelearning.github.io/webmcp/) standard via a polyfill.\n\n- 📦 **Just an npm package.** `npm i @aralroca/gui-agent` and `import`. No script injection, no browser extension, no headless browser.\n- 🧩 **Standards-based.** Tools are registered on `document.modelContext` (WebMCP). The polyfill (`@mcp-b/webmcp-polyfill`) installs it where browsers don't natively support it yet.\n- 🎯 **Producer + consumer in one.** Expose your app's actions as precise tools *and* fall back to text-based DOM driving for everything else.\n- 🧠 **Bring your own LLM.** A tiny provider-agnostic interface, plus a ready-made Vercel AI SDK adapter.\n- 🪶 **Headless core.** No UI imposed — wire it to your own chat, command bar, or voice.\n- ✨ **Opt-in visualizer.** Status chips per action, an animated gradient glow around the element being acted on, and a backdrop blur that spotlights it (`@aralroca/gui-agent/ui`).\n\n> ⚠️ Early/experimental. WebMCP is a moving [W3C draft](https://webmachinelearning.github.io/webmcp/); APIs may change.\n\n_Thinking is cheap. But what if your agents didn't have to think so much—and instead had **TOTAL CONTROL** of your app (**AUTOMATICALLY**)? Take a look **[Janux Framework](https://github.com/aralroca/Janux)** for more than gui-agent lib_\n\n\n## Table of Contents\n\n- [Install](#install)\n- [Quick start](#quick-start)\n- [How it works](#how-it-works)\n - [How the agent picks a tool](#how-the-agent-picks-a-tool)\n- [Compared to page-agent](#compared-to-page-agent)\n- [API](#api)\n - [Core](#core-aralrocagui-agent)\n - [Visualizing the agent](#-visualizing-the-agent-aralrocagui-agentui)\n - [Bring your own LLM](#bring-your-own-llm)\n - [AI SDK adapter](#ai-sdk-adapter-aralrocagui-agentai-sdk)\n - [React](#react-aralrocagui-agentreact)\n- [Safety](#safety)\n- [Demo](#demo)\n- [Develop](#develop)\n- [License](#license)\n\n## Install\n\n```bash\nnpm i @aralroca/gui-agent\n# optional peers, depending on what you use:\nnpm i ai zod # for the AI SDK adapter / Zod schemas\n```\n\n## Quick start\n\n```ts\nimport { defineTool, GuiAgent } from \"@aralroca/gui-agent\";\nimport { createAiSdkLlm } from \"@aralroca/gui-agent/ai-sdk\";\n\n// 1. Expose your app's actions as tools (producer side, optional).\ndefineTool({\n name: \"go_to_tab\",\n description: \"Switch the console to a tab.\",\n inputSchema: { type: \"object\", properties: { tab: { type: \"string\" } }, required: [\"tab\"] },\n annotations: { readOnlyHint: true },\n execute: ({ tab }) => router.push(`/${tab}`),\n});\n\n// 2. Run the agent. It discovers your tools + synthesizes DOM tools (click/fill/read).\nconst agent = new GuiAgent({\n llm: createAiSdkLlm({ model: \"anthropic/claude-opus-4-8\" }),\n confirm: async (call) => window.confirm(`Allow \"${call.name}\"?`), // HITL gate\n});\n\nawait agent.run(\"Invite jane@acme.com to the team as admin\");\n```\n\nImporting the package installs the WebMCP polyfill automatically — **no `