![\"MIT](\"https://img.shields.io/github/license/AmrDab/clawdcursor?color=a855f7\")
\n ![\"Latest](\"https://img.shields.io/github/v/release/AmrDab/clawdcursor?color=22c55e&label=release\")
\n ![\"npm\"](\"https://img.shields.io/npm/v/clawdcursor?color=cb3837&logo=npm\")
\n ![\"Node](\"https://img.shields.io/badge/node-%E2%89%A520-339933?logo=nodedotjs&logoColor=white\"){kind=icon}
\n ![\"Cross-platform\"](\"https://img.shields.io/badge/platform-win%20%7C%20mac%20%7C%20linux-0ea5e9\"){kind=icon}
\n ![\"clawdcursor](\"https://glama.ai/mcp/servers/AmrDab/clawdcursor/badges/score.svg\")
\n ![\"Tests\"](\"https://img.shields.io/github/actions/workflow/status/AmrDab/clawdcursor/cross-platform.yml?branch=main&label=tests\")
\n ![\"Discord\"](\"https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white\"){kind=icon}
\n\n Desktop control for any AI agent. Compiles the screen into a UI map and acts on elements by stable id (screenshot/vision only as a last resort),
\n verifies its own actions, and stops the moment you say so. Local · cross-OS · any model.\n
\n \n
\n Quickstart ·\n What's new ·\n Why it's different ·\n The engine ·\n How it works ·\n Tools ·\n Platforms ·\n Changelog\n
\n\n\n\n---\n\n## What's new\n\n**2.0.0** — **A tool, driven by your agent.** (First published as 1.6.0; renumbered because it is a breaking change.) ([full changelog](CHANGELOG.md))\n- **Breaking:** the built-in agent loop and all model settings are gone. Claude, Codex, Cursor, a local Llama or any MCP client drives the six tools (`accessibility`, `system`, `window`, `browser`, `computer`, `batch`); the `task` tool and the `--provider/--model/--api-key` flags are removed.\n- **Breaking:** the policy gate is gone, so `open_file`, closing a window, clipboard writes and clicking \"Send\" by name work. What still protects you: Stop, the pause card, no stray input into AI assistant windows, a short per-OS list of session-ending keys, and your MCP host's own approval prompts. If your host auto-approves tool calls, nothing asks before an irreversible action: see [SECURITY.md](SECURITY.md) (an opt-in confirm mode is coming in 2.1).\n- A thin outline around the window the agent is working in replaces the full-screen glow; the pill sits on that window's monitor.\n- Results you can trust: a value is \"set\" only when the field shows it; `copy_all_text` never presses keys outside a browser; `open_app` never names another app's window; actions by name never press a hidden element.\n- Keys follow the OS: the agent is told its modifier (`cmd` or `ctrl`) up front, and shortcuts are offered only for this OS.\n- Second monitors above or left of the main one, Retina zoom areas, and slow-starting apps now work as expected.\n\n**1.5.27** — **Results you can trust, and logs that say what happened.**\n- Ticks, dropdowns and expanders read and change the same way on Windows, macOS and Linux; `toggle` reports the state it actually reached, and actions by name aim at the control, not its label.\n- A stop always answers \"stopped by the user\" first, and also blocks hover, clipboard writes, page script and tab switches.\n- Typing goes only to the window the agent was working in, dialogs like Save As included.\n- Linux reads deep web pages, and a search that runs out of time or elements says so instead of \"not found\".\n- Logs in `~/.clawdcursor/logs/` show every call's timing and outcome, slow internal steps, why the server exited, and who stopped it, with no typed text, error text or window titles.\n\n**1.5.26** — **The agent asks you on screen, and the pill says what it is doing.**\n- Security (Windows, high): text the agent typed or copied could run as PowerShell code ([GHSA-9ppf-38v8-h97v](https://github.com/AmrDab/clawdcursor/security/advisories/GHSA-9ppf-38v8-h97v)). Fixed; upgrade.\n- The pause prompt is a card: the agent can offer 2-6 choices to click, or a text box, and gets your exact answer back. A choice is never a safety approval.\n- A redesigned pill: what the agent is doing right now (\"Typing in Edge\"), how long it has been running, and a Stop button.\n- A field below the fold is scrolled into view instead of refused (verified on Windows; the macOS and Linux calls are in but not yet run on a real desktop).\n- macOS: when a password field holds secure input, `window focus` names the app holding it instead of failing vaguely.\n\n**1.5.25** — **A stop really stops, and keys only go to the window the agent was working in.**\n- Double-click the pill and every clawdcursor process stops: a running wait ends at once, the rest of a batch never runs, and it stays stopped (even across restarts) until you click Resume.\n- Every key, paste and click first checks that the agent's window is still in front. If you switched apps, nothing is sent and the agent is told so.\n- Safety gate: key combos are blocked however their modifiers are spelled, `shortcuts_execute` and compound actions are gated like the tools they run, and every element an invoke may act on is checked.\n- `--no-vision` now really keeps screenshots from every model; the daemon is loopback-only (use an SSH tunnel for remote), and `server.port` in config works.\n- macOS: the Screen Recording check works without the helper built, and `doctor` asks the same question screenshots do.\n\n**1.5.24** — **The AI never touches your pill, and failures stop looking like results.**\n- While a pause waits for you, nothing reads, clicks or types. The pill and pause prompt are never a target, even from another clawdcursor process.\n- On macOS, a screenshot without Screen Recording permission is an error, not a wallpaper image, and open popovers are readable.\n- The browser clawdcursor drives comes to the front when it navigates.\n- A failed OCR read reports its error instead of \"No text detected\", and OCR `confidence` is only shown when the engine really has one.\n\n**1.5.23** — **Windows: finding an element by name no longer matches stray letters.** A lone \"C\" on another page could match \"User code 0\", and \"Don't Save\" could match **Save**. A partial match now has to be a real part of the name you asked for.\n\n**1.5.22** — **Your turn is as short as possible.** When an agent needs you (a sign-in, a 2FA code), it asks for that one step and does the rest itself. With `done_when` (what shows on screen once you're done), the prompt turns from blue back to amber the moment you finish, with no Resume click.\n\n**1.5.21**\n- **`set_value` never guesses a field.** With no field name or role it used to type into the window's largest text box; it now asks you to name the field.\n- **macOS: no screen-recording permission no longer breaks everything.** Only screenshots and OCR fail, and they say how to fix it; windows, keys and accessibility keep working.\n- **Cleaner page text on Windows** (no placeholder characters), and a security update to a dependency.\n\n**1.5.20**\n- **The on-screen pill no longer blocks the agent's own clicks.** It sits where browsers keep their tabs and address bar; it now steps aside for a moment before every click, drag or scroll, then comes back. Windows, macOS and Linux.\n- **Reading a page no longer highlights it on Linux and macOS either.** `copy_all_text` reads the document through accessibility, as on Windows: exact text, nothing selected, your clipboard untouched.\n- **Your clipboard is put back after typing** (`smart_type` too), and only once the paste has landed, so a slow app can't paste what you had copied.\n- **macOS:** `window focus` focuses the process you asked for, a locked screen says so, sidebar clicks are verified, and a bare modifier key (`shift`) works.\n\n**1.5.19** — **Typing never pastes your own clipboard.** clawdcursor types by pasting; on Windows a clipboard write could fail silently and the paste then sent whatever *you* had copied. Every paste now checks the clipboard first and types key by key when it doesn't hold the text.\n\n**1.5.18**\n- **\"Something else\" on the pause prompt.** When the agent pauses for you, the prompt has a third button between Resume and End task: type an answer instead of doing the step — \"use the test account instead\", a code, a change of plan — and press Enter. The agent continues from what you wrote. Windows, macOS and Linux; over MCP `pause_for_user` returns `{status:\"replied\", reply}`.\n- **Reading a page no longer highlights it (Windows).** `copy_all_text` reads the document the way a screen reader does: exact text, no Ctrl+A, nothing selected, your clipboard untouched.\n- **MCP tool annotations** on every tool, and `window` / `system` now describe every action, its side effects and what it returns.\n\n**1.5.17**\n- **macOS: text outside ASCII is typed and pasted as itself** — \"café 日本語\" no longer comes out as \"cafa aaa\" or \"caf√©\".\n- **`copy_all_text` never selects an editor or a list** (Ctrl+A there selected a whole document or every email).\n\n**1.5.16**\n- **It pauses for you at a roadblock, then carries on.** A sign-in, a 2FA code, a CAPTCHA, an upgrade page: instead of giving up, the agent pauses. The glow around the screen turns **blue** and an on-screen prompt says what it needs (\"Sign in to Outlook, then click Resume\"). Click **Resume** and it continues where it left off, or **End task** to stop. Your own agent can do the same over MCP with `system({ action: \"pause_for_user\", message })`; from a terminal, `clawdcursor resume` (or `--end`).\n- **The glow and banner on every OS.** macOS and Linux now get the same screen-edge glow and banner as Windows: amber while the agent works, blue while it waits for you. Double-click the banner to stop everything.\n- **Blind mode on macOS caught up with Windows.** A native accessibility helper lists windows in 0.3 s (was 11 s) and reads a window's tree in 0.06–0.2 s. Typing is refused while a system dialog owns the keyboard, and the agent is told when another app's dialog took its keys.\n- **Sharper blind mode on Windows and Linux.** Classic controls no longer read as blank panes, the open dialog is what gets read, no 10 s stall on buttons that open a modal, the right field is found by name, and focusing a window no longer un-maximizes it.\n- **Upgrade walls are a roadblock.** The agent is told to stop at `Upgrade`, `Subscribe` or `Start trial` and ask you on the pause prompt.\n- **Know what you're running.** `clawdcursor versions` lists every installed copy (global npm, Claude Desktop extension, Claude Code plugin), and the installers say what they updated.\n\nEverything else is in the [changelog](CHANGELOG.md).\n\n---\n\n## What it is\n\nClawd Cursor is a **local MCP server** that gives any tool-calling agent — Claude Code, Cursor, Windsurf, OpenClaw, the Claude Agent SDK, or your own loop — safe control of the **real desktop**. It clicks, types, reads the screen, opens apps, and drives any GUI the way a human would: native apps, the browser, even a canvas.\n\nMost \"let an agent use the computer\" tools take a screenshot and feed it to a vision model — slow, expensive, and brittle. Clawd Cursor **compiles the screen into one UI map**: it fuses the accessibility tree and OCR into a confidence-scored set of elements, each tagged with a stable `el_NN` id, and acts on elements by id — not pixel coordinates. Coordinates appear only in the last-resort screenshot/vision tier (live pixels off the current frame), for canvas-only apps or tasks that genuinely need spatial reasoning. The result is cheaper, faster, private, and — uniquely — it **checks that each action actually did what it claimed**.\n\n> **If a human can do it on a screen, your agent can too.** No API, no integration, no problem — only the right sequence of reads, clicks, keys, and waits. Use it as the **last-mile fallback**: native API exists? Use it. CLI? Use it. Clawd Cursor is for the click, the legacy app, the GUI with no public surface.\n\n---\n\n## Why it's different\n\nThe desktop-agent space is crowded. The closest **install-and-go** peers are [Windows-MCP](https://github.com/CursorTouch/Windows-MCP) and [Terminator](https://t8r.tech/) (desktop MCP servers); browser-only tools (browser-use, Playwright MCP) are adjacent; and [OmniParser](https://github.com/microsoft/OmniParser) / [UI-TARS](https://github.com/bytedance/UI-TARS-desktop) are vision-centric *parsing approaches* you'd build an agent around, not products you install. Here's the honest comparison across those approaches — what Clawd Cursor does that the popular options don't:\n\n| | **Clawd Cursor** | browser-use | Playwright MCP | OmniParser / UI-TARS | computer-use |\n|------------------------------------------|:-----------------------:|:-----------:|:--------------:|:--------------------:|:------------:|\n| Any desktop app, not just the web | ✅ | web only | web only | ✅ | ✅ |\n| Cross-OS (Windows + macOS + Linux) | ✅ | — | — | varies | sandbox |\n| Perception **without a vision model** | ✅ compiled a11y + OCR map | DOM | a11y tree | ❌ vision-centric | ❌ vision |\n| **Verifies its own actions** (deviation) | ✅ | — | — | — | — |\n| A Stop that holds (every process, across restarts) | ✅ | — | — | — | — |\n| Any model / vendor | ✅ | ✅ | not an agent | model-specific | Claude only |\n| MCP-native (one config, any host) | ✅ | library | test framework | — | tool-use API |\n| Local-only, no cloud required | ✅ | ✅ | ✅ | needs a model | screens → cloud |\n\nThree things here are genuinely rare:\n\n1. **Cheapest-tier-first perception, fully local.** Accessibility tree (free) → OCR (cheap) → screenshot (expensive — the *only* tier that puts pixels in the model's context; \"screenshot\" and \"vision\" are the same step). The agent climbs only when it must, so token cost tracks task difficulty — and with a local model, nothing leaves the machine. Vision-centric agents (OmniParser, UI-TARS) need a screenshot in the model for *every* observation.\n2. **It verifies.** Pass `expect` on a consequential action and Clawd Cursor re-checks the live screen (with a short settle window for async UIs) and reports a **DEVIATION** instead of a hollow \"success.\" A completed task can't be marked done on evidence that was already true before it acted.\n3. **You stay in control, and the roles are clear.** clawdcursor is a tool layer that runs with your own permissions: what to do is decided by your agent and by your MCP host's permission prompts, which clawdcursor feeds with accurate read-only / destructive tool annotations. What clawdcursor guarantees itself is small and holds for every call: a Stop that every process honours, a pause only you can answer, no stray input into the AI assistant's own window, and a short list of key combos (lock, log out) it never sends. See [SECURITY.md](SECURITY.md).\n\nPlus: whenever an agent is driving, an on-screen **banner that says what it is doing** (\"Typing in Edge\") and a thin amber **outline around the window it is working in** (nothing over the window, nothing on the rest of the screen), on Windows, macOS and Linux — **press Stop on the banner to stop.** A stop holds until you give control back: the agent's next request comes up as a prompt, and only your **Resume** lets it act again (or run `clawdcursor resume`). Keys and clicks also go only to the window the agent was working in: if you switch apps while it waits, its next input is refused instead of landing in your window. When the agent needs you (a sign-in, a 2FA code), the outline turns blue and the banner says what to do, with **Resume** and **End task**. A human at the machine always knows, and always has a kill switch.\n\n---\n\n## Install\n\nclawdcursor is an MCP server published to npm — install it into **any** MCP-capable\nagent (Claude Code, Claude Desktop, Cursor, Windsurf, Zed, OpenAI Codex, or your own\nloop) the same way you install any other MCP server.\n\n### Claude Desktop — one-click extension (no terminal)\n\nDownload the extension for your OS and open it — Claude Desktop installs it:\n\n| Windows | macOS | Linux |\n|---|---|---|\n| [clawdcursor-win32.mcpb](https://github.com/AmrDab/clawdcursor/releases/latest/download/clawdcursor-win32.mcpb) | [clawdcursor-darwin.mcpb](https://github.com/AmrDab/clawdcursor/releases/latest/download/clawdcursor-darwin.mcpb) | [clawdcursor-linux.mcpb](https://github.com/AmrDab/clawdcursor/releases/latest/download/clawdcursor-linux.mcpb) |\n\n1. Double-click the file (or **Settings → Extensions → Advanced settings → Install\n Extension…** and pick it).\n2. Turn on **Allow clawdcursor to control this computer** — that is the one-time\n desktop-control consent; until it is on, every tool returns a consent prompt.\n3. **macOS:** the first time macOS asks, allow Claude under **Privacy & Security →\n Accessibility** and **Screen Recording**.\n\nNo Node.js, npm or JSON editing needed — Claude Desktop runs the bundled server\nitself. Everything below is for other agents, or for Claude Desktop without the\nextension.\n\n### 1 — Install the engine + grant consent (once)\n\n```bash\nnpm i -g clawdcursor\nclawdcursor consent --accept # one-time desktop-control consent (required)\nclawdcursor grant # macOS only — approve Accessibility + Screen Recording\n```\n\n> Hosts that support MCP elicitation (e.g. Claude Code, VS Code) also let you skip\n> `consent --accept`: the first tool call asks **\"Allow clawdcursor to control this\n> computer\"** right in the app.\n\n> **Zero-install** also works — swap `clawdcursor` for `npx -y clawdcursor` in any\n> snippet below and npx fetches it on demand. A **global install is recommended**\n> anyway: it's pinnable and inspectable on disk (safer for a tool with full desktop\n> control than auto-fetching `latest` every run), and it's the path on which the macOS\n> native helper builds at install time. Requires **Node.js 20+**.\n\n> **Per-OS prerequisites.** **Windows** installs clean — `sharp` and `@nut-tree-fork/nut-js`\n> ship prebuilt binaries, so no C++/Python build tools are needed. **macOS** needs Xcode\n> Command Line Tools (`xcode-select --install`) for screenshots / vision; core\n> accessibility-driven control still works without them. **Linux** needs a few system\n> packages npm can't install: `tesseract-ocr` (OCR), `python3-gi` + `gir1.2-atspi-2.0`\n> (accessibility tree), and — on Wayland — `ydotool` (synthetic input).\n\n### 2 — Add it to your agent (pick your host)\n\n**One click** (Cursor, VS Code):\n\n[](https://cursor.com/install-mcp?name=clawdcursor&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsImNsYXdkY3Vyc29yIiwibWNwIiwiLS1jb21wYWN0Il19)\n[](https://vscode.dev/redirect/mcp/install?name=clawdcursor&config=%7B%22name%22%3A%22clawdcursor%22%2C%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22clawdcursor%22%2C%22mcp%22%2C%22--compact%22%5D%7D)\n\nEvery host runs the same command — `clawdcursor mcp --compact`, or\n`npx -y clawdcursor mcp --compact` with no global install. Only the config shape\ndiffers between hosts:\n\n**Claude Code**\n```bash\nclaude mcp add clawdcursor -s user -- clawdcursor mcp --compact\n```\n\n**Cursor · Windsurf · Claude Desktop · Cline · Gemini CLI** — the common `mcpServers` shape:\n```json\n{\n \"mcpServers\": {\n \"clawdcursor\": { \"command\": \"clawdcursor\", \"args\": [\"mcp\", \"--compact\"] }\n }\n}\n```\nClaude Desktop reads `%APPDATA%\\Claude\\claude_desktop_config.json` (Windows) or\n`~/Library/Application Support/Claude/claude_desktop_config.json` (macOS).\nGemini CLI reads `~/.gemini/settings.json`.\n\n**VS Code (Copilot)** — `.vscode/mcp.json` uses `servers`, **not** `mcpServers`:\n```json\n{\n \"servers\": {\n \"clawdcursor\": { \"command\": \"clawdcursor\", \"args\": [\"mcp\", \"--compact\"] }\n }\n}\n```\n\n**Zed** — `context_servers` in `settings.json`:\n```json\n{\n \"context_servers\": {\n \"clawdcursor\": { \"command\": \"clawdcursor\", \"args\": [\"mcp\", \"--compact\"] }\n }\n}\n```\n\n**OpenAI Codex** — `~/.codex/config.toml`. The first launch can take a few seconds,\nso raise the 10 s default startup timeout:\n```toml\n[mcp_servers.clawdcursor]\ncommand = \"clawdcursor\"\nargs = [\"mcp\", \"--compact\"]\nstartup_timeout_sec = 30\n```\n\n**opencode** — `opencode.json` uses `mcp`, and `command` is an array:\n```json\n{\n \"mcp\": {\n \"clawdcursor\": { \"type\": \"local\", \"command\": [\"clawdcursor\", \"mcp\", \"--compact\"] }\n }\n}\n```\n\n**Any other MCP host** (Goose, Continue, …) — register a **stdio** server with command\n`clawdcursor` and args `mcp --compact`.\n\n> **Windows:** some hosts start the server without a shell, and then can't find\n> `clawdcursor` or `npx` (both are `.cmd` shims). If the server won't start, wrap it in\n> `cmd /c` — e.g. `\"command\": \"cmd\", \"args\": [\"/c\", \"clawdcursor\", \"mcp\", \"--compact\"]`,\n> or `claude mcp add clawdcursor -s user -- cmd /c clawdcursor mcp --compact`.\n\nThat's the whole setup. Ask your agent: *\"open Outlook and reply to the latest email\nfrom Sarah.\"*\n\n### Or: one-command plugin (Claude Code)\n\nSkip the manual config — this repo ships a plugin that registers the tools **and**\nbundles the usage skill in one step. It resolves the package's `bin` (never a\nhard-coded `dist/` path), so an upgrade can't break it:\n\n```bash\nclaude plugin marketplace add AmrDab/clawdcursor\nclaude plugin install clawdcursor@clawdcursor\n```\n\n\n clawdcursor.com ·\n Discord ·\n Changelog ·\n npm\n
\n","readmeFilename":"README.md"}