{"_id":"@bentolabs/automate-tauri-desktop-app","_rev":"3-5443df7bfee6cabb122d9ece22100d8e","name":"@bentolabs/automate-tauri-desktop-app","dist-tags":{"latest":"0.2.0"},"versions":{"0.2.0":{"name":"@bentolabs/automate-tauri-desktop-app","version":"0.2.0","author":{"name":"Bento"},"_id":"@bentolabs/automate-tauri-desktop-app@0.2.0","maintainers":[{"name":"bentolabs","email":"kaushik@bentolabs.ai"}],"dist":{"shasum":"181758910084d42e57fd4b831e56ccbb837073a6","tarball":"https://registry.npmjs.org/@bentolabs/automate-tauri-desktop-app/-/automate-tauri-desktop-app-0.2.0.tgz","fileCount":5,"integrity":"sha512-010aSgYv1CARXMGgXXv1mR9nkZL21+oKASbqFdHM+RSmWfeg0ElTTW0l8SX0iKi4pXIbhQ1hT7RLnxxQj5heCw==","signatures":[{"sig":"MEYCIQDtKaD3XInnhT+4CYIpx2snnmYqmG7PHYeh1InTM1DzvQIhAOT5nkvH8QzQBQSgwxB937lE4j8kXF0I/3Xa1dyMR0RL","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":200930},"main":"./dist-js/index.cjs","type":"module","types":"./dist-js/index.d.ts","module":"./dist-js/index.js","exports":{"types":"./dist-js/index.d.ts","import":"./dist-js/index.js","require":"./dist-js/index.cjs"},"gitHead":"bab8cd08fd458b23432800a19acb1afbd290fecb","scripts":{"build":"rollup -c","clean":"rm -rf node_modules dist-js target mcp-server-ts/node_modules mcp-server-ts/dist","pretest":"pnpm build","build:all":"pnpm build && pnpm build-plugin && pnpm build:server","build-plugin":"cargo build --release","build:server":"cd mcp-server-ts && pnpm i && pnpm build","prepublishOnly":"pnpm build"},"_npmUser":{"name":"bentolabs","email":"kaushik@bentolabs.ai"},"_npmVersion":"11.7.0","description":"A Tauri v2 plugin and MCP server that let AI agents (Claude Code, Cursor, etc.) interact with your desktop app — screenshots, DOM access, clicking, typing, scrolling, and more.","directories":{},"_nodeVersion":"25.4.0","dependencies":{"@tauri-apps/api":">=2.2.0"},"_hasShrinkwrap":false,"devDependencies":{"tslib":"^2.6.2","rollup":"^4.9.6","typescript":"^5.3.3","@rollup/plugin-typescript":"^11.1.6"},"_npmOperationalInternal":{"tmp":"tmp/automate-tauri-desktop-app_0.2.0_1772551542746_0.38833023297278535","host":"s3://npm-registry-packages-npm-production"}}},"time":{"created":"2026-03-03T15:25:42.657Z","modified":"2026-04-15T18:41:14.826Z","0.2.0":"2026-03-03T15:25:42.907Z"},"author":{"name":"Bento"},"description":"A Tauri v2 plugin and MCP server that let AI agents (Claude Code, Cursor, etc.) interact with your desktop app — screenshots, DOM access, clicking, typing, scrolling, and more.","maintainers":[{"email":"kaushik@bentolabs.ai","name":"bentolabs-ai"},{"email":"manas@bentolabs.ai","name":"manas-bento"},{"email":"ronak@bentolabs.ai","name":"ronak-bento"}],"readme":"# automate-tauri-desktop-app\n\nA Tauri v2 plugin and MCP server that let AI agents (Claude Code, Cursor, etc.) interact with your desktop app — screenshots, DOM access, clicking, typing, scrolling, and more.\n\nForked from [P3GLEG/tauri-plugin-mcp](https://github.com/P3GLEG/tauri-plugin-mcp) with reliability fixes for real-world Claude Code usage.\n\n## What's fixed in this fork\n\n- Smart screenshot capture without stealing window focus (macOS)\n- Fullscreen/maximized window screenshot support\n- Auto-focus before interactive tools (click, type, scroll)\n- Auto-initialization of plugin listeners on import\n- Vite HMR support (listeners re-register on hot reload)\n- Adaptive scroll wait (replaces fixed 350ms timeout)\n- Page map budgets to prevent runaway DOM walks on large pages\n- `querySelectorAll` for scope selectors (returns all matches, not just first)\n- Shebang in MCP server build output for direct CLI execution\n\n## Install\n\n### npm (guest-js bindings)\n```bash\nnpm install @bentolabs/automate-tauri-desktop-app\n```\n\n### MCP Server CLI\n```bash\nnpm install -g @bentolabs/automate-tauri-desktop-app-server\n# or run directly\nnpx @bentolabs/automate-tauri-desktop-app-server\n```\n\n### Rust (Cargo)\n```toml\n[dependencies]\ntauri-plugin-mcp = { git = \"https://github.com/kaushik-bento/tauri-plugin-mcp\" }\n```\n\n## Tools\n\nThe MCP server exposes 10 tools to AI agents:\n\n| Tool | Description |\n|------|-------------|\n| **take_screenshot** | Captures window screenshot. Saves full image to disk with small thumbnail inline (token-efficient). No focus steal on macOS. |\n| **query_page** | Inspect page. Modes: `map` (numbered element refs), `html` (raw DOM), `state` (URL/title/scroll), `find_element` (CSS pixel coords), `app_info` (app metadata). |\n| **click** | Click at x/y or by selector (ref, id, class, tag, text). Auto-resolves element position. |\n| **type_text** | Type into focused element, target by selector, or batch-fill forms via `fields` array. Works with inputs, textareas, contentEditable, React, Lexical, Slate. |\n| **mouse_action** | Hover, scroll (direction/amount/ref/top/bottom), or drag. |\n| **navigate** | `goto` URL, `back`/`forward` with optional delta, `reload`. |\n| **execute_js** | Run arbitrary JS in the webview. Universal escape hatch. |\n| **manage_storage** | localStorage get/set/remove/clear/keys. Cookies get/clear. |\n| **manage_window** | List/focus/minimize/maximize/close windows, zoom, devtools, webview state. |\n| **wait_for** | Wait for text/element to appear or disappear. Use after async actions. |\n\n## Setup\n\n### 1. Register the plugin in your Tauri app\n\nOnly include in development builds:\n\n```rust\n#[cfg(debug_assertions)]\n{\n    builder = builder.plugin(tauri_plugin_mcp::init_with_config(\n        tauri_plugin_mcp::PluginConfig::new(\"YOUR_APP_NAME\".to_string())\n            .start_socket_server(true)\n            // IPC socket (default — recommended)\n            .socket_path(\"/tmp/tauri-mcp.sock\")\n            // Or TCP: .tcp_localhost(4000)\n            // Multi-webview: .default_webview_label(\"preview\".to_string())\n            // Auth for TCP: .auth_token(\"my-secret-token\".to_string())\n    ));\n}\n```\n\n### 2. Import guest-js (auto-initializes)\n\n```ts\nimport '@bentolabs/automate-tauri-desktop-app';\n// Listeners register automatically on import — no setup call needed.\n// HMR is handled automatically if using Vite.\n```\n\n### 3. Configure your AI agent\n\n#### IPC Mode (default, recommended)\n\n```json\n{\n  \"mcpServers\": {\n    \"tauri-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"@bentolabs/automate-tauri-desktop-app-server\"]\n    }\n  }\n}\n```\n\nCustom socket path:\n```json\n{\n  \"mcpServers\": {\n    \"tauri-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"@bentolabs/automate-tauri-desktop-app-server\"],\n      \"env\": {\n        \"TAURI_MCP_IPC_PATH\": \"/custom/path/to/socket\"\n      }\n    }\n  }\n}\n```\n\n#### TCP Mode\n\nFor Docker, remote debugging, or when IPC doesn't work:\n\n```json\n{\n  \"mcpServers\": {\n    \"tauri-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"@bentolabs/automate-tauri-desktop-app-server\"],\n      \"env\": {\n        \"TAURI_MCP_CONNECTION_TYPE\": \"tcp\",\n        \"TAURI_MCP_TCP_HOST\": \"127.0.0.1\",\n        \"TAURI_MCP_TCP_PORT\": \"4000\"\n      }\n    }\n  }\n}\n```\n\n## Building from source\n\n```bash\nnpm install\nnpm run build            # JS guest bindings\ncargo build --release    # Rust plugin\n\n# MCP server\ncd mcp-server-ts\nnpm install && npm run build\n```\n\n## Architecture\n\n```\nAI Agent (Claude Code, Cursor, etc.)\n    ↕ MCP protocol (stdio)\nMCP Server (@bentolabs/automate-tauri-desktop-app-server)\n    ↕ IPC socket or TCP\nTauri Plugin (Rust)\n    ↕ Tauri events with correlation IDs\nGuest JS (webview)\n    ↕ DOM APIs\nYour Application\n```\n\n### Security\n\n- Auth token support for TCP connections (constant-time comparison)\n- Token file written with `0o600` permissions, deleted on shutdown\n- Non-loopback TCP without auth token is rejected\n\n### Platform notes\n\n- **macOS**: Native `NSEvent` injection — no Accessibility permissions needed. Smart window capture without focus steal.\n- **Windows/Linux**: JS-based input fallback (`isTrusted=false`, ~80% coverage)\n- **Screenshots**: macOS uses Core Graphics per-window capture; Windows uses native capture; Linux uses `xcap`\n\n## Troubleshooting\n\n1. **\"Connection refused\"** — Ensure your Tauri app is running and the socket server started. Check both sides use the same connection mode (IPC or TCP).\n\n2. **\"Socket file not found\" (IPC)** — Check socket path exists (`/tmp` on macOS/Linux). Try TCP mode as alternative.\n\n3. **\"Permission denied\"** — On Unix, check file permissions for the socket. TCP mode avoids this.\n\n4. **Testing your setup:**\n   ```bash\n   npx @modelcontextprotocol/inspector npx @bentolabs/automate-tauri-desktop-app-server\n   ```\n\n## License\n\nMIT\n","readmeFilename":"README.md"}