# VisionClaw

A personal assistant agent that runs on your desktop (macOS). It receives command messages from pre-configured channels (Gmail, Telegram, Discord) and executes tasks autonomously using desktop control and browser automation. Results are sent back through the same channel.

> **Platform note**: Desktop automation (mouse, keyboard, screenshots) requires macOS with `cliclick` installed. Browser automation and channel integrations work cross-platform, but full computer-use features are macOS-only.

## Features

- **Autonomous Desktop Agent**: Runs continuously as a long-running process on your computer
- **Gmail Identity**: The agent has its own Gmail account for email and Google Calendar
- **Multi-Channel Support**: Receives commands via Gmail, Telegram, Discord
- **Desktop Control**: Takes screenshots, controls mouse/keyboard, runs terminal commands
- **Browser Automation**: Navigate and interact with web pages via Playwright
- **Google Calendar**: Manages its own schedule for recurring tasks and reminders
- **Fast Responder**: Auto-acknowledges messages while the agent is busy working on a task
- **Self-Improving**: Can add new skills to itself and upgrade to new versions
- **Runtime Observability**: Built-in HTTP obs page for live logs while the agent is running

## Architecture

VisionClaw is built on the [Claude Agent SDK V2](https://platform.claude.com/docs/en/agent-sdk/typescript-v2-preview). It runs as a single-threaded agent with a wake/sleep loop triggered by incoming messages or a periodic heartbeat.

```
src/
  index.ts              # CLI entry point
  logger.ts             # Structured logger
  onboarding/           # Interactive setup wizard (Gmail, OAuth, channels)
  agent/                # Core agent loop, session, context, fast-responder
  tools/                # Custom tools (notify, browser, calendar, screenshot, etc.)
  channels/             # Channel adapters (Gmail, Telegram, Discord)
  email/                # Gmail email tool implementation
  calendar/             # Google Calendar integration
  memory/               # Persistent memory store
  skills/               # Skill installation logic
  config/               # Configuration management
  obs/                  # Runtime observability HTTP server
```

## Prerequisites

- Node.js >= 24.12.0
- An Anthropic API key
- A dedicated Gmail account for the agent
- Google OAuth access, either through the built-in VisionClaw Google app or your own Google Cloud OAuth2 credentials

## Setup

```bash
# Install dependencies
pnpm install

# Build
pnpm run build

# Run (starts onboarding if not configured)
pnpm start

# Or run in development mode
pnpm run dev

# Reconfigure an existing profile (add/remove channels, rotate keys)
visionclaw reconfigure --profile default
```

## Utility scripts

### Upload a file to Volcengine TOS

This repo includes a helper script that uploads a local file to TOS using environment variables for credentials/config.

```bash
export TOS_ACCESS_KEY_ID="..."
export TOS_ACCESS_KEY_SECRET="..."
export TOS_REGION="cn-beijing"
export TOS_BUCKET="your-bucket"
export TOS_ENDPOINT="tos-cn-beijing.volces.com" # optional

npx tsx scripts/upload-to-tos.ts --file ./local.bin --key uploads/local.bin
```

The first run triggers an interactive onboarding wizard that will:
1. Ask for your Anthropic API key
2. Ask for a dedicated Gmail address for the agent
3. Walk through Google OAuth2 authorization (Gmail + Calendar scopes), using the default VisionClaw Google app when available or your own Google Cloud credentials
4. Optionally configure Telegram and Discord

The bundled default Google app, if you want one in your build, is configured in `src/google/default-oauth-app.ts`.

Configuration is stored per profile at `~/.visionclaw/profiles/<profile>/config.json`.

## Observability (HTTP)

When the agent is running, it serves a local observability page showing live logs.

- URL: `http://127.0.0.1:3101/obs`
- SSE stream: `GET /obs/events`
- Snapshot: `GET /obs/snapshot`

This is controlled via **advanced config** (not asked during onboarding):

```json
{
  "obs": {
    "enabled": true,
    "host": "127.0.0.1",
    "port": 3101,
    "bufferSize": 1000
  }
}
```

## Channels

| Channel   | Requirements                    | Status      |
|-----------|----------------------------------|-------------|
| Gmail     | Gmail account (required)         | Always on   |
| Telegram  | Bot token from @BotFather        | Optional    |
| Discord   | Bot token + channel allowlist    | Optional    |

## Custom Tools

| Tool                   | Description |
|------------------------|-------------|
| `wait`                 | Pause execution for a specified duration |
| `notify_user`          | Send a message back through a channel (text + optional attachments) |
| `finish`               | Signal task completion, return to sleep |
| `computer_use_screenshot` | Capture desktop screenshot |
| `browser`              | Open a Chrome instance with CDP for Playwright automation |
| `manage_email`         | List, search, read, send, reply, and manage Gmail messages |
| `manage_calendar`      | Manage Google Calendar events |
| `manage_skills`        | Install, list, create, and delete skills |
| `memory`               | Persistent memory storage across wake cycles |
| `upgrade`              | Check for and install updates |
| `computer_use_click`   | Click on a UI element described in natural language |
| `computer_use_type`    | Type text into the focused field |
| `computer_use_key`     | Press a key or key combination |
| `computer_use_scroll`  | Scroll at a target location |
| `computer_use_drag`    | Drag from one element to another |

## Development

See the **[Development Environment Setup Guide](docs/development-setup.md)** for a comprehensive walkthrough of setting up a local dev environment, including system dependencies, external accounts, environment variables, testing, and macOS permissions.

## License

[Apache License 2.0](LICENSE)
