# llm-inspector

> LLM Inspector

![Proxy Overview](docs/Proxy.png)

## What is llm-inspector?

**llm-inspector** 是一个 LLM API 透明代理调试工具，能够捕获 AI 编程工具（Claude Code、OpenCode、Cursor、Cody 等）与 LLM 提供商之间的所有 API 请求和响应，在 Web UI 中实时展示：

- 系统提示词 (System Prompts)
- 工具定义 (Tool Definitions)
- 消息内容 (Messages)
- SSE 流式数据块 (Streaming Chunks)
- Token 用量 (Token Usage)

**专为理解 AI 编程工具底层 API 调用行为而设计。帮助开发者看清 LLM API 的"最后一公里"。**

## Key Advantages / 核心优势

| 优势 | 说明 |
|------|------|
| **透明代理、零侵入** | 只需设置环境变量即可拦截所有流量，无需修改 AI 编程工具本身 |
| **结构化 API 可视化** | 交互式 JSON 树查看器，自动解析并渲染 Markdown 文本、思考块、工具调用块 |
| **完整的 SSE 流式解析** | 捕获并重建每个流式事件，原始事件时间线一目了然 |
| **Token 用量实时追踪** | 单请求 + 会话级 + 全局级 Token 统计 |
| **多提供商自动路由** | 支持 Anthropic、OpenAI、DeepSeek、MiniMax、Qwen、ZhipuAI，自动识别格式和认证 |
| **客户端进程追踪** | 自动关联 PID、工作目录、项目文件夹，区分请求来源 |
| **持久化存储 + 导出** | 磁盘日志存储 + ZIP 导出，支持离线分析 |

## Quick Start / 快速开始

### npm 安装

```bash
npm install -g @tonyclaw/llm-inspector
llm-inspector
```

### Docker

```bash
cd docker && docker compose up -d
```

### 源码运行

```bash
bun install
bun run dev
```

打开浏览器访问 http://localhost:25947 查看实时捕获的请求。

### 配置 AI 编程工具

配置你的 AI 编程工具，将 API 请求路由到 llm-inspector 代理：

```bash
# Claude Code
ANTHROPIC_BASE_URL=http://localhost:25947/proxy claude

# OpenCode
LLM_BASE_URL=http://localhost:25947/proxy opencode

# Cursor, Cody 等
ANTHROPIC_BASE_URL=http://localhost:25947/proxy <your-tool>
```

### 验证代理工作正常

1. 启动 llm-inspector 后，打开浏览器访问 http://localhost:25947
2. 配置好环境变量后，向代理发送一个请求
3. 观察 Web UI 中是否实时显示捕获的请求和响应

![Proxy Hello](docs/Proxy%20Hello.png)

---

## MCP Server (Coding Agent Integration)

llm-inspector ships with an embedded [Model Context Protocol](https://modelcontextprotocol.io)
(MCP) server, mounted at `POST http://localhost:25947/api/mcp` on the same port as
the Web UI. Any MCP-aware Coding Agent running on the same machine can
discover and invoke 11 `inspector_*` tools to read captured traffic and
manage provider configuration without a human round-trip.

### Tool catalog (11 tools, all prefixed `inspector_`)

**Read (6) — cheap discovery of what was just sent**

| Tool | Purpose |
|------|---------|
| `inspector_list_logs` | Recent captured logs in reverse-chronological order, with parsed previews of the last user message and the assistant response. Default limit 3, hard-capped at 5. |
| `inspector_get_log` | Full `CapturedLog` for a single id, including `rawRequestBody` and `responseText` with no truncation. Streaming chunks are NOT included. |
| `inspector_get_log_chunks` | SSE streaming chunks for a single log, in order received. |
| `inspector_list_sessions` | All session ids that have been observed in captured logs. |
| `inspector_list_models` | Distinct model names seen across all logs. |
| `inspector_list_providers` | All configured providers, including `apiKey` in plaintext. |

**Action (2)**

| Tool | Purpose |
|------|---------|
| `inspector_get_provider` | Full `ProviderConfig` for a single id, `apiKey` in plaintext. |
| `inspector_replay_log` | Re-send a captured request body to its upstream and return the response summary. |

**Provider write (3) — mutate provider config**

| Tool | Purpose |
|------|---------|
| `inspector_add_provider` | Persist a new provider to `<dataDir>/providers.json`. |
| `inspector_update_provider` | Patch-style update of an existing provider. |
| `inspector_test_provider` | Run the UI's connectivity test against a provider. |

**Explicitly NOT exposed** (kept UI-only by design):

- `delete_provider` — irreversible.
- `clear_log_cache` — only clears the in-memory LRU; misleads agents that expect a full wipe.
- `get_runtime_config` / `set_runtime_config` — runtime policy surface; UI is the source of truth.

### Connecting an MCP client

The server speaks **HTTP Streamable transport** (JSON-RPC over HTTP). The
simplest way to point a client at it is to register the endpoint URL. For
example, in Claude Code's `.mcp.json`:

```json
{
  "mcpServers": {
    "llm-inspector": {
      "type": "http",
      "url": "http://localhost:25947/api/mcp"
    }
  }
}
```

For raw clients that send JSON-RPC directly:

```bash
curl -X POST http://localhost:25947/api/mcp \
  -H 'content-type: application/json' \
  -H 'accept: application/json, text/event-stream' \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-06-18",
      "capabilities": {},
      "clientInfo": { "name": "my-client", "version": "0.0.0" }
    }
  }'
```

### Posture and limitations

- **Localhost only, no auth.** The MCP endpoint inherits inspector's
  existing localhost binding. Any process that can call `/api/mcp` can also
  read `<dataDir>/providers.json` directly. Do not run inspector on a
  host where untrusted local users can reach it.
- **Provider API keys are returned in plaintext** by `inspector_list_providers`
  and `inspector_get_provider`. This is intentional — redacting in MCP but
  not in the underlying JSON file would be performative security.
- **Reflexive loop:** `inspector_list_logs` may include the agent's own
  recent `/proxy` calls. Filter by `clientPid` (your own PID) or by
  timestamp on the client side. The tool description surfaces this warning.
- **List limits are hardcoded** at default 3, max 5. Not user-configurable.
- **Deferred to v1.1** (not in this change): MCP Resources for logs
  (`mcp://llm-inspector/log/{id}`), notifications/subscriptions for live
  log push, filtering out reflexive-loop entries server-side, full-text
  or time-range search, MCP-level auth.

---

## Documentation / 文档目录

- [特性 (Features)](docs/Features.md) - 核心功能介绍
- [安装 (Installation)](docs/Installation.md) - 安装指南
- [使用 (Usage)](docs/Usage.md) - 详细使用说明
- [架构 (Architecture)](docs/Architecture.md) - 系统架构
- [开发 (Development)](docs/Development.md) - 开发指南
- [MCP Server](docs/MCP-Server.md) - 把 llm-inspector 当作 MCP 工具接入 Coding Agent
