{"_id":"@arpagon/pi-web-providers","name":"@arpagon/pi-web-providers","dist-tags":{"latest":"1.2.0"},"versions":{"1.2.0":{"name":"@arpagon/pi-web-providers","version":"1.2.0","description":"Configurable web access extension for pi with per-tool provider routing for search, contents, answers, and research across Claude, Cloudflare, Codex, Custom CLI, Exa, Gemini, Perplexity, Parallel, and Valyu.","type":"module","keywords":["pi-package","pi-extension","coding-agent","web-search","claude","cloudflare","codex","custom-cli","exa","gemini","perplexity","parallel","valyu"],"author":{"name":"arpagon"},"license":"MIT","repository":{"type":"git","url":"git+https://github.com/arpagon/pi-web-providers.git"},"engines":{"node":">=24.0.0"},"publishConfig":{"access":"public"},"pi":{"extensions":["./dist/index.js"]},"scripts":{"build":"rm -rf dist && esbuild src/index.ts --bundle --format=esm --platform=node --outfile=dist/index.js --external:@mariozechner/pi-coding-agent --external:@mariozechner/pi-ai --external:@mariozechner/pi-tui --external:@sinclair/typebox --external:@anthropic-ai/claude-agent-sdk --external:@google/genai --external:@openai/codex-sdk --external:@perplexity-ai/perplexity_ai --external:exa-js --external:parallel-web --external:valyu-js","prepare":"npm run build","prepack":"npm run build","check":"tsc --noEmit","format":"biome format --write .","format:check":"biome format .","test":"vitest run","test:watch":"vitest"},"dependencies":{"@anthropic-ai/claude-agent-sdk":"^0.2.71","@google/genai":"^1.44.0","@openai/codex-sdk":"^0.111.0","@perplexity-ai/perplexity_ai":"^0.26.1","exa-js":"^2.7.0","parallel-web":"^0.3.1","valyu-js":"^2.5.9","zod":"^4.1.11"},"devDependencies":{"@biomejs/biome":"^2.4.6","@mariozechner/pi-ai":"*","@mariozechner/pi-coding-agent":"*","@sinclair/typebox":"*","@types/node":"^24.3.0","esbuild":"^0.25.10","typescript":"^5.9.3","vitest":"^4.0.18"},"_id":"@arpagon/pi-web-providers@1.2.0","gitHead":"0b28f996d2b51e4159b528a2c951b794b3a215de","bugs":{"url":"https://github.com/arpagon/pi-web-providers/issues"},"homepage":"https://github.com/arpagon/pi-web-providers#readme","_nodeVersion":"20.19.6","_npmVersion":"10.8.2","dist":{"integrity":"sha512-2lUBq7qGrZrng6D3Nw5hsKDa33zewWJs3kIl9FS3rW/XdHcPyXeeprGp6HbhXlyEc8tPb+i1rjCeSiOzXBF5qA==","shasum":"68d64f4e7ad1122af82d475b1708bfbba6751e69","tarball":"https://registry.npmjs.org/@arpagon/pi-web-providers/-/pi-web-providers-1.2.0.tgz","fileCount":10,"unpackedSize":307007,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIATfh8pVd00VA9zqF91RbXmKguspQYfaRlcVgKXw+AmdAiBKqwDIArV2SQ81F8TwbD9X9vb3G0LTtjAp4f+pbOx4Fw=="}]},"_npmUser":{"name":"arpagon","email":"arpagon@gmail.com"},"directories":{},"maintainers":[{"name":"arpagon","email":"arpagon@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/pi-web-providers_1.2.0_1773931504058_0.5302472710787638"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-19T14:45:03.937Z","1.2.0":"2026-03-19T14:45:04.195Z","modified":"2026-03-19T14:45:04.421Z"},"maintainers":[{"name":"arpagon","email":"arpagon@gmail.com"}],"description":"Configurable web access extension for pi with per-tool provider routing for search, contents, answers, and research across Claude, Cloudflare, Codex, Custom CLI, Exa, Gemini, Perplexity, Parallel, and Valyu.","homepage":"https://github.com/arpagon/pi-web-providers#readme","keywords":["pi-package","pi-extension","coding-agent","web-search","claude","cloudflare","codex","custom-cli","exa","gemini","perplexity","parallel","valyu"],"repository":{"type":"git","url":"git+https://github.com/arpagon/pi-web-providers.git"},"author":{"name":"arpagon"},"bugs":{"url":"https://github.com/arpagon/pi-web-providers/issues"},"license":"MIT","readme":"# π pi-web-providers\n\nA _meta_ web extension for [pi](https://pi.dev) that routes search, content\nextraction, answers, and research through configurable per-tool providers.\n\n## Why?\n\nMost web extensions hard-wire a single backend. **pi-web-providers** lets you\nmix and match providers per tool instead, so `web_search`, `web_contents`,\n`web_answer`, and `web_research` can each use a different backend or be turned\noff entirely.\n\n## β¨ Features\n\n- **Multiple providers** β Claude, Cloudflare, Codex, Custom CLI, Exa, Gemini,\n Perplexity, Parallel, Valyu\n- **Batched search and answers** β run several related queries in a single\n `web_search` or `web_answer` call and get grouped results back in one response\n- **Async contents prefetch** β optionally start background `web_contents`\n extraction from `web_search` results and reuse the cached pages later\n\n## π¦ Install\n\n```bash\npi install npm:pi-web-providers\n```\n\n## βοΈ Configure\n\nRun:\n\n```text\n/web-providers\n```\n\nThis edits the global config file `~/.pi/agent/web-providers.json`. The\nsettings UI mirrors the three sections below: tools, providers, and generic\nsettings.\n\nEach tool can be routed to any compatible provider:\n\n| Provider | search | contents | answer | research | Auth |\n| -------------- | :----: | :------: | :----: | :------: | -------------------------------------------- |\n| **Claude** | β | | β | | Local Claude Code auth |\n| **Cloudflare** | | β | | | `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` |\n| **Codex** | β | | | | Local Codex CLI auth |\n| **Exa** | β | β | β | β | `EXA_API_KEY` |\n| **Gemini** | β | | β | β | `GOOGLE_API_KEY` |\n| **Perplexity** | β | | β | β | `PERPLEXITY_API_KEY` |\n| **Parallel** | β | β | | | `PARALLEL_API_KEY` |\n| **Valyu** | β | β | β | β | `VALYU_API_KEY` |\n\nAdvanced option: `custom-cli` is a configurable adapter provider that can route\nany managed tool through a local wrapper command using a JSON stdin/stdout\ncontract.\n\nSee [`example-config.json`](example-config.json) for a full default\nconfiguration.\n\n### Tools\n\nEach managed tool maps to one provider id or `null` for off under the top-level\n`tools` key. A tool is only exposed when it is mapped to a compatible provider\nand that provider is currently available. Tool-specific settings live under\n`toolSettings`; today this covers `toolSettings.search.prefetch`.\n\n#### `web_search`\n\nSearch the public web for up to 10 queries in one call. It returns grouped\ntitles, URLs, and snippets for each query.\n\n\nParameters and behavior
\n\n| Parameter | Type | Default | Description |\n| ------------ | -------- | -------- | -------------------------------------------------------------- |\n| `queries` | string[] | required | One or more search queries to run (max 10) |\n| `maxResults` | integer | `5` | Result count per query, clamped to `1β20` |\n| `options` | object | β | Provider-specific search options and local `prefetch` settings |\n\n`web_search.options.prefetch` is local-only and not forwarded into the provider\nSDK. It accepts `provider`, `maxUrls`, `ttlMs`, and `contentsOptions`, and\nstarts a background page-extraction workflow only when `prefetch.provider` is\nset. `/web-providers` can also persist default search prefetch settings under\n`toolSettings.search.prefetch`.\n\n \n\n#### `web_contents`\n\nRead the main text from one or more web pages. It reuses cached pages when they\nmatch and fetches only missing or stale URLs.\n\n\nParameters and behavior
\n\n| Parameter | Type | Default | Description |\n| --------- | -------- | -------- | ------------------------------------ |\n| `urls` | string[] | required | One or more URLs to extract |\n| `options` | object | β | Provider-specific extraction options |\n\n`web_contents` reuses any matching cached pages already present in the local\ncontent storeβwhether they came from prefetch or an earlier readβand only\nfetches missing or stale URLs.\n\n \n\n#### `web_answer`\n\nAnswer one or more questions using web-grounded evidence. When you ask more\nthan one question, the response is grouped into per-question sections.\n\n\nParameters and behavior
\n\n| Parameter | Type | Default | Description |\n| --------- | -------- | -------- | ---------------------------------------------------- |\n| `queries` | string[] | required | One or more questions to answer in one call (max 10) |\n| `options` | object | β | Provider-specific options |\n\nResponses are grouped into per-question sections when more than one question is\nprovided.\n\n \n\n#### `web_research`\n\nInvestigate a topic across web sources and produce a longer report. The\nprovider-specific `options` stay native to each SDK, and runtime options\noverride provider configuration when both are set.\n\n\nParameters and behavior
\n\n| Parameter | Type | Default | Description |\n| --------- | ------ | -------- | -------------------------- |\n| `input` | string | required | Research brief or question |\n| `options` | object | β | Provider-specific options |\n\n`options` are provider-native and provider-specific. Equivalent concepts can use\ndifferent field names across SDKsβfor example Perplexity uses `country`, Exa\nuses `userLocation`, and Valyu uses `countryCode`. Runtime `options` override\nprovider-native config, but managed tool inputs and tool wiring stay fixed.\n\n \n\n\nTimeout, retry, and delivery modes
\n\nThe extension accepts local control fields for robustness: `requestTimeoutMs`,\n`retryCount`, and `retryDelayMs` on request/response tools, plus\n`pollIntervalMs`, `timeoutMs`, `maxConsecutivePollErrors`, and `resumeId` on\n`web_research` for lifecycle-based research providers. These fields are handled\nby the extension and are not forwarded into the provider SDK call.\n\n- Exa and Valyu research support polling, overall deadlines, and resume IDs\n but reject `requestTimeoutMs` and do not retry non-idempotent job creation.\n- Perplexity research runs in streaming foreground mode and only supports\n `requestTimeoutMs`, `retryCount`, and `retryDelayMs`.\n\nProviders deliver results in one of three modes:\n\n- **Silent foreground** β no intermediate output; result returned when done.\n- **Streaming foreground** β progress updates while running, but the result is\n still only usable after the tool finishes.\n- **Background research** β the provider runs in the background; if\n interrupted, the run can be resumed later via `resumeId`.\n\n \n\n### Providers\n\nThe built-in providers below are thin adapters around official SDKs.\n\n\nClaude
\n\n- SDK: `@anthropic-ai/claude-agent-sdk`\n- Uses Claude Code's built-in `WebSearch` and `WebFetch` tools behind a\n structured JSON adapter\n- Runs in **silent foreground** mode\n- Supports request-shaping `options` such as `model`, `thinking`, `effort`, and\n `maxTurns`\n- Great for search plus grounded answers if you already use Claude Code locally\n\n \n\n\nCloudflare
\n\n- API: [Cloudflare Browser Rendering REST API](https://developers.cloudflare.com/browser-rendering/rest-api/)\n- Uses the `/markdown` endpoint to render pages in headless Chrome on\n Cloudflare's edge network and return clean Markdown\n- Runs in **silent foreground** mode\n- No external SDK dependency β uses `fetch` against the REST API\n- Handles JavaScript-rendered pages, dynamic content, and complex layouts\n- Reports per-URL success/failure with structured content entries\n- Requires a Cloudflare API token with **Browser Rendering** permissions and\n an account ID\n- Free tier: 10 min/day browser time, 6 req/min; Workers Paid ($5/mo):\n 10 hrs/month, 600 req/min\n\n \n\n\nCodex
\n\n- SDK: `@openai/codex-sdk`\n- Runs in read-only mode with web search enabled\n- Runs in **silent foreground** mode\n- Supports request-shaping `web_search.options` such as `model`,\n `modelReasoningEffort`, and `webSearchMode`\n- Best if you already use the local Codex CLI and auth flow\n\n \n\n\nExa
\n\n- SDK: `exa-js`\n- Search, contents, and answer run in **silent foreground** mode\n- Research runs in **background research** mode and supports `resumeId`\n- Neural, keyword, hybrid, and deep-research search modes\n- Inline text-content extraction on search results\n\n \n\n\nGemini
\n\n- SDK: `@google/genai`\n- Search and answer run in **silent foreground** mode\n- Research runs in **background research** mode and supports `resumeId`\n- Google Search grounding for answers\n- Deep-research agents via Google's Gemini API\n- Supports provider-native request options such as `model`, `config`,\n `generation_config`, and `agent_config` depending on the tool\n\n \n\n\nPerplexity
\n\n- SDK: `@perplexity-ai/perplexity_ai`\n- `web_search` and `web_answer` run in **silent foreground** mode\n- `web_research` runs in **streaming foreground** mode (no `resumeId` support)\n- Uses Perplexity Search for `web_search`\n- Uses Sonar for `web_answer` and `sonar-deep-research` for `web_research`\n- Supports provider-specific `web_search.options` such as `country`,\n `search_mode`, `search_domain_filter`, and `search_recency_filter`\n\n \n\n\nParallel
\n\n- SDK: `parallel-web`\n- Runs in **silent foreground** mode\n- Agentic and one-shot search modes\n- Page content extraction with excerpt and full-content toggles\n- Supports provider-native search and extraction options from the Parallel SDK\n\n \n\n\nValyu
\n\n- SDK: `valyu-js`\n- Search, contents, and answer run in **silent foreground** mode\n- Research runs in **background research** mode and supports `resumeId`\n- Web, proprietary, and news search types\n- Supports provider-native options such as `countryCode`, `responseLength`, and\n search/source filters\n- Configurable response length for answers and research\n\n \n\n### Custom CLI provider\n\nThe `custom-cli` provider lets you bring your own wrapper command for any\nmanaged tool. Each capability can point at a different local command under\n`providers[\"custom-cli\"].native`.\n\nThe repo includes actual wrapper examples under\n[`examples/custom-cli/wrappers/`](examples/custom-cli/wrappers/). They are\nsmall bash scripts that use `jq` for JSON handling. Each one uses a different\nbackend pattern:\n\n- `codex --search exec` for `web_search`\n- Gemini API via `curl` for `web_contents`\n- `claude -p` for `web_answer`\n- Perplexity API via `curl` for `web_research`\n\n\nConfiguration example
\n\nCopy the example wrappers into a local `./wrappers/` directory, then configure:\n\n```json\n{\n \"tools\": {\n \"search\": \"custom-cli\",\n \"contents\": \"custom-cli\",\n \"answer\": \"custom-cli\",\n \"research\": \"custom-cli\"\n },\n \"providers\": {\n \"custom-cli\": {\n \"enabled\": true,\n \"native\": {\n \"search\": {\n \"argv\": [\"bash\", \"./wrappers/codex-search.sh\"]\n },\n \"contents\": {\n \"argv\": [\"bash\", \"./wrappers/gemini-contents.sh\"]\n },\n \"answer\": {\n \"argv\": [\"bash\", \"./wrappers/claude-answer.sh\"]\n },\n \"research\": {\n \"argv\": [\"bash\", \"./wrappers/perplexity-research.sh\"]\n }\n }\n }\n }\n}\n```\n\nThose example wrappers deliberately use different local CLIs and APIs so you\ncan see several wrapper styles in one setup without extra glue code.\n\nEach capability can also set an optional `cwd` and `env` block. Use `cwd` when\none wrapper must run from a specific directory. Use `env` for per-command\nvariables; each value can be a literal string, an environment variable name, or\n`!command`.\n\n`web_research` runs as a foreground wrapper command, so local polling controls\n(`pollIntervalMs`, `timeoutMs`, `maxConsecutivePollErrors`) and `resumeId` do\nnot apply to `custom-cli`.\n\nWrapper contract:\n\n- `stdin`: one JSON request object with `capability` plus the per-call managed\n inputs (`query`, `urls`, `input`, `maxResults`, `options`, `cwd`)\n- `stdout`: one JSON response object\n - `search`: `{ \"results\": [{ \"title\", \"url\", \"snippet\" }] }`\n - `contents` / `answer` / `research`: `{ \"text\": \"...\", \"summary\"?: \"...\", \"itemCount\"?: 1, \"metadata\"?: {} }`\n- `stderr`: optional progress lines\n- exit code `0`: success\n- non-zero exit code: failure\n\n \n\nSee [`examples/custom-cli/README.md`](examples/custom-cli/README.md) for a\ncopy-and-pasteable setup, and see\n[`examples/custom-cli/wrappers/`](examples/custom-cli/wrappers/) for the actual\nwrapper files.\n\n### Generic settings\n\nThe `genericSettings` block sets shared execution defaults that apply to all\nproviders unless overridden in a provider's `policy` block:\n\n| Field | Default | Description |\n| ---------------------------------- | ---------- | ---------------------------------------------- |\n| `requestTimeoutMs` | `30000` | Maximum time for a single provider request |\n| `retryCount` | `3` | Retries for transient failures |\n| `retryDelayMs` | `2000` | Initial delay before retrying |\n| `researchPollIntervalMs` | `3000` | How often to poll long-running research jobs |\n| `researchTimeoutMs` | `21600000` | Overall deadline for research before returning |\n| `researchMaxConsecutivePollErrors` | `3` | Consecutive poll failures before stopping |\n\n## π License\n\n[MIT](LICENSE)\n","readmeFilename":"README.md","_rev":"1-7bd5bdbebc45742f63f1d83813ffcec3"}