{"_id":"@codingcoffee/pi-websearch-crawl4ai","_rev":"6-7f45cb08bad3d4dc0c45b6b1809fc912","name":"@codingcoffee/pi-websearch-crawl4ai","dist-tags":{"latest":"0.2.1"},"versions":{"0.1.0":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.1.0","keywords":["pi","pi-coding-agent","pi-extension","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"url":"https://codingcoffee.dev","name":"Ameya Shenoy"},"license":"MIT","_id":"@codingcoffee/pi-websearch-crawl4ai@0.1.0","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"homepage":"https://pi-websearch-crawl4ai.codingcoffee.me/","bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"6868a895cc7502b82f93462b3b344ea517e072fb","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.1.0.tgz","fileCount":4,"integrity":"sha512-MNy9hNihgQh8yY1uqhlYN9D04a83pQDl50fYjz9Z0Xb743TYVGU7253vU2/vU9hYuv2FS5rO1h+rSUwrRebwxQ==","signatures":[{"sig":"MEUCIQDAgoRq49pPW1ktMZh8m/TCHE5BD8rzXdkNXi5CQU9XrwIgZeepmwco4yNVPDW+3j1m6K0BNWVgSWlqv59ahxQpXSU=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23549},"type":"module","gitHead":"e36b4637f6e2c2566ecfd64e7ea27056f0fbabb2","scripts":{"build":"echo 'nothing to build'","check":"echo 'nothing to check'","clean":"echo 'nothing to clean'"},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"repository":{"url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git","type":"git"},"_npmVersion":"11.12.1","description":"pi extension: fetch web content via a Crawl4AI server — use when the bash tool (curl/wget) is disabled.","directories":{},"_nodeVersion":"24.3.0","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/pi-websearch-crawl4ai_0.1.0_1776838442673_0.11580899303207892","host":"s3://npm-registry-packages-npm-production"}},"0.1.1":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.1.1","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"url":"https://codingcoffee.dev","name":"Ameya Shenoy"},"license":"MIT","_id":"@codingcoffee/pi-websearch-crawl4ai@0.1.1","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"homepage":"https://pi-websearch-crawl4ai.codingcoffee.me","bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"04de9ae9348bbccd15aa345fbb88b21ed54be134","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.1.1.tgz","fileCount":4,"integrity":"sha512-SJ1+j05U++I3eM2iUzmrK2MltKeowofcBBNhFaZHg8GusE3dIgY2qCjHhqQLOINWNznRSYFHUsXKiF7ktE5cgw==","signatures":[{"sig":"MEYCIQD2Ruv6G1eevYEV9Ps5t5sRzqayA8DTU3n9pDXfR+UvQAIhAJ3dd7KxkUnj/0o04o2w5yFxzcitKUUkfDEU76TXLyMb","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23528},"type":"module","gitHead":"06dfd09c9d3879d259cadb14f0fbb8b564994023","scripts":{"build":"echo 'nothing to build'","check":"echo 'nothing to check'","clean":"echo 'nothing to clean'"},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"repository":{"url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git","type":"git"},"_npmVersion":"11.12.1","description":"a pi extension to let your LLM crawl & see the web","directories":{},"_nodeVersion":"24.3.0","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/pi-websearch-crawl4ai_0.1.1_1776863236282_0.9918006129187962","host":"s3://npm-registry-packages-npm-production"}},"0.1.2":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.1.2","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"url":"https://codingcoffee.dev","name":"Ameya Shenoy"},"license":"MIT","_id":"@codingcoffee/pi-websearch-crawl4ai@0.1.2","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"homepage":"https://pi-websearch-crawl4ai.codingcoffee.dev","bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"pi":{"extensions":["./index.ts"]},"dist":{"shasum":"38f057e3443a6c172d1a3c9a2379301a75a56015","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.1.2.tgz","fileCount":4,"integrity":"sha512-ceCiAi+CbkVInltjxQLi2iNRoJjX7n1N1WvMP5TXVPcU3oTYs2xbK6uAB9OaWMf1LSvmGCU/vRTRgZMK30wQIA==","signatures":[{"sig":"MEQCIB9TO9jU3EqQ49pAyVbfxZf8dtMp0DR+As++9AK8ITrpAiB96sezdzX71iaXMFwgmGjFQ1i/ghIOJBDIuu+N7zRIfw==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23529},"type":"module","gitHead":"cb6a7de8f050db630a377f5ce96b7ec2939efe05","scripts":{"build":"echo 'nothing to build'","check":"echo 'nothing to check'","clean":"echo 'nothing to clean'"},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"repository":{"url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git","type":"git"},"_npmVersion":"11.12.1","description":"a pi extension to let your LLM crawl & see the web","directories":{},"_nodeVersion":"24.3.0","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/pi-websearch-crawl4ai_0.1.2_1776863527099_0.41910975522625926","host":"s3://npm-registry-packages-npm-production"}},"0.1.3":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.1.3","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"url":"https://codingcoffee.dev","name":"Ameya Shenoy"},"license":"MIT","_id":"@codingcoffee/pi-websearch-crawl4ai@0.1.3","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"homepage":"https://pi-websearch-crawl4ai.codingcoffee.dev","bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"pi":{"image":"pi-websearch-crawl4ai.codingcoffee.dev/og.png","extensions":["./index.ts"]},"dist":{"shasum":"1d2450ca075d0febe2c050af731c38cee7fbd013","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.1.3.tgz","fileCount":4,"integrity":"sha512-MlZhbAOayRt0voQcjTmgJ47dEuOrvztq03CbEK5u9wOH5Bz6W7cwCZ6PhTDMjuI1KK9suITwV2Q8WvNZgrgLGQ==","signatures":[{"sig":"MEUCIHXSmpeCdsMqFrJSjShslA4r5WGTp47sKTSkg+bMtSZ5AiEArK4BC3r7qH+xPt9mQlROZWErmorkG+kBWmhN9kKBxmA=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23591},"type":"module","gitHead":"82416eed254bdf5e7cf851755d7295e4a827c63d","scripts":{"build":"echo 'nothing to build'","check":"echo 'nothing to check'","clean":"echo 'nothing to clean'"},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"repository":{"url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git","type":"git"},"_npmVersion":"11.13.0","description":"a pi extension to let your LLM crawl & see the web","directories":{},"_nodeVersion":"24.3.0","_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/pi-websearch-crawl4ai_0.1.3_1776914166853_0.0019828282925846707","host":"s3://npm-registry-packages-npm-production"}},"0.2.0":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.2.0","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"url":"https://codingcoffee.dev","name":"Ameya Shenoy"},"license":"MIT","_id":"@codingcoffee/pi-websearch-crawl4ai@0.2.0","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"homepage":"https://pi-websearch-crawl4ai.codingcoffee.dev","bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"pi":{"image":"pi-websearch-crawl4ai.codingcoffee.dev/og.png","extensions":["./index.ts"]},"dist":{"shasum":"accb3560712946c2d5d58e90dfc472d88a33a8c3","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.2.0.tgz","fileCount":4,"integrity":"sha512-s44u69ungAu1kRBQkXHACr5kuueGOtXVWGqI6nQ6ZL/6zKlD/5VBrBir2Jmp/TaRfbQiU5BhZI9HA3PEiz8TaQ==","signatures":[{"sig":"MEUCIQDg0IrFZ+hrnd6C1TjthmrANL0kdHFyF1dG5QFWKtwlfgIgfwnQ2SKxiPiNwkBdXjBqFoshwBP47nND12pbM9lkXgs=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":23873},"type":"module","gitHead":"84494417f2fcce3f820bbfd568481e79ff262f7c","scripts":{"build":"echo 'nothing to build'","check":"echo 'nothing to check'","clean":"echo 'nothing to clean'"},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"repository":{"url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git","type":"git"},"_npmVersion":"11.13.0","description":"a pi extension to let your LLM crawl & see the web","directories":{},"_nodeVersion":"24.3.0","_hasShrinkwrap":false,"devDependencies":{"@sinclair/typebox":"^0.34.0","@mariozechner/pi-ai":"^0.70.0","@mariozechner/pi-coding-agent":"^0.70.0"},"peerDependencies":{"@sinclair/typebox":"*","@mariozechner/pi-ai":"*","@mariozechner/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-websearch-crawl4ai_0.2.0_1777230802565_0.573514895344835","host":"s3://npm-registry-packages-npm-production"}},"0.2.1":{"name":"@codingcoffee/pi-websearch-crawl4ai","version":"0.2.1","description":"a pi extension to let your LLM crawl & see the web","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"author":{"name":"Ameya Shenoy","url":"https://codingcoffee.dev"},"license":"MIT","homepage":"https://pi-websearch-crawl4ai.codingcoffee.dev","repository":{"type":"git","url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git"},"bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"type":"module","pi":{"extensions":["./index.ts"],"image":"https://pi-websearch-crawl4ai.codingcoffee.dev/og.png"},"peerDependencies":{"@mariozechner/pi-coding-agent":"*","@mariozechner/pi-ai":"*","@sinclair/typebox":"*"},"devDependencies":{"@mariozechner/pi-coding-agent":"^0.70.0","@mariozechner/pi-ai":"^0.70.0","@sinclair/typebox":"^0.34.0"},"scripts":{"clean":"echo 'nothing to clean'","build":"echo 'nothing to build'","check":"echo 'nothing to check'"},"gitHead":"59473dd4145286c8090ca032a94f06a5cdefd234","_id":"@codingcoffee/pi-websearch-crawl4ai@0.2.1","_nodeVersion":"24.3.0","_npmVersion":"11.13.0","dist":{"integrity":"sha512-BntAiFq0KiAtDED4LW3yI0avYnw5/KTvtSJcXfoVeo3mSrtLSeM6DM2OBhN40WtL1cM7t3Q4MK/sTihUIPUmyA==","shasum":"16d05fc1bf3e9d6caa7be981cfb40be3b5fcc948","tarball":"https://registry.npmjs.org/@codingcoffee/pi-websearch-crawl4ai/-/pi-websearch-crawl4ai-0.2.1.tgz","fileCount":4,"unpackedSize":23881,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQDbcVugmqVyGXWZe+BdxEDC3g+IzQQ3QAsryTeqldFsogIhAN4cHlJbhMXorjEa7QVCxIEkcSPiNA7KxlsaIUfHP57z"}]},"_npmUser":{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"},"directories":{},"maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/pi-websearch-crawl4ai_0.2.1_1777230895496_0.5912732821053579"},"_hasShrinkwrap":false}},"time":{"created":"2026-04-22T06:14:02.588Z","modified":"2026-04-26T19:14:55.712Z","0.1.0":"2026-04-22T06:14:02.822Z","0.1.1":"2026-04-22T13:07:16.440Z","0.1.2":"2026-04-22T13:12:07.235Z","0.1.3":"2026-04-23T03:16:07.018Z","0.2.0":"2026-04-26T19:13:22.717Z","0.2.1":"2026-04-26T19:14:55.617Z"},"bugs":{"url":"https://github.com/codingcoffee/pi-websearch-crawl4ai/issues"},"author":{"name":"Ameya Shenoy","url":"https://codingcoffee.dev"},"license":"MIT","homepage":"https://pi-websearch-crawl4ai.codingcoffee.dev","keywords":["pi-package","pi-extension","pi","llm-tools","pi-coding-agent","crawl4ai","web-scraping","llm","agent","markdown","fetch"],"repository":{"type":"git","url":"git+https://github.com/codingcoffee/pi-websearch-crawl4ai.git"},"description":"a pi extension to let your LLM crawl & see the web","maintainers":[{"name":"codingcoffee","email":"shenoy.ameya@gmail.com"}],"readme":"# pi-websearch-crawl4ai\n\nA [pi](https://github.com/badlogic/pi-mono) extension that lets the agent fetch\ncontent from the web via a running [Crawl4AI](https://docs.crawl4ai.com/) server.\n\n**Intended use:** running pi with the `bash` tool disabled (so `curl` / `wget`\nare unavailable) while still letting the model read and crawl web pages.\n\n## What it gives the LLM\n\nSix tools, all talking to a Crawl4AI server:\n\n| Tool              | Endpoint           | Purpose                                                   |\n| ----------------- | ------------------ | --------------------------------------------------------- |\n| `web_fetch`       | `POST /md`         | Fetch a URL → clean Markdown (filters: fit/raw/bm25/llm)  |\n| `web_fetch_html`  | `POST /html`       | Sanitized HTML for DOM-aware tasks                        |\n| `web_crawl`       | `POST /crawl`      | Multi-URL crawl with typed `BrowserConfig`/`CrawlerRunConfig` |\n| `web_execute_js`  | `POST /execute_js` | Run JS snippets on a page and read back JSON              |\n| `web_screenshot`  | `POST /screenshot` | Full-page PNG screenshot (returned inline)                |\n| `web_ask`         | `GET  /ask`        | Query the Crawl4AI library's own docs (for configuring it) |\n\nPlus commands: `/crawl4ai-status`, `/crawl4ai-url <url>`, `/crawl4ai-token <tok>`.\n\n## Prerequisites\n\nYou need a Crawl4AI server reachable from where pi runs. The fastest path:\n\n```bash\ndocker run -d \\\n  -p 11235:11235 \\\n  --name crawl4ai \\\n  --shm-size=1g \\\n  unclecode/crawl4ai:latest\n\n# Sanity check\ncurl http://localhost:11235/health\n```\n\nSee the [Crawl4AI Docker guide](https://github.com/unclecode/crawl4ai/blob/main/deploy/docker/README.md)\nfor GPU, LLM keys, `config.yml`, JWT auth, etc.\n\n## Install as a pi extension\n\n```bash\n# project-local\nmkdir -p .pi/extensions\nln -s \"$(pwd)/pi-websearch-crawl4ai\" .pi/extensions/crawl4ai\n\n# or global\nmkdir -p ~/.pi/agent/extensions\nln -s \"$(pwd)/pi-websearch-crawl4ai\" ~/.pi/agent/extensions/crawl4ai\n```\n\npi auto-discovers `index.ts` via the `\"pi\".extensions` field in `package.json`.\n\nAlternatively, for a one-off test:\n\n```bash\npi -e ./pi-websearch-crawl4ai/index.ts\n```\n\n## Configuration\n\nPrecedence: CLI flag > env var > default.\n\n| Setting   | Env                   | Flag                       | Default                   |\n| --------- | --------------------- | -------------------------- | ------------------------- |\n| Base URL  | `CRAWL4AI_BASE_URL`   | `--crawl4ai-url <url>`     | `http://localhost:11235`  |\n| Auth token| `CRAWL4AI_TOKEN`      | `--crawl4ai-token <tok>`   | (none)                    |\n\nAt runtime you can also:\n\n- `/crawl4ai-status` — show current config + `/health`\n- `/crawl4ai-url http://host:11235` — change base URL for this session\n- `/crawl4ai-token <jwt>` — set bearer token (empty clears it)\n\n## Example use\n\nRunning pi with only read-only tools and this extension:\n\n```bash\npi --tools read,write,edit,web_fetch,web_crawl\n> \"Read https://example.com and summarize it.\"\n```\n\nThe model will call `web_fetch` instead of reaching for `bash`/`curl`.\n\n## How `web_crawl` typed configs work\n\nCrawl4AI accepts configuration objects shaped as\n`{\"type\":\"ClassName\",\"params\":{...}}`. Example you (or the model) can pass:\n\n```json\n{\n  \"urls\": [\"https://example.com\", \"https://httpbin.org/html\"],\n  \"browser_config\": { \"type\": \"BrowserConfig\", \"params\": { \"headless\": true } },\n  \"crawler_config\": {\n    \"type\": \"CrawlerRunConfig\",\n    \"params\": { \"cache_mode\": \"bypass\", \"stream\": false }\n  }\n}\n```\n\nIf you need to remind the model what's available, it can call `web_ask` with a\nquery like `\"CrawlerRunConfig parameters\"` to pull the Crawl4AI library docs.\n\n## Security note\n\nExtensions run with your user's full permissions. The tools here can fetch\narbitrary URLs via your Crawl4AI server. If that's a problem, run Crawl4AI with\nrate limiting / allowlists configured in its `config.yml`, and/or restrict\nwhich tools pi activates via `--tools`.\n","readmeFilename":"README.md"}