{"_id":"@benvargas/pi-firecrawl","_rev":"4-13cae890ea80af88f9d7752e709fb64f","name":"@benvargas/pi-firecrawl","dist-tags":{"latest":"1.1.0"},"versions":{"1.0.0":{"name":"@benvargas/pi-firecrawl","version":"1.0.0","keywords":["pi","pi-package","pi-extension","pi-coding-agent","ai","firecrawl","web-scraping","crawling","data-extraction"],"author":{"name":"Ben Vargas"},"license":"MIT","_id":"@benvargas/pi-firecrawl@1.0.0","maintainers":[{"name":"benvargas","email":"ben@vargas.com"}],"homepage":"https://github.com/ben-vargas/pi-packages/tree/main/packages/pi-firecrawl#readme","bugs":{"url":"https://github.com/ben-vargas/pi-packages/issues"},"pi":{"extensions":["./extensions/index.ts"]},"dist":{"shasum":"3fe874cf26ffb1f06caf3249a35632ebaaf88fcd","tarball":"https://registry.npmjs.org/@benvargas/pi-firecrawl/-/pi-firecrawl-1.0.0.tgz","fileCount":4,"integrity":"sha512-yrVG2gyFzR4RR6fPkA2UuF2TepS2CrgxdEeNX11FRDyMul+TUUyED3w7cCINMjugGhS50YIEmzYLnWRniM3ODQ==","signatures":[{"sig":"MEUCIQD16AAnDQP5+zIsBDUl0xvEn97lkjAimuoCkUyReFhryAIgSKYcXQgoSd7mwBQjEtg1fJAf4YeWnWfjJvPOF1bUZZA=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":26062},"type":"module","gitHead":"ec3457a8ede5cc6f4a9669ef69991d73ee16541c","_npmUser":{"name":"benvargas","email":"ben@vargas.com"},"repository":{"url":"git+https://github.com/ben-vargas/pi-packages.git","type":"git","directory":"packages/pi-firecrawl"},"_npmVersion":"11.7.0","description":"Firecrawl tools for pi - Web scraping, crawling, and data extraction via Firecrawl's REST API","directories":{},"_nodeVersion":"24.11.0","_hasShrinkwrap":false,"peerDependencies":{"@sinclair/typebox":"*","@mariozechner/pi-ai":"*","@mariozechner/pi-coding-agent":"*"},"_npmOperationalInternal":{"tmp":"tmp/pi-firecrawl_1.0.0_1770840930099_0.9302797866793888","host":"s3://npm-registry-packages-npm-production"}},"1.0.1":{"name":"@benvargas/pi-firecrawl","version":"1.0.1","keywords":["pi","pi-package","pi-extension","pi-coding-agent","ai","firecrawl","web-scraping","crawling","data-extraction"],"author":{"name":"Ben Vargas"},"license":"MIT","_id":"@benvargas/pi-firecrawl@1.0.1","maintainers":[{"name":"benvargas","email":"ben@vargas.com"}],"homepage":"https://github.com/ben-vargas/pi-packages/tree/main/packages/pi-firecrawl#readme","bugs":{"url":"https://github.com/ben-vargas/pi-packages/issues"},"pi":{"extensions":["./extensions/index.ts"]},"dist":{"shasum":"8f6da1efa3f923fd6ce24326fff1289591daea92","tarball":"https://registry.npmjs.org/@benvargas/pi-firecrawl/-/pi-firecrawl-1.0.1.tgz","fileCount":4,"integrity":"sha512-5AxbbWJ9pGieTyUoJFJlewAhSKcecL/YAuTET9HReFDurquzMTfcAfU9Zo9tNNfpyBem6IDmfTu8Qq8NT8/wKw==","signatures":[{"sig":"MEUCIQDfrVJ4BYB2B+gClqsSdUWwVtFgemSu4QIExAk6ULSGOQIgQUa5+uaE1D+jslxjyqyjrFeiSNdbUWAhkIuq6F8DRgA=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":26233},"type":"module","gitHead":"d9370e9ba085ca27e91b113b6df06559f9aa7ba5","_npmUser":{"name":"benvargas","email":"ben@vargas.com"},"repository":{"url":"git+https://github.com/ben-vargas/pi-packages.git","type":"git","directory":"packages/pi-firecrawl"},"_npmVersion":"11.9.0","description":"Firecrawl tools for pi - Web scraping, crawling, and data extraction via Firecrawl's REST API","directories":{},"_nodeVersion":"24.14.0","_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-ai":">=0.74.0","@earendil-works/pi-coding-agent":">=0.74.0"},"_npmOperationalInternal":{"tmp":"tmp/pi-firecrawl_1.0.1_1779738047878_0.5207726636322696","host":"s3://npm-registry-packages-npm-production"}},"1.0.2":{"name":"@benvargas/pi-firecrawl","version":"1.0.2","keywords":["pi","pi-package","pi-extension","pi-coding-agent","ai","firecrawl","web-scraping","crawling","data-extraction"],"author":{"name":"Ben Vargas"},"license":"MIT","_id":"@benvargas/pi-firecrawl@1.0.2","maintainers":[{"name":"benvargas","email":"ben@vargas.com"}],"homepage":"https://github.com/ben-vargas/pi-packages/tree/main/packages/pi-firecrawl#readme","bugs":{"url":"https://github.com/ben-vargas/pi-packages/issues"},"pi":{"extensions":["./extensions/index.ts"]},"dist":{"shasum":"2fd137d1faa7b838ed213fcb315433ab819f686c","tarball":"https://registry.npmjs.org/@benvargas/pi-firecrawl/-/pi-firecrawl-1.0.2.tgz","fileCount":4,"integrity":"sha512-l4CYsR5+BYFpOxo4gv4TJ+DyP5u3vrxMiVDwZCh2XD8xNfD6R/dQLWqNEmspCwYGB+tjJV0ryd8Qt46oSoz+Aw==","signatures":[{"sig":"MEQCIDV38MR2rcP66jWTUglUWd2ALyArvlHLOWCyVdRpS4BvAiAH9MCfmkKTRbf/ht3VBW1k2smIIAU/xDGNI8vgLsb4Tg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":26233},"type":"module","gitHead":"a14dbe2ba398271392483f31d0f5f62e3cb33a98","_npmUser":{"name":"benvargas","email":"ben@vargas.com"},"repository":{"url":"git+https://github.com/ben-vargas/pi-packages.git","type":"git","directory":"packages/pi-firecrawl"},"_npmVersion":"11.9.0","description":"Firecrawl tools for pi - Web scraping, crawling, and data extraction via Firecrawl's REST API","directories":{},"_nodeVersion":"24.14.0","_hasShrinkwrap":false,"peerDependencies":{"typebox":"*","@earendil-works/pi-ai":">=0.74.0","@earendil-works/pi-coding-agent":">=0.74.0"},"_npmOperationalInternal":{"tmp":"tmp/pi-firecrawl_1.0.2_1780888317984_0.12749256371212736","host":"s3://npm-registry-packages-npm-production"}},"1.1.0":{"name":"@benvargas/pi-firecrawl","version":"1.1.0","description":"Firecrawl tools for pi - Web scraping, crawling, and data extraction via Firecrawl's REST API","keywords":["pi","pi-package","pi-extension","pi-coding-agent","ai","firecrawl","web-scraping","crawling","data-extraction"],"type":"module","pi":{"extensions":["./extensions/index.ts"]},"peerDependencies":{"@earendil-works/pi-ai":">=0.74.0","@earendil-works/pi-coding-agent":">=0.74.0","typebox":"*"},"repository":{"type":"git","url":"git+https://github.com/ben-vargas/pi-packages.git","directory":"packages/pi-firecrawl"},"author":{"name":"Ben Vargas"},"license":"MIT","bugs":{"url":"https://github.com/ben-vargas/pi-packages/issues"},"homepage":"https://github.com/ben-vargas/pi-packages/tree/main/packages/pi-firecrawl#readme","gitHead":"fc0a6ac2bfb712f8e4d0fbf72857fff629837619","_id":"@benvargas/pi-firecrawl@1.1.0","_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-fqI7zTA44v3SStqRdlJjTJqmYXnUVrwWsV/wY+tgrJhyq0rp4GoctpDQ9+iVmGA+QzXNZKVoXWdrXc44HDEJ/w==","shasum":"67efda7c9ee2b5b3cfa081bc011d1ff06a95df4b","tarball":"https://registry.npmjs.org/@benvargas/pi-firecrawl/-/pi-firecrawl-1.1.0.tgz","fileCount":4,"unpackedSize":27736,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIQCf5xg6/KFUtgJckx8piM6N8HfpTi4inGM/CZxAvY3rigIgHc11EbkJHyDWDCP55Abm4vNimg0LtPk3Cp//dV2zx/8="}]},"_npmUser":{"name":"benvargas","email":"ben@vargas.com"},"directories":{},"maintainers":[{"name":"benvargas","email":"ben@vargas.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/pi-firecrawl_1.1.0_1786047034492_0.30018392058959975"},"_hasShrinkwrap":false}},"time":{"created":"2026-02-11T20:15:29.996Z","modified":"2026-08-06T20:10:34.784Z","1.0.0":"2026-02-11T20:15:30.307Z","1.0.1":"2026-05-25T19:40:48.019Z","1.0.2":"2026-06-08T03:11:58.124Z","1.1.0":"2026-08-06T20:10:34.615Z"},"bugs":{"url":"https://github.com/ben-vargas/pi-packages/issues"},"author":{"name":"Ben Vargas"},"license":"MIT","homepage":"https://github.com/ben-vargas/pi-packages/tree/main/packages/pi-firecrawl#readme","keywords":["pi","pi-package","pi-extension","pi-coding-agent","ai","firecrawl","web-scraping","crawling","data-extraction"],"repository":{"type":"git","url":"git+https://github.com/ben-vargas/pi-packages.git","directory":"packages/pi-firecrawl"},"description":"Firecrawl tools for pi - Web scraping, crawling, and data extraction via Firecrawl's REST API","maintainers":[{"name":"benvargas","email":"ben@vargas.com"}],"readme":"# @benvargas/pi-firecrawl\n\nFirecrawl tools for [pi](https://github.com/badlogic/pi-mono), the AI coding agent.\n\nProvides focused Firecrawl REST API access for:\n\n- `firecrawl_scrape`\n- `firecrawl_map`\n- `firecrawl_search`\n\n## Features\n\n- **Scrape** (`firecrawl_scrape`) — Scrape a single URL\n- **Map** (`firecrawl_map`) — Discover URLs on a site\n- **Search** (`firecrawl_search`) — Search the web and optionally scrape results\n- **Tool allowlist** — Enable/disable tools via `tools` config\n- **Configurable limits** — Client-side truncation with configurable max bytes/lines\n- **Multiple config sources** — JSON config, environment variables, or CLI flags\n- **Direct REST API** — Simple, fast, no protocol overhead\n\n## Installation\n\n```bash\npi install npm:@benvargas/pi-firecrawl\n```\n\nOr try without installing:\n\n```bash\npi -e npm:@benvargas/pi-firecrawl\n```\n\n## Setup\n\nGet an API key from [firecrawl.dev](https://firecrawl.dev) and configure it using one of these methods.\n\n### Option 1: Environment Variable\n\n```bash\nexport FIRECRAWL_API_KEY=\"your_firecrawl_api_key\"\npi\n```\n\n### Option 2: JSON Config\n\nCreate `firecrawl.json` in pi's agent extensions directory (`~/.pi/agent/extensions/firecrawl.json` by default; the agent directory honors `PI_CODING_AGENT_DIR`):\n\n```json\n{\n  \"url\": \"https://api.firecrawl.dev\",\n  \"apiKey\": \"your_firecrawl_api_key\",\n  \"headers\": null,\n  \"tools\": [\"firecrawl_scrape\", \"firecrawl_map\", \"firecrawl_search\"],\n  \"timeoutMs\": 30000,\n  \"maxBytes\": 51200,\n  \"maxLines\": 2000\n}\n```\n\nIf you want to disable search, remove it from the `tools` array.\n\n### Option 3: CLI Flags\n\n```bash\npi --firecrawl-api-key=your_firecrawl_api_key\n```\n\n## Usage\n\nOnce installed, the AI can use these tools when appropriate:\n\n```text\n\"Scrape https://example.com and get the main content\"\n\"Map docs.example.com and list likely API pages\"\n\"Search for React tutorials and summarize top results\"\n```\n\n## Tools\n\n### firecrawl_scrape\n\nScrape a single URL.\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `url` | Target URL (required) | — |\n| `formats` | Output formats array | — |\n| `onlyMainContent` | Extract only main content | — |\n| `waitFor` | Wait time in milliseconds | — |\n| `timeout` | Request timeout | — |\n| `mobile` | Use mobile viewport | — |\n| `includeTags` | Tags to include | — |\n| `excludeTags` | Tags to exclude | — |\n| `skipTlsVerification` | Skip TLS verification | — |\n| `piMaxBytes` | Client-side max bytes override | — |\n| `piMaxLines` | Client-side max lines override | — |\n\n### firecrawl_map\n\nDiscover URLs on a site.\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `url` | Base URL to map (required) | — |\n| `search` | Search query to filter URLs | — |\n| `sitemap` | Sitemap handling mode (`include`, `skip`, `only`) | — |\n| `includeSubdomains` | Include subdomains | — |\n| `limit` | Maximum URLs to return | — |\n| `ignoreQueryParameters` | Ignore query parameters | — |\n| `piMaxBytes` | Client-side max bytes override | — |\n| `piMaxLines` | Client-side max lines override | — |\n\n### firecrawl_search\n\nSearch the web and optionally scrape results.\n\n| Parameter | Description | Default |\n|-----------|-------------|---------|\n| `query` | Search query (required) | — |\n| `limit` | Maximum results | — |\n| `lang` | Language code | — |\n| `country` | Country code | — |\n| `scrapeOptions` | Scrape options for results | — |\n| `piMaxBytes` | Client-side max bytes override | — |\n| `piMaxLines` | Client-side max lines override | — |\n\n## Configuration\n\n### Config File Locations\n\nConfig files are loaded in order (first match wins):\n\n1. Path from `--firecrawl-config` flag\n2. Path from `FIRECRAWL_CONFIG` environment variable\n3. `./.pi/extensions/firecrawl.json` (project-level)\n4. `<agent-dir>/extensions/firecrawl.json` (global — `<agent-dir>` is pi's agent directory, `~/.pi/agent` by default or `$PI_CODING_AGENT_DIR` when set)\n5. `~/.pi/agent/extensions/firecrawl.json` (legacy global — only checked when `$PI_CODING_AGENT_DIR` relocates the agent directory, so configs written before the relocation keep working)\n\nIf none exist, the extension writes a default config to the global path (`<agent-dir>/extensions/firecrawl.json`) with `apiKey` set to `null`.\n\n### Config File Format\n\n```json\n{\n  \"url\": \"https://api.firecrawl.dev\",\n  \"apiKey\": null,\n  \"headers\": null,\n  \"tools\": [\"firecrawl_scrape\", \"firecrawl_map\", \"firecrawl_search\"],\n  \"timeoutMs\": 30000,\n  \"maxBytes\": 51200,\n  \"maxLines\": 2000\n}\n```\n\n### Environment Variables\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `FIRECRAWL_URL` | API base URL | `https://api.firecrawl.dev` |\n| `FIRECRAWL_API_KEY` | Firecrawl API key (Bearer token) | — |\n| `FIRECRAWL_TIMEOUT_MS` | Request timeout (ms) | `30000` |\n| `FIRECRAWL_TOOLS` | Comma-separated tool list | — |\n| `FIRECRAWL_MAX_BYTES` | Max bytes to keep from output | `51200` |\n| `FIRECRAWL_MAX_LINES` | Max lines to keep from output | `2000` |\n| `FIRECRAWL_CONFIG` | Path to JSON config file | — |\n\n### CLI Flags\n\n| Flag | Description |\n|------|-------------|\n| `--firecrawl-url` | Override the Firecrawl API base URL |\n| `--firecrawl-api-key` | Firecrawl API key |\n| `--firecrawl-timeout-ms` | HTTP timeout (milliseconds) |\n| `--firecrawl-config` | Path to JSON config file |\n| `--firecrawl-tools` | Comma-separated tool list |\n| `--firecrawl-max-bytes` | Max bytes to keep from output |\n| `--firecrawl-max-lines` | Max lines to keep from output |\n\n## Output Truncation\n\nTool output is automatically truncated to prevent context window overflow:\n\n- **Default limits**: 51200 bytes and 2000 lines\n- **Client overrides**: models can request `piMaxBytes`/`piMaxLines` (clamped to configured max)\n- **Temp files**: when truncated, full output is saved to a temp file with the path included in the response\n\n## Requirements\n\n- pi v0.74.0 or later\n- Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)\n\n## Uninstall\n\n```bash\npi remove npm:@benvargas/pi-firecrawl\n```\n\n## License\n\nMIT\n","readmeFilename":"README.md"}