{"_id":"@apidojo/x-scraper","name":"@apidojo/x-scraper","dist-tags":{"latest":"1.1.0"},"versions":{"1.1.0":{"name":"@apidojo/x-scraper","version":"1.1.0","description":"The fastest and cheapest way to scrape tweets from X (Twitter). Wraps the Apify Twitter Scraper Lite actor with a developer-friendly API.","type":"module","main":"./dist/index.cjs","module":"./dist/index.mjs","types":"./dist/index.d.ts","exports":{".":{"import":{"types":"./dist/index.d.ts","default":"./dist/index.mjs"},"require":{"types":"./dist/index.d.ts","default":"./dist/index.cjs"}}},"bin":{"x-scraper":"src/cli.js"},"scripts":{"build":"node build.js && tsc","test":"vitest run","test:coverage":"vitest run --coverage","test:watch":"vitest","prepublishOnly":"npm run build"},"engines":{"node":">=18"},"keywords":["twitter","x","scraper","tweets","apify","twitter-scraper","x-scraper","social-media"],"license":"MIT","repository":{"type":"git","url":"git+https://github.com/apidojo-io/x-scraper.git"},"dependencies":{"apify-client":"^2","commander":"^14.0.3"},"devDependencies":{"@vitest/coverage-v8":"^3.2.4","esbuild":"^0.25.0","typescript":"^5.0.0","vitest":"^3.0.0"},"_id":"@apidojo/x-scraper@1.1.0","gitHead":"7b485a27e4049a718950daff12bfdc90f35596ad","bugs":{"url":"https://github.com/apidojo-io/x-scraper/issues"},"homepage":"https://github.com/apidojo-io/x-scraper#readme","_nodeVersion":"20.20.1","_npmVersion":"10.8.2","dist":{"integrity":"sha512-sV4SwsVJsFnA5KraYZpFRdq9hRMdmvqLNFccvunag850bS8gcdwquzA+p7HHgTYT0WqZ9dX6Z7FHXEfuukkbTQ==","shasum":"b9fd7f6112268261105d3af1009d5b524d38fdba","tarball":"https://registry.npmjs.org/@apidojo/x-scraper/-/x-scraper-1.1.0.tgz","fileCount":14,"unpackedSize":141460,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@apidojo%2fx-scraper@1.1.0","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQD3l7GdpERF8gTLmCNePIKaeEiOnn0l80pC4DfcVjvPJAIhAP1AQYENu4vU2LlK1Kgnd4gC1Q1EY5n7JlYbP0M5CAiW"}]},"_npmUser":{"name":"apidojo","email":"apidojo@hotmail.com"},"directories":{},"maintainers":[{"name":"apidojo","email":"apidojo@hotmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/x-scraper_1.1.0_1773478342248_0.5082138085689285"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-14T08:52:22.179Z","1.1.0":"2026-03-14T08:52:22.398Z","modified":"2026-03-14T08:52:22.746Z"},"maintainers":[{"name":"apidojo","email":"apidojo@hotmail.com"}],"description":"The fastest and cheapest way to scrape tweets from X (Twitter). Wraps the Apify Twitter Scraper Lite actor with a developer-friendly API.","homepage":"https://github.com/apidojo-io/x-scraper#readme","keywords":["twitter","x","scraper","tweets","apify","twitter-scraper","x-scraper","social-media"],"repository":{"type":"git","url":"git+https://github.com/apidojo-io/x-scraper.git"},"bugs":{"url":"https://github.com/apidojo-io/x-scraper/issues"},"license":"MIT","readme":"# x-scraper\n\nThe fastest and cheapest way to scrape tweets from X (Twitter). Battle-tested infrastructure used by tens of thousands of customers including enterprise teams.\n\nWraps the [Apify Twitter Scraper Lite](https://apify.com/apidojo/twitter-scraper-lite?fpr=yhdrb) actor with a developer-friendly, class-based API. Every method creates an Apify actor run under the hood.\n\n## Prerequisites\n\n- **Node.js 16+**\n- An **Apify account on a paid plan** — [sign up here](https://apify.com/?fpr=yhdrb)\n- An **Apify API token** — [get it here](https://console.apify.com/account/integrations)\n\n## Installation\n\n```bash\nnpm install @apidojo/x-scraper\n```\n\n## CLI Usage\n\nInstall globally to use from the command line:\n\n```bash\nnpm install -g @apidojo/x-scraper\n```\n\n### Setup\n\n```bash\n# Store your Apify token (saved to ~/.x-scraper/config.json)\nx-scraper init\n\n# Or pass it per-command\nx-scraper profile NASA --token apify_api_xxx\n\n# Or set as environment variable\nexport APIFY_TOKEN=apify_api_xxx\n```\n\n### Commands\n\n```bash\n# Fetch tweets from a profile\nx-scraper profile NASA --sort Latest --max-items 50\n\n# Search by keyword\nx-scraper keyword \"artificial intelligence\" --lang en --sort Latest\n\n# Search by multiple keywords (joined with OR)\nx-scraper keywords bitcoin ethereum solana\n\n# Search by hashtag\nx-scraper hashtag AI MachineLearning --sort Latest\n\n# Fetch a single tweet by URL\nx-scraper url https://x.com/elonmusk/status/1728108619189874825\n\n# Fetch from multiple URLs\nx-scraper urls https://x.com/a/status/1 https://x.com/b/status/2\n\n# Tweets from a handle within a date range\nx-scraper profile-date-range NASA --since 2024-01-01 --until 2024-06-01\n\n# Tweets near a location\nx-scraper location coffee --location \"San Francisco\" --radius 10mi\n\n# Tweets with minimum engagement\nx-scraper engagement bitcoin --min-likes 1000 --min-retweets 100\n\n# Multiple profiles in one run\nx-scraper profiles elonmusk naval paulg\n\n# Mentions, media filters, cashtags, verified users, and more\nx-scraper mention NASA\nx-scraper media NASA\nx-scraper images NASA\nx-scraper videos NASA\nx-scraper cashtag BTC ETH SOL\nx-scraper verified cryptocurrency\nx-scraper no-retweets elonmusk\nx-scraper links TechCrunch\nx-scraper conversation 1728108619189874825\n\n# Raw actor input (full control)\nx-scraper search --input '{\"searchTerms\":[\"from:NASA filter:media\"]}'\nx-scraper execute --input '{\"searchTerms\":[\"test\"],\"maxItems\":10}'\n```\n\n### Output Options\n\n```bash\n# Pretty JSON (default)\nx-scraper profile NASA\n\n# Compact JSON (pipe-friendly)\nx-scraper profile NASA --json\n\n# JSONL (one object per line)\nx-scraper profile NASA --jsonl\n\n# Save to file\nx-scraper profile NASA --output tweets.json\n\n# Debug mode (shows internal logs)\nx-scraper profile NASA --debug\n```\n\n### Global Flags\n\n| Flag | Description |\n|---|---|\n| `--token <token>` | Apify API token (overrides env/config) |\n| `--sort <order>` | `Latest`, `Top`, or `\"Latest + Top\"` |\n| `--max-items <n>` | Maximum number of items |\n| `--lang <code>` | Language filter (ISO 639-1) |\n| `--timeout <secs>` | Run timeout in seconds |\n| `--output <file>` | Save output to file |\n| `--json` | Compact JSON output |\n| `--jsonl` | JSONL output (one JSON per line) |\n| `--debug` | Enable debug mode |\n\n## Quick Start (Library)\n\n```javascript\n// ESM\nimport { XScraper } from '@apidojo/x-scraper';\n\n// CommonJS\nconst { XScraper } = require('@apidojo/x-scraper');\n\nconst scraper = new XScraper({ token: 'apify_api_xxxxxxxxxxxx' });\n\n// Fetch latest tweets from NASA\nconst tweets = await scraper.getTweetsByProfile('NASA', { sort: 'Latest', maxItems: 50 });\nconsole.log(tweets);\n```\n\n## Core Methods\n\n### `execute(input, options?)`\n\nRaw passthrough — pass any valid actor input object directly. Use this when you know exactly what input the actor expects.\n\n```javascript\nconst tweets = await scraper.execute({\n  searchTerms: ['from:NASA filter:media'],\n  sort: 'Latest',\n  maxItems: 100,\n});\n```\n\n### `search(input, options?)`\n\nRun the scraper synchronously — waits for the run to finish and returns all items. Options like `sort`, `maxItems`, and `lang` are merged into the input automatically.\n\n```javascript\nconst tweets = await scraper.search(\n  { searchTerms: ['from:NASA'] },\n  { sort: 'Latest', maxItems: 50, lang: 'en' }\n);\n```\n\n### `searchAsync(input, options?)`\n\nStart a run without waiting. Returns a `RunHandle` for polling and fetching results later. Use this for large runs that may exceed the sync timeout.\n\n```javascript\nconst run = await scraper.searchAsync(\n  { searchTerms: ['from:NASA'] },\n  { sort: 'Latest' }\n);\n\nawait run.waitForFinish();\nconst tweets = await run.getItems();\n```\n\n### `stream(input, options?)`\n\nRun the scraper and stream results as an async iterator. Items are yielded one by one, with auto-pagination.\n\n```javascript\nfor await (const tweet of scraper.stream({ searchTerms: ['from:NASA'] })) {\n  console.log(tweet.text);\n}\n```\n\n## Convenience Methods\n\nAll convenience methods accept an `options` object:\n\n| Option | Type | Description |\n|---|---|---|\n| `sort` | `string` | `'Latest'`, `'Top'`, or `'Latest + Top'` |\n| `maxItems` | `number` | Maximum number of items to return |\n| `lang` | `string` | ISO 639-1 language code (e.g. `'en'`) |\n| `timeout` | `number` | Timeout in seconds (default: 120) |\n| `fullResponse` | `boolean` | Return `{ items, runId, datasetId }` instead of just items |\n\n### Profiles\n\n```javascript\n// Tweets from a single profile\nconst tweets = await scraper.getTweetsByProfile('NASA', { sort: 'Latest' });\n\n// Tweets from multiple profiles in one run\nconst tweets = await scraper.getTweetsByMultipleProfiles(\n  ['elonmusk', 'naval', 'paulg'],\n  { sort: 'Latest' }\n);\n\n// Tweets from a handle within a date range\nconst tweets = await scraper.getTweetsByHandleInDateRange(\n  'NASA', '2024-01-01', '2024-06-01',\n  { sort: 'Latest' }\n);\n\n// Exclude retweets from a handle\nconst tweets = await scraper.getTweetsExcludingRetweetsByHandle('elonmusk', {\n  sort: 'Latest',\n});\n```\n\n### Search\n\n```javascript\n// Search by keyword\nconst tweets = await scraper.getTweetsByKeyword('artificial intelligence', {\n  lang: 'en',\n  sort: 'Latest',\n});\n\n// searchTweets is an alias for getTweetsByKeyword\nconst tweets = await scraper.searchTweets('artificial intelligence');\n\n// Search by multiple keywords (joined with OR)\nconst tweets = await scraper.getTweetsByKeywords(\n  ['bitcoin', 'ethereum', 'solana'],\n  { lang: 'en', sort: 'Latest' }\n);\n\n// searchTweetsByMultipleKeywords is an alias for getTweetsByKeywords\nconst tweets = await scraper.searchTweetsByMultipleKeywords(['AI', 'ML', 'deep learning']);\n\n// Search by hashtag\nconst tweets = await scraper.getTweetsByHashtag(['AI', 'MachineLearning'], {\n  sort: 'Latest',\n});\n\n// Search by cashtag\nconst tweets = await scraper.getTweetsByCashtag(['BTC', 'ETH', 'SOL'], {\n  lang: 'en',\n  sort: 'Latest',\n});\n```\n\n### Single Tweet & URLs\n\n```javascript\n// Fetch a single tweet by URL\nconst tweets = await scraper.getTweetByUrl(\n  'https://x.com/elonmusk/status/1728108619189874825'\n);\n\n// Fetch from multiple URLs (tweets, lists, profiles)\nconst tweets = await scraper.getTweetsByUrls([\n  'https://x.com/elonmusk/status/1728108619189874825',\n  'https://twitter.com/i/lists/1234567890',\n]);\n```\n\n### Conversations\n\n```javascript\n// Fetch replies in a conversation thread\nconst replies = await scraper.getTweetsByConversationId('1728108619189874825', {\n  sort: 'Latest',\n});\n```\n\n### Mentions\n\n```javascript\n// Tweets mentioning a user\nconst tweets = await scraper.getTweetsByMention('NASA', { sort: 'Latest' });\n```\n\n### Media Filters\n\n```javascript\n// Tweets with any media (images or videos)\nconst tweets = await scraper.getTweetsWithMediaByHandle('NASA');\n\n// Tweets with images only\nconst tweets = await scraper.getTweetsWithImagesByHandle('NASA');\n\n// Tweets with videos only\nconst tweets = await scraper.getTweetsWithVideosByHandle('NASA');\n\n// Tweets with links\nconst tweets = await scraper.getTweetsWithLinksByHandle('TechCrunch');\n```\n\n### Engagement & Filters\n\n```javascript\n// Tweets with minimum engagement\nconst tweets = await scraper.getTweetsWithMinEngagement('bitcoin', {\n  minLikes: 1000,\n  minRetweets: 100,\n  sort: 'Top',\n});\n\n// Tweets from verified users only\nconst tweets = await scraper.getTweetsByVerifiedUsers('cryptocurrency', {\n  sort: 'Top',\n});\n```\n\n### Location\n\n```javascript\n// Tweets near a location\nconst tweets = await scraper.getTweetsByLocation('coffee', 'San Francisco', '10mi', {\n  sort: 'Latest',\n});\n```\n\n## Full Response Mode\n\nBy default, methods return just the items array. Pass `{ fullResponse: true }` to get run metadata:\n\n```javascript\nconst result = await scraper.getTweetsByProfile('NASA', {\n  sort: 'Latest',\n  fullResponse: true,\n});\n\nconsole.log(result.items);     // Array of tweet objects\nconsole.log(result.runId);     // Apify run ID\nconsole.log(result.datasetId); // Apify dataset ID\n```\n\n## Async Runs with RunHandle\n\nFor large runs that may take longer than the sync timeout:\n\n```javascript\nconst run = await scraper.searchAsync({ searchTerms: ['from:NASA'], sort: 'Latest' });\n\n// Poll until the run finishes\nawait run.waitForFinish();\n\n// Get all items\nconst tweets = await run.getItems();\n\n// Or stream items\nfor await (const tweet of run.stream()) {\n  console.log(tweet.text);\n}\n\n// Access run metadata\nconsole.log(run.runId);\nconsole.log(run.datasetId);\nconsole.log(run.status);\n```\n\n## Error Handling\n\nAll errors from Apify are thrown directly. Wrap calls in try/catch:\n\n```javascript\ntry {\n  const tweets = await scraper.getTweetsByProfile('NASA');\n} catch (error) {\n  // Apify errors: 401 unauthorized, 404 not found, 429 rate limit, run failures\n  console.error('Scraper error:', error.message);\n}\n```\n\nThe constructor throws immediately if no token is provided:\n\n```javascript\ntry {\n  const scraper = new XScraper({});\n} catch (error) {\n  // \"APIFY_TOKEN is required. Pass it as: new XScraper({ token: \"apify_api_xxx\" })\"\n}\n```\n\n## Tweet Object Shape\n\nEach item returned is a tweet object:\n\n```json\n{\n  \"type\": \"tweet\",\n  \"id\": \"1728108619189874825\",\n  \"url\": \"https://x.com/elonmusk/status/1728108619189874825\",\n  \"text\": \"More than 10 per human on average\",\n  \"retweetCount\": 11311,\n  \"replyCount\": 6526,\n  \"likeCount\": 104121,\n  \"quoteCount\": 2915,\n  \"createdAt\": \"Fri Nov 24 17:49:36 +0000 2023\",\n  \"lang\": \"en\",\n  \"isReply\": false,\n  \"isRetweet\": false,\n  \"isQuote\": true,\n  \"author\": {\n    \"userName\": \"elonmusk\",\n    \"name\": \"Elon Musk\",\n    \"id\": \"44196397\",\n    \"followers\": 172669889,\n    \"isVerified\": true,\n    \"isBlueVerified\": true\n  }\n}\n```\n\n## API Reference\n\n### Constructor\n\n| Parameter | Type | Default | Description |\n|---|---|---|---|\n| `token` | `string` | *required* | Apify API token |\n| `timeout` | `number` | `120` | Default timeout in seconds for sync runs |\n\n### Core Methods\n\n| Method | Returns | Description |\n|---|---|---|\n| `execute(input, options?)` | `Promise<items\\|FullResponse>` | Raw input passthrough |\n| `search(input, options?)` | `Promise<items\\|FullResponse>` | Sync run with option merging |\n| `searchAsync(input, options?)` | `Promise<RunHandle>` | Async run, returns handle |\n| `stream(input, options?)` | `AsyncGenerator<tweet>` | Async iterator over items |\n\n### Convenience Methods\n\n| Method | Parameters | Builds Query |\n|---|---|---|\n| `getTweetsByProfile` | `(handle, opts?)` | `from:{handle}` |\n| `getTweetsByHandleInDateRange` | `(handle, since, until, opts?)` | `from:{handle} since:... until:...` |\n| `getTweetsByHashtag` | `(hashtags, opts?)` | `#tag1 #tag2` |\n| `getTweetsByKeyword` | `(keyword, opts?)` | `{keyword}` |\n| `searchTweets` | `(keyword, opts?)` | Alias for `getTweetsByKeyword` |\n| `getTweetsByKeywords` | `(keywords[], opts?)` | `kw1 OR kw2 OR kw3` |\n| `searchTweetsByMultipleKeywords` | `(keywords[], opts?)` | Alias for `getTweetsByKeywords` |\n| `getTweetsByConversationId` | `(id, opts?)` | `conversation_id:{id}` |\n| `getTweetByUrl` | `(url, opts?)` | `startUrls: [url]` |\n| `getTweetsByUrls` | `(urls[], opts?)` | `startUrls: [...urls]` |\n| `getTweetsByLocation` | `(query, location, radius, opts?)` | `{q} near:\"{loc}\" within:{r}` |\n| `getTweetsByMultipleProfiles` | `(handles[], opts?)` | `[\"from:a\",\"from:b\"]` |\n| `getTweetsByCashtag` | `(cashtags, opts?)` | `$X OR $Y` |\n| `getTweetsByMention` | `(handle, opts?)` | `@{handle}` |\n| `getTweetsWithMediaByHandle` | `(handle, opts?)` | `from:{handle} filter:media` |\n| `getTweetsWithImagesByHandle` | `(handle, opts?)` | `from:{handle} filter:images` |\n| `getTweetsWithVideosByHandle` | `(handle, opts?)` | `from:{handle} filter:videos` |\n| `getTweetsWithMinEngagement` | `(query, opts?)` | `{q} min_faves:{n} min_retweets:{n}` |\n| `getTweetsByVerifiedUsers` | `(query, opts?)` | `{q} filter:verified` |\n| `getTweetsExcludingRetweetsByHandle` | `(handle, opts?)` | `from:{handle} -filter:retweets` |\n| `getTweetsWithLinksByHandle` | `(handle, opts?)` | `from:{handle} filter:links` |\n\n### RunHandle\n\n| Property/Method | Type | Description |\n|---|---|---|\n| `runId` | `string` | The Apify run ID |\n| `datasetId` | `string` | The default dataset ID |\n| `status` | `string` | Last known run status |\n| `waitForFinish(opts?)` | `Promise<RunHandle>` | Poll until terminal status |\n| `getItems(opts?)` | `Promise<items\\|FullResponse>` | Fetch all dataset items |\n| `stream()` | `AsyncGenerator<tweet>` | Async iterator over items |\n\n## Pricing\n\nThis package uses the [Apify Twitter Scraper Lite](https://apify.com/apidojo/twitter-scraper-lite) actor with event-based pricing. You only pay for what you use. See the [actor pricing page](https://apify.com/apidojo/twitter-scraper-lite#pricing) for current rates.\n\n## License\n\nMIT\n","readmeFilename":"README.md","_rev":"1-47550f88a4c536d83dc7f754468865dc"}