{"_id":"@chcaa/twitter-academic-search","_rev":"4-3d9ff801e32b14042effe75667a19390","name":"@chcaa/twitter-academic-search","dist-tags":{"latest":"1.0.0"},"versions":{"1.0.0":{"name":"@chcaa/twitter-academic-search","version":"1.0.0","author":{"name":"Peter B. Vahlstrup"},"license":"ISC","_id":"@chcaa/twitter-academic-search@1.0.0","maintainers":[{"name":"jedglow","email":"jedrzej.glowaczewski@gmail.com"},{"name":"csiztom","email":"tom.csiz@gmail.com"},{"name":"madsvestergaard","email":"mrv@cas.au.dk"},{"name":"pbvahlst","email":"imvpbv@cc.au.dk"}],"homepage":"https://github.com/centre-for-humanities-computing/twitter-academic-search#readme","bugs":{"url":"https://github.com/centre-for-humanities-computing/twitter-academic-search/issues"},"bin":{"search":"bin/search.js","hydrate":"bin/hydrate.js","conversations":"bin/conversations.js","twitter-academic-search":"bin/search.js"},"dist":{"shasum":"78a54b5f36201fcc5847e157b9b5e876abd9d28d","tarball":"https://registry.npmjs.org/@chcaa/twitter-academic-search/-/twitter-academic-search-1.0.0.tgz","fileCount":13,"integrity":"sha512-ILhaV36c4sm3u12ORMERz4QnBxHCPWHD9x9nrw5ixzn16F6fUP0Y/GKJmvLjOQERrlDn2zZRyngpTgso1OX6yA==","signatures":[{"sig":"MEUCIQDDfWNdBtjhQvzWIAEXfrJAy1CRxWWyrVzr8pUi03+r9wIgB8P2WJxA2Lwp6Ox4qPBhdzYCI/s9I3DKyUE/sVSJGPM=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":55548,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v4.10.10\r\nComment: https://openpgpjs.org\r\n\r\nwsFzBAEBCAAGBQJkFBi0ACEJED1NWxICdlZqFiEECWMYAoorWMhJKdjhPU1b\r\nEgJ2Vmo0LRAAln3lQbB2DQHNPOpsDMy/R6oh1sZRdsHWvvKs5WK34Kh6ANRn\r\ncVVqaAGeFkbaQo47oDe7VMcwIuuebaPocbikOib7w+farxnEwvw95/AZ9L8P\r\nO85BogYxUhUkbgt6R363jlCK7bQlsZuWaShBpJTCF6vA4tNmU+VE5CuQQa4r\r\n8WJMtvOcR6KHOYLCyASwoxggLCYbsl0pC9IiTYkwyxt5BrHJDaN3/xkxa64Q\r\niHdOO0cXa0gU0tZIReYBROqRXcfDxTf3fnYzbuNhv7h8ZLp9Q5jBcajviqcg\r\n2BLUWy1o1XbKO8yEnKpxhm9GxFif+IWHDr7WexLAT36sKzHQttg4opNpBr8V\r\n5Hf04H6qUgaSqtf5dW+s3LooYUxCWbGNc9k4aTzjeLBdRBAYcB40nBbNV5p+\r\nU1p5TVCDvvdQpEIx+KcLT5a8LzKqCoVWUMiF5C7s8lJz3R9JKRmPmEjykyp6\r\nB2X914BqwZ4Aux2BhVhaNwrfhXrdL+Swd9Af4S2ECkvLBCfbMUcoc/NZajrQ\r\nEFAe7FMnZSpS8CAhBjwmtSKKnWPpK3ufvQIneGoRRJKNifYNvcVm9MZ+G6bU\r\nO8VJ28pXCob+3abriiFrMzYxvsKPY/D0c6DysdiRKnf/zjyKzmg+EIgy9W+R\r\nqsuNEiHtzRGNY1vX7p1uTu4wQEEjqvZ9qls=\r\n=9XXI\r\n-----END PGP SIGNATURE-----\r\n"},"gitHead":"42f366b0112e7d848327a26e559a64772eecea44","_npmUser":{"name":"pbvahlst","email":"imvpbv@cc.au.dk"},"repository":{"url":"git+https://github.com/centre-for-humanities-computing/twitter-academic-search.git","type":"git"},"_npmVersion":"9.3.1","description":"A tool for using the twitter v2 api","directories":{},"_nodeVersion":"18.15.0","dependencies":{"got":"^11.8.2","delay":"^5.0.0","luxon":"^1.26.0","lodash":"^4.17.21","commander":"^7.1.0","await-lock":"^2.1.0","oauth-1.0a":"^2.2.6","n-readlines":"^1.0.1","twitter-lite":"^1.1.0"},"_hasShrinkwrap":false,"_npmOperationalInternal":{"tmp":"tmp/twitter-academic-search_1.0.0_1679038644080_0.6583442253667182","host":"s3://npm-registry-packages"}}},"time":{"created":"2023-03-17T07:37:24.009Z","modified":"2025-02-06T13:30:03.743Z","1.0.0":"2023-03-17T07:37:24.331Z"},"bugs":{"url":"https://github.com/centre-for-humanities-computing/twitter-academic-search/issues"},"author":{"name":"Peter B. Vahlstrup"},"license":"ISC","homepage":"https://github.com/centre-for-humanities-computing/twitter-academic-search#readme","repository":{"url":"git+https://github.com/centre-for-humanities-computing/twitter-academic-search.git","type":"git"},"description":"A tool for using the twitter v2 api","maintainers":[{"email":"imvpbv@cc.au.dk","name":"pbvahlst"},{"email":"bfog@cc.au.dk","name":"donbjarkone"}],"readme":"# Twitter Academic Search\r\nA tool for using the twitter v2 api\r\n\r\n## Installation\r\n- Install [node.js](https://nodejs.org/en/download/) version 16.x or higher\r\n\r\n## Usage \r\nRun one of the following:\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest search -h \r\n```\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest conversations -h \r\n```\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest hydrate -h \r\n```\r\n\r\n## Local Usage\r\n- Clone this repository\r\n- Navigate to the root of the repository and run\r\n```\r\n$ npm install\r\n```\r\n\r\nYou can now the run files directly by navigating\r\nto the `.src/cli` of the repository and run one of the following \r\ndepending on what you want to do (will display the help manual).\r\n```\r\n$ node search -h \r\n```\r\n```\r\n$ node conversations -h \r\n```\r\n```\r\n$ node hydrate -h \r\n```\r\n### Search\r\nSearch the full twitter history using the twitter [query language](https://developer.twitter.com/en/docs/twitter-api/tweets/search/integrate/build-a-query).\r\n\r\n#### CLI options\r\n- **`-k, --api-credentials <apiCredentials>`** [required] - The twitter API bearerToken OR the consumerKey and consumerSecret as a string in the format KEY:SECRET OR a file containing one of the two\r\n- **`-q, --query <query>`** [required] - The twitter search query in the twitter [query language](https://developer.twitter.com/en/docs/twitter-api/tweets/search/integrate/build-a-query)\r\n- **`-d, --destination <directory>`** [required] - The directory where the result should be stored\r\n- **`-p, filename <string>`** [optional] - The name of the result file\r\n- **`-f, --from <date>`** [required] - The date to fetch data from in the format \"yyyy-mm-dd\" or a timestamp in ms\r\n- **`-t, --to <date>`** [required] - The date to fetch data to (included) in the format \"yyyy-mm-dd\" or a timestamp in ms. To fetch a single day this should be the same as \"from\" i using the \"yyyy-mm-dd\" format\r\n- **`-w, --resume-token <string>`** [optional] - Resume token used to resume if e.g. an error occurred or the fetch was aborted\r\n- **`-z, --development-mode`** [optional] - Should logging data be printed to the stdout\r\n\r\n#### Example\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest search -k KEY:SECRET -q \"war OR peace\" -d \"/data/twitter\" -p \"war_peace\" -f \"2020-10-10\" -t \"2021-01-01\"\r\n```\r\n\r\n### Conversations\r\nFetch full conversations based on a conversation id.\r\n\r\n#### CLI options\r\n- **`-k, --api-credentials <apiCredentials>`** [required] - The twitter API bearerToken OR the consumerKey and consumerSecret as a string in the format KEY:SECRET OR a file containing one of the two\r\n- **`-i, --ids <string|path>`** [required] - A comma separated list of ids (\"123,345,678\") or a path to a file with ids - one id pr. line\r\n- **`-d, --destination <directory>`** [required] - The directory where the result should be stored\r\n- **`-f, --from <date>`** [required] - The date to fetch data from in the format \"yyyy-mm-dd\" or a timestamp in ms\r\n- **`-t, --to <date>`** [required] - The date to fetch data to (included) in the format \"yyyy-mm-dd\" or a timestamp in ms. To fetch a single day this should be the same as \"from\" i using the \"yyyy-mm-dd\" format\r\n- **`-z, --development-mode`** [optional] - Should logging data be printed to the stdout\r\n\r\n#### Example\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest conversations -k KEY:SECRET -i \"1234,3444\" -d \"/data/twitter\" -f \"2020-10-10\" -t \"2021-01-01\"\r\n```\r\n\r\n### Hydrate\r\nHydrate a list of tweet id's to their full json-form.\r\n\r\n#### CLI options\r\n- **`-k, --api-credentials <apiCredentials>`** [required] - The twitter API bearerToken OR the consumerKey and consumerSecret as a string in the format KEY:SECRET OR a file containing one of the two\r\n- **`-i, --ids <string|path>`** [required] - A comma separated list of ids (\"123,345,678\") or a path to a file with ids - one id pr. line\r\n- **`-d, --destination <directory>`** [required] - The directory where the result should be stored\r\n- **`-p, --filename <string>`** [required] - The name of the result file\r\n- **`-z, --development-mode`** [optional] - Should logging data be printed to the stdout\r\n\r\n#### Example\r\n```\r\n$ npx -p @chcaa/twitter-academic-search@latest conversations -k KEY:SECRET -i \"1234,3444\" -d \"/data/twitter\" -p \"corona-tweets\"\r\n```\r\n\r\n#### Hint\r\nFor large sets of id's it is advisable to split the list into smaller chunks of e.g. 100.000 id's \r\npr. file or even smaller. Then if something goes wrong (the v2 of the API i sometimes unstable) it\r\nis easier to just start over instead figuring out where to resume from. Furtermore this approach makes\r\nit possible to use multiple access tokens for faster hydration - just start a fetch for every\r\naccess token available and divide the id's between them.\r\n","readmeFilename":"README.md"}