{"_id":"@achannarasappa/locust","_rev":"4-5e8717a02169a9133edf322b5d507bd1","name":"@achannarasappa/locust","dist-tags":{"latest":"1.3.0"},"versions":{"1.0.0":{"name":"@achannarasappa/locust","version":"1.0.0","description":"Distributed serverless web crawling/web scraping with support for js execution","main":"lib/index.js","directories":{"example":"example","lib":"lib"},"scripts":{"coverage":"nyc --reporter=text --reporter=lcov mocha --exit --grep '@server' --invert --timeout 10000 './lib/*.spec.js'","lint":"eslint ./lib","lint:fix":"eslint --fix ./lib","test":"mocha --exit --timeout 10000 './lib/*.spec.js'","test:watch":"npm run test -- --watch"},"author":{"name":"Ani Channarasappa"},"license":"MIT","devDependencies":{"chai":"^4.2.0","eslint":"^6.0.1","husky":"^2.7.0","mocha":"^5.2.0","nyc":"^14.1.1","puppeteer":"^2.0.0","sinon":"^7.2.3"},"dependencies":{"@achannarasappa/eslint-config":"^1.0.4","ioredis":"^4.3.0","joi":"^14.3.1","puppeteer-core":"^1.19.0","ramda":"^0.26.1"},"eslintConfig":{"extends":"@achannarasappa/eslint-config","env":{"mocha":true,"node":true},"rules":{"consistent-return":"off"}},"nyc":{"branches":100,"lines":100,"functions":100,"statements":100,"all":true,"check-coverage":true,"skip-full":true,"include":["lib/**/*.js"],"exclude":["lib/**/*.spec.js"]},"husky":{"hooks":{"pre-commit":"npm run lint"}},"gitHead":"2f245bc9c23d032e195af3d211f5a03d8939cae4","_id":"@achannarasappa/locust@1.0.0","_npmVersion":"6.4.1","_nodeVersion":"10.14.1","_npmUser":{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"},"dist":{"integrity":"sha512-qvBxewfiU7F2h9//1Q+Ug2vU2eDUh82EO1yVXfIAoGzdzrO2+Wp0HK3S6dWyYoVORz9DVy8LBbq/drJUWa67Fg==","shasum":"a9af3099f6b8189a2e7e222460865cc894b327b1","tarball":"https://registry.npmjs.org/@achannarasappa/locust/-/locust-1.0.0.tgz","fileCount":23,"unpackedSize":705344,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJd0FZ5CRA9TVsSAnZWagAAWVMP/i1g3AHQQTnPVqy7mD30\nTz8yR0usVqIKEszHSkJW2YufMsbq91enZ3oVqfMCnk6Ld2/pQPPRFWVWwGCr\nEDWGFiFlXzo2ad4KhzJC4fQJdmmyMnWy+wvSiP5bunMTqEoYSzwjr6tVqJh/\nJDaaJSqryg2lGKxDJ/YLLnV6HDuYdR252Wdevxh696G0vkpCN5q9F/itiPwA\nq24b490PQT1LEBCMhY7tBb8/VUblBQdLNoITsDF4XBcPibCpbP/OpRrvthwy\nOdFnpz1LIm7gmMV/t7Gj0aU6jTCB70HejjYKO5l3UB0HDu0yL5ztbymV0IjS\npkx0njr+j6gtwebbvdQh7mTzwJdaj3drC3t+twSSVn99KlxkNzmjcqGjLuDa\nk7IWgZ7K/IV5cyWtW+1ae6AONwx9J0+R18YACJbqWQH5UCspAFFieXqOIho8\nqVO0ubzmuPkue3onNmsINNLVTL4M2mKE8D/FIykubbDdoPY32nht0VpQlLEP\nckJzJ8HuymWHtunNUkJw9dPBnb5RDl6pXyvbSjM1nlQF/zcONn2QjPs1+c/Y\nTMV0gDFGa64gpaZAdlb+OE9LhpA2NM4GB3T0cOr8WkKjKE1vWlGPMaO9Hf9i\nsnZ8nHMC9nlHNHDPUCJ6LCix1yhIOrBEMqD8I1/JSQHCyUCRnICUhnLjtzUh\n4RMl\r\n=viVk\r\n-----END PGP SIGNATURE-----\r\n","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIFHlUY6eZJ3f647uxJiVp++Igw9WitB4YXxuZYKQtna9AiB9ClmQv3Z8yAzLV5ieh2f3jJrcqGuna+Cs1y9voKPg4g=="}]},"maintainers":[{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/locust_1.0.0_1573934713284_0.6889537303472673"},"_hasShrinkwrap":false},"1.1.0":{"name":"@achannarasappa/locust","version":"1.1.0","description":"Distributed serverless web crawling/web scraping with support for js execution","main":"lib/index.js","directories":{"example":"example","lib":"lib"},"scripts":{"coverage":"nyc --reporter=text --reporter=lcov mocha --exit --grep '@server' --invert --timeout 10000 './lib/*.spec.js'","coverage:coveralls":"npm run coverage && ./node_modules/.bin/nyc report --reporter=text-lcov | coveralls","lint":"eslint ./lib","lint:fix":"eslint --fix ./lib","test":"mocha --exit --timeout 10000 './lib/*.spec.js'","test:watch":"npm run test -- --watch"},"author":{"name":"Ani Channarasappa"},"license":"MIT","devDependencies":{"chai":"^4.2.0","coveralls":"^3.0.7","eslint":"^6.0.1","husky":"^2.7.0","mocha":"^5.2.0","nyc":"^14.1.1","puppeteer":"^2.0.0","sinon":"^7.2.3"},"dependencies":{"@achannarasappa/eslint-config":"^1.0.4","ioredis":"^4.3.0","joi":"^14.3.1","puppeteer-core":"^1.19.0","ramda":"^0.26.1"},"eslintConfig":{"extends":"@achannarasappa/eslint-config","env":{"mocha":true,"node":true},"rules":{"consistent-return":"off"}},"nyc":{"branches":100,"lines":100,"functions":100,"statements":100,"all":true,"check-coverage":true,"skip-full":true,"include":["lib/**/*.js"],"exclude":["lib/**/*.spec.js"]},"husky":{"hooks":{"pre-commit":"npm run lint"}},"gitHead":"298da6a4db4d55f2d5ef18ae0bffb98cc03c9215","_id":"@achannarasappa/locust@1.1.0","_npmVersion":"6.4.1","_nodeVersion":"10.14.1","_npmUser":{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"},"dist":{"integrity":"sha512-f/ZNPMhuRUEQvzzrSYMrP4i1d1cD6343YyUUu6yDYGuo0lhJDDUL+NP0cO2BC+EqT/w6pwtfTwlScS32DT7+Zw==","shasum":"b92c7f32076142c78f8f8f7a9391d84451da4f83","tarball":"https://registry.npmjs.org/@achannarasappa/locust/-/locust-1.1.0.tgz","fileCount":24,"unpackedSize":706159,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJd2XOfCRA9TVsSAnZWagAAC+YP/1Xwg0sytIaIrbqeCJPk\nNpSEGXzvoa5wePjZ2zTIj0WU2aGbWvf+Dlp2sIS/znte/xt1BmpSBfFgOrQE\nw1i4BhYv0Z+aFUQR8L+pNsnaXw906NOpZ+IQEDTEEpSgaOGCYX6ICYpkbGjq\nkqVZSMZu8U0PSOIfngrYJ4QSu9QuyTxsoAjWVlEzDLRpS75R/K/7o1OqkmmV\nIEGJvAWMR0jjcfE5GhgBUn4v/QP14io8WFqwgP7cBc/wWu/xet3R6F+eZw57\nKqQUahj2y65jfzfFbSVgGrBjaPeRQAMYBwcipuUH+Lg9DfExRBJavzVuZxpe\nZJn1Jgzcvd+pCdkUZtoY3veFvNgx9FeWbONWmLnqEu9pE4TiJtb13oOXlQZF\n81Oy/inEX2cpv11cnKzuYC7XFYiTFaHxXuPNL9VUdvwRBCO4Nfxb9LZZzkli\n7Jl5LlQgcf/PExg/k9K9sp1GfhIoWpAHWWs8LQ7XNETAeEkdLTd36MyMJodU\ncruAjPDuBe2TlZQdQ32yER+u20LoAGMa2/cUkfn8n6fd/LlgU/5vTWVQo89+\nM8lLRxotDPofgi2wJcT5Xu/dUFMuvcpmYZPhMNO/ktJ0gtoFGXK18OYS/Cld\nkUeYTI5gzQjkdiY4QHCORc+WHG2xf5HvCuAVKfvqgl9MSTuFjbQnUEyrXriB\n2WMj\r\n=L5Zc\r\n-----END PGP SIGNATURE-----\r\n","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIE86BL+h6tN4QkcbPG676MLAfLxjMXjZ7Qa/l1XxxB0rAiAe0ZSXo/sPZKMXk0uYZsQtU1MWyL7gJlNtuGWjWrwVMw=="}]},"maintainers":[{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/locust_1.1.0_1574531998787_0.8182683704558791"},"_hasShrinkwrap":false},"1.2.0":{"name":"@achannarasappa/locust","version":"1.2.0","description":"Distributed serverless web crawling/web scraping with support for js execution","main":"lib/index.js","repository":{"type":"git","url":"git+https://github.com/achannarasappa/locust.git"},"directories":{"lib":"lib"},"scripts":{"coverage":"nyc --reporter=text --reporter=lcov mocha --exit --grep '@server' --invert --timeout 10000 './lib/*.spec.js'","coverage:coveralls":"npm run coverage && ./node_modules/.bin/nyc report --reporter=text-lcov | coveralls","lint":"eslint ./lib","lint:fix":"eslint --fix ./lib","test":"mocha --exit --timeout 10000 './lib/*.spec.js'","test:watch":"npm run test -- --watch"},"author":{"name":"Ani Channarasappa"},"license":"MIT","devDependencies":{"chai":"^4.2.0","coveralls":"^3.0.9","eslint":"^6.8.0","husky":"^2.7.0","mocha":"^5.2.0","nyc":"^14.1.1","puppeteer":"^2.0.0","sinon":"^7.5.0"},"dependencies":{"@achannarasappa/eslint-config":"^1.0.4","ioredis":"^4.14.1","joi":"^14.3.1","puppeteer-core":"^2.0.0","puppeteer-extra":"^3.1.7","ramda":"^0.26.1"},"eslintConfig":{"extends":"@achannarasappa/eslint-config","env":{"mocha":true,"node":true},"rules":{"consistent-return":"off"}},"nyc":{"branches":100,"lines":100,"functions":100,"statements":100,"all":true,"check-coverage":true,"skip-full":true,"include":["lib/**/*.js"],"exclude":["lib/**/*.spec.js"]},"husky":{"hooks":{"pre-commit":"npm run lint && npm audit"}},"gitHead":"9e65b02163cb9ac32b73eb15e15bbc7d95072fc8","bugs":{"url":"https://github.com/achannarasappa/locust/issues"},"homepage":"https://github.com/achannarasappa/locust#readme","_id":"@achannarasappa/locust@1.2.0","_npmVersion":"6.4.1","_nodeVersion":"10.14.1","_npmUser":{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"},"dist":{"integrity":"sha512-BGkkGa8oxhwxXQ8IQHbD7AvsrCsjpbkeosalfiyPfal545B9aqJU8LxUd8RFqLR24LQf+MmK5RpOVL58HstP5Q==","shasum":"3ec8337986e8d97698c2eca40399146b0a66a626","tarball":"https://registry.npmjs.org/@achannarasappa/locust/-/locust-1.2.0.tgz","fileCount":24,"unpackedSize":709529,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJeBnm6CRA9TVsSAnZWagAAaVwP/2mEvnao0C6rcTWdKf4e\nT+ii4Dy7ONx5sNnUtmxb9PKj2eBJJu0QoeLwrnZDsxWRIzn6QV4WXQWZ8zwJ\n7pq1KrJV32UaWG8Of49pDNXZBf0Qi/xQJMtvNoPVcmbxZPxxzYbprPndVS2G\n3hX+IoH4pJZFJ4UNmqDBi03Ug1Jp/2mSWVcOsIZA9KUL+3VV9BYQjMmGji6j\nTMfcVUPthqdv4gCRhqPaenjZSCUCd7Z5ku0u5JNuI2WKojfJG2ES4yJ5qQaW\ng1/eCUzwODj5U3tZLs2A6n/W+irIrUOHGFPZkUJEwtuM1/ELmMfRw1Mun3FJ\nH5YtQ/kjPcBW+teM+XqV1KLiQ7wGPHc/J7mvQheeH0AMrtLMACLPV2ncY8Fb\nIei+7jM6Ky40vHSztfAvyzFVVMfF+nbvOAoC7ZrwN9bKYdAcduB+i0AC9TOG\nlumt14oihZ04hsOMAgcSdAR0CYI0CnpcHVWHZDT7vc5UW8TDS4I2dquCQuXj\n77FlplWoJpMyePeqzIG1knC9751RlX6NSyjPqfjGS/HWZ5cLuPnJEQmXP99a\nEElxGLeuTIwDBEAbXRGZHrWUv7BXtIqkM02fI3G6LDRmFfhIjyXMTZLm6oes\n5Zi/kZHzIq7xJNVDUZDSjwi6liFLOWLoARJ1A8bNs12kId4ackdiYOoJ6Xw6\ncHoI\r\n=Jftk\r\n-----END PGP SIGNATURE-----\r\n","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQD8KPO03sPljqRFh2BTTBBeWJ6JoxAp9264rfGJ6LoTvAIgDXA5j+sJTJ2TLhcWTFf5Je7fd3qTIkv0cFLHqyZyaSY="}]},"maintainers":[{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/locust_1.2.0_1577482681687_0.4178940702006766"},"_hasShrinkwrap":false},"1.3.0":{"name":"@achannarasappa/locust","version":"1.3.0","description":"Distributed serverless web crawling/web scraping with support for js execution","main":"lib/index.js","repository":{"type":"git","url":"git+https://github.com/achannarasappa/locust.git"},"directories":{"lib":"lib"},"scripts":{"coverage":"nyc --reporter=text --reporter=lcov mocha --exit --grep '@server' --invert --timeout 10000 './lib/*.spec.js'","coverage:coveralls":"npm run coverage && ./node_modules/.bin/nyc report --reporter=text-lcov | coveralls","lint":"eslint ./lib","lint:fix":"eslint --fix ./lib","test":"mocha --exit --timeout 10000 './lib/*.spec.js'","test:watch":"npm run test -- --watch"},"author":{"name":"Ani Channarasappa"},"license":"MIT","devDependencies":{"chai":"^4.2.0","coveralls":"^3.0.9","eslint":"^6.8.0","husky":"^2.7.0","mocha":"^7.2.0","nyc":"^14.1.1","puppeteer":"^2.0.0","sinon":"^7.5.0"},"dependencies":{"@achannarasappa/eslint-config":"^1.0.4","ioredis":"^4.14.1","joi":"^14.3.1","puppeteer-core":"^2.0.0","ramda":"^0.26.1","winston":"^3.2.1"},"eslintConfig":{"extends":"@achannarasappa/eslint-config","env":{"mocha":true,"node":true},"rules":{"consistent-return":"off"}},"nyc":{"branches":100,"lines":100,"functions":100,"statements":100,"all":true,"check-coverage":true,"skip-full":true,"include":["lib/**/*.js"],"exclude":["lib/**/*.spec.js"]},"husky":{"hooks":{"pre-commit":"npm run lint && npm audit"}},"gitHead":"3d90a10d31e873ffecc4d9ccbd16c6e19a89d109","bugs":{"url":"https://github.com/achannarasappa/locust/issues"},"homepage":"https://github.com/achannarasappa/locust#readme","_id":"@achannarasappa/locust@1.3.0","_nodeVersion":"12.17.0","_npmVersion":"6.14.4","dist":{"integrity":"sha512-Yv1AClKcBsaG5YbvDdHXqECOkPwT2By1hw7J+Xk6kQLc90/AIaZff2+067ehcsrs0FKe3BcyzA9mH9HujjnwSA==","shasum":"6e618bec7c7325c6ce3308d07796b6b722c3d8c8","tarball":"https://registry.npmjs.org/@achannarasappa/locust/-/locust-1.3.0.tgz","fileCount":26,"unpackedSize":729741,"npm-signature":"-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJe36D7CRA9TVsSAnZWagAAtLkP/3pt+7Q6uZWqVJJ4hdCO\ns9YCuZrpC7NUeEOsfzQ4LPmcC9zQyVbPSZyolO/Tb4Kt4qaTHt1A48nmyIVd\nzGkS7y8HGWTX0WYSjf+1M/4nJ/0i0tVdsYdiSk7xKw3kdVxHRwUzQr/SsYkQ\nmGuLA7NKiutN6MKnVEYZo4EUS4rFl09HNjEQzrjLcoOgzKEIh29ZYHAcnCfH\n+X6cuFpuXpBxyHf0XjAnGQBtmcWyPVsllj+5CYpXU+IddzdRY54I/6vwqAjZ\nRmDbOgh1BoY461wHtgdCxI8dzhIev8qZd29T5eS5zLVjwT53gPwmpA1NlFXz\nNMf+UCKAiqn2IsG111QtpTR5o66IvnNMb7+nOktSXnICXFL1cqNRls+OqMfx\nENPJugOUhKOxRGohRycxQPCXuFJR1j/24oiH7m9Hd8WpcTz1WL7kA10T26mn\nsQ5rFKNr0mRFDPtGY/+OktKlMYF3+g+POAXka0cw/ZlFZ0f7pAAAYf3l4Z/L\nYZQwTmwDwiDfhkvuev13vXJluTKMFlFL6V0Z3CIZb156X8F9z5euXTh2VXxC\nVuZtvl/g44w3UsMvD5UKmK+IarBOWwWOyFBqd7q6BU37f+ljZtuprQKLshj9\nKSAkyi/s9/uHZ4GYpfpyqGL83J8JPzyj4DJxe0g6M6CBsHwfapTD4RhlndBn\nmoKA\r\n=ZB7t\r\n-----END PGP SIGNATURE-----\r\n","signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEQCIDJhPdFRkKPDSGrTTaO9UQw2s3kdSkOm6pFNKVY233CcAiAZE/TAARFSyGFY8xXUYrMzCHEB6uYYB1f6tCxd74joOw=="}]},"maintainers":[{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"}],"_npmUser":{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"},"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/locust_1.3.0_1591714043469_0.7515833091963908"},"_hasShrinkwrap":false}},"time":{"created":"2019-11-16T20:05:13.284Z","1.0.0":"2019-11-16T20:05:13.441Z","modified":"2022-04-04T11:10:48.589Z","1.1.0":"2019-11-23T17:59:58.882Z","1.2.0":"2019-12-27T21:38:01.834Z","1.3.0":"2020-06-09T14:47:23.587Z"},"maintainers":[{"name":"achannarasappa","email":"ani.channarasappa@gmail.com"}],"description":"Distributed serverless web crawling/web scraping with support for js execution","author":{"name":"Ani Channarasappa"},"license":"MIT","readme":"[![Build Status](https://travis-ci.com/achannarasappa/locust.svg?branch=master)](https://travis-ci.com/achannarasappa/locust) [![Coverage Status](https://coveralls.io/repos/github/achannarasappa/locust/badge.svg?branch=master)](https://coveralls.io/github/achannarasappa/locust?branch=master)\n\n<p align=\"center\">\n<img align=\"center\" width=\"100\" src=\"https://locust.dev/img/locust-logo.svg\" />\n</p>\n<h1 align=\"center\">Locust</h2>\n<p align=\"center\">\nDistributed web data discovery and collection framework\n</p>\n\n## Quick Start\n\n```\nnpm install @achannarasappa/locust\n```\n\n## Features\n\n* Configuration driven jobs\n* Distributed execution model to support serverless architectures\n* Handle client-side JavaScript execution\n* Data extraction using CSS selectors\n* Depth-based stop condition along with support for custom stop conditions\n* Robust dev tooling with [locust-cli](https://github.com/achannarasappa/locust-cli) to build and test jobs\n\n## Use Cases\n\n* Web indexing (i.e. web crawling)\n* Web data extraction (i.e. web scraping)\n\n## Reference\n\n* Documentation\n  * [Quick start guide](https://locust.dev/docs/getting_started)\n  * [API](https://locust.dev/docs/api)\n  * [CLI](https://locust.dev/docs/cli)\n* [Examples](https://github.com/achannarasappa/locust-examples)\n* Related\n  * [locust-cli](https://github.com/achannarasappa/locust-cli)\n  * [locust-aws-terraform](https://github.com/achannarasappa/locust-aws-terraform)\n  * [locust-website](https://github.com/achannarasappa/locust-website)","readmeFilename":"README.md","homepage":"https://github.com/achannarasappa/locust#readme","repository":{"type":"git","url":"git+https://github.com/achannarasappa/locust.git"},"bugs":{"url":"https://github.com/achannarasappa/locust/issues"}}