{"_id":"spyder","_rev":"39-7f5f049569bdd88399e03e068e3d465b","name":"spyder","dist-tags":{"latest":"0.6.3"},"versions":{"0.1.0":{"name":"spyder","version":"0.1.0","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.1.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"43758133c114018c7d136f88e93a6550d3b65522","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.1.0.tgz","integrity":"sha512-mQkMU/kGKRf3Mng4k2pEzTi32aAneD/HjDlpOmNkhYa02CLD0JBDT2DwfMZXnsZdvzkxN4lrM/hvstEU73YzvQ==","signatures":[{"sig":"MEUCIH9+JbSNHvjymcbj1yzRWW/ucwmKhaR1CPRgLfDPKF90AiEArJwuhwPHsRyXCx8K+ZCnL8SvsCpux04mDpAVERvVGgQ=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","commander":"~2.1.0"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.2.0":{"name":"spyder","version":"0.2.0","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.2.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"9ec7a6e4f603072fe327b3430718e80ed0cb538e","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.2.0.tgz","integrity":"sha512-UjWnXISQw8zulQyEwS4M5kZ2EKNfJCN6ZmMfkHQ+nZvIkT0kQxX2l10WkaVz+iEmWQrLBvE4fVXEC27XH5eSOQ==","signatures":[{"sig":"MEYCIQCEwSLon8xMDo+qxlAqudwP6cS2wK5kcVrTJsJ5qPNoZgIhAMpIJiqduj4bWN+mYeuJbUFPMJjbWhyCnqIEuV78gcgV","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.3.0":{"name":"spyder","version":"0.3.0","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.3.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"e374e23d34803faaa8d1df136643e2b02c76c62c","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.3.0.tgz","integrity":"sha512-cCt4iLBPezzBQUx7jxZkEOVXgxBN1wLa3QXRh8VLKB4zBZCOYNxHGyT2oLWLAHWHhLLL4btRTNK5UdOjl1thNw==","signatures":[{"sig":"MEQCIB81B02eY2tj/mg2M0qCkHOb4V8HxAdRITaH+6UBtDZTAiBwiNZvD442JsOrtM2IUB5L+fekV8uxZQMCSmqLxSEqzg==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.3.1":{"name":"spyder","version":"0.3.1","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.3.1","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"92a402b4348b9405fa3f42833596b78925c12d4a","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.3.1.tgz","integrity":"sha512-d+EOGvtvnT+W6R283xExOQilPLBTpSKWM07FZsSEdF2cqOuXvRunKQ5xWkIVg1ySqjsoc5h7PlRlOYhNAGNCoA==","signatures":[{"sig":"MEQCIEPijlNRB4GNtw6DJ76ELln/ZR62Qpt+av9rkVfoSr3yAiAdplhUcmvZ042Do9tPmS6b7yedKMMxwZwIR4OPKGDbzg==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.3.2":{"name":"spyder","version":"0.3.2","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.3.2","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"0f1f41e0f2d2f1af03b8e1062918beb9b6b2b3c6","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.3.2.tgz","integrity":"sha512-Fv7if4UakNiwN4WhsMw2rwHthyLeHKI76Lm+VYZWfCAP11dzMMqjG29I9lYELt5ZBKsxsElI1DvVTb4oGYtzdg==","signatures":[{"sig":"MEUCIQC3RDOptVUgYGiMht7Gg4vkLoNYJl0NOL3NgI6j6t/xsAIgPGim2J1ZhkHhXBboQShtqlJ7b/Pq/R4bNmcUB6L6a9o=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.3.3":{"name":"spyder","version":"0.3.3","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.3.3","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"df96a6d7ecaece877ce19f3f2e44139517ec957f","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.3.3.tgz","integrity":"sha512-RZ6OdMJUE1yZ2mfXKr5UE+AxC6ArqtMoAFPdwcFzeXjUBamU2SC9FfeZYDLAyQCqcLTEB7LlHJAIV/TZq8uftQ==","signatures":[{"sig":"MEUCIBn3dvO0Nqgus4IkXOOV9SCV98MABOgHmKnqTzsVnGtoAiEAnis1e9KDoKcpaFA7LwmYXX74P7WtIZqe1NG7GeyZCRw=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.4.0":{"name":"spyder","version":"0.4.0","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.4.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"2600a1f0ce535296149607276b9739efab95fe2f","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.4.0.tgz","integrity":"sha512-GmgbZeJo210FR4R8WaKiaICYxli1PoY4/zzvVhStwUHN8Puzkze62BY47oBYlRwpFzr2bZaWOk0u/nIKnM4LUw==","signatures":[{"sig":"MEUCIQCNdmqL+I5usYvjETxN6mQdD0QWl/6CkZQ0Xz+CESXaBwIgAY5DS9E1PWcO+UYmE/htzsHWyDkj9J0MuyZ6t1jsRwo=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.4.1":{"name":"spyder","version":"0.4.1","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.4.1","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"cc73c3669bb223018c754edae15d79f78c86aafd","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.4.1.tgz","integrity":"sha512-BesIq9V5LVQQNBg0cj0yYtbsIETUN+RPDLqWRyfmOauC+BCm0YdS2+UnjOiia+ERVzf+UKuUgDbG74LWh9ZnaA==","signatures":[{"sig":"MEUCIDxmarxgKDILNw3gHFMX9RgfGtB10Jzck4MUQzS2LlRdAiEAtDXsYyeSlQH5EQlWDW0J5eFBoAaXYvWllI6oOF3DjxU=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.4.2":{"name":"spyder","version":"0.4.2","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.4.2","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"0124e29ea48dac4cba5bd74458a103192032d426","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.4.2.tgz","integrity":"sha512-HNnIR2esY+WH5SIYoEAe25SWXrrcx14mbBPQVpnzEGt/nfnj3/Ua4DnpC5HB1r2zT1YU/6SiWPlOs1lc9v3I9g==","signatures":[{"sig":"MEQCIHQjuLZrtcrExaBngJqpJ1dvLeHJK4LQM0E02IQLYjzQAiBS7n3Bp3KVa7yZlFd08aB/zNhHRk0O/i6W1JBfRpIUJA==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"~1.0.3","wait":"~0.1.0","async":"~0.2.10","annomath":"~0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"~0.13.1","request":"~2.33.0"}},"0.4.3":{"name":"spyder","version":"0.4.3","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.4.3","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"48f77ea03a723fb3c3bfe48b28bd4f626663bd10","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.4.3.tgz","integrity":"sha512-r+vSHBpHcicd9vzoshRGtru42KD9THmx2tht2mS/SJ7zIpp58d3rD5q5zQJEa/xFPCR5Ijrqw8y01A+p80rNRg==","signatures":[{"sig":"MEUCIHkKMXpro2xoAC62ZNDn2skHV3IP7QwYGX/902EGPHl6AiEAm/QL3HitvrtwXYF3iVnJPh0SGWcJazJuNtqB7RN0J9s=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.3.25","description":"Web indexer and scraper runner","directories":{},"dependencies":{"cron":"1.0.4","wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.5.0":{"name":"spyder","version":"0.5.0","keywords":[],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.5.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"88dbdb3b8b2ebed36cb4c427a66ae9ae1b83ecdb","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.5.0.tgz","integrity":"sha512-I32bgnW2nKoG1tY9TMcjwVjle5BWagWqeYsu3gdsvg0hvKlk6doyTscNv+8xWmG7sIb9oRkX/Rcv4CfWgEH6Lg==","signatures":[{"sig":"MEQCIGjPbPJH3lqTfV+pU5dRpE7MTekGxLnaSf1BGSy+b2XjAiB0iTcAq2HkXAG5PIwfbiJ8kdnYX3VltAyKs+BjZA9rkw==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.6","description":"Web indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.5.1":{"name":"spyder","version":"0.5.1","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.5.1","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./index.js"},"dist":{"shasum":"f8abb5059e0597b98f1e10fe2ab21ac17143ae69","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.5.1.tgz","integrity":"sha512-109jwKvMsZh4naDgxSgqvCepdMG6XEfQQhohbInbbUDbhlZAIIRg6kh/rphJp1/YJQeSFThLEXf3vxadpRY5Dw==","signatures":[{"sig":"MEYCIQDJ7pdtJyo5PaE9yLTrka96PmX4TYF6j5KiWt8N9nUaGAIhAI1upJHGuuRdaFytuYji5lDbDCUIorjtbXAUpO/6O0xk","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.6","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.5.2":{"name":"spyder","version":"0.5.2","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.5.2","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./cli.js"},"dist":{"shasum":"501debfb2966eb3773a1f4006ff6788139da4b7e","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.5.2.tgz","integrity":"sha512-sHwlbo8P8vXD7MxOwyO2n1tMgczy562or0IxQjrBDYG9rm+6eJrNnzMTSUlLPQQoQMPq5xYvqy5wpHFoT/cjGA==","signatures":[{"sig":"MEYCIQDRKBp4wToFBsR/N0lEYsQC4Ba5P8xI9T4Cdy9hpqcJ4gIhAJVRr4FFLUQ67WeAB2GRyhht1S7QxRjesb57gxiBtJi6","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"main":"./index.js","_from":".","_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.6","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.6.0":{"name":"spyder","version":"0.6.0","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.6.0","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./cli.js"},"dist":{"shasum":"8fa20dbbfc956e4c3c16d238c0d2bd57b702ea2e","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.6.0.tgz","integrity":"sha512-vGsBrl8atiKbo1JtiBM6pHmljEuCNl7xDbcxhX9gHriia48E3cYp1SBp2FjQns0AX9+Cyte9KX7hYFayeQrOqQ==","signatures":[{"sig":"MEQCIHXm881W2veOcsTlFmrd0XchJPST7r+G5kf6TlI9gxCqAiA3wGrzyDza3vdawskNzNn0uMTU3ajUnGeBM1l3RMW54Q==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"main":"./index.js","_from":".","_shasum":"8fa20dbbfc956e4c3c16d238c0d2bd57b702ea2e","gitHead":"64bf9109abc83b37aa5601d852d0da51bfabfe95","scripts":{},"_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.15","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.6.1":{"name":"spyder","version":"0.6.1","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.6.1","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./cli.js"},"dist":{"shasum":"9b6b36f807bb132ef097fb92f4ded0bc9644b5ea","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.6.1.tgz","integrity":"sha512-iEFKFbrmDiTLdCsbY8uKK3oNNofSJmyuY0jzLQbOJvDNSbFLt5d+3tLmoNGNNrTNAWE+NnBY4hkUq9a8gQygNA==","signatures":[{"sig":"MEYCIQC764jTGEYQWzMGfcPHWtnRcYx9oLOMV5zo4hmJdPsRFwIhAMGdD4rYWPzrWWvJmeTAscL1cw6oz6vMpz/esHNamEaG","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"main":"./index.js","_from":".","_shasum":"9b6b36f807bb132ef097fb92f4ded0bc9644b5ea","gitHead":"b68950fb9233de9b14d2c10a1dee7b2d0d4462f4","scripts":{},"_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.15","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.6.2":{"name":"spyder","version":"0.6.2","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.6.2","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./cli.js"},"dist":{"shasum":"25953238115f048c36ea9be8ef131fac68946fd1","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.6.2.tgz","integrity":"sha512-ICWImoijRzj93lueUSInL0lcogQZWmL24KoEwNFmWFhz27Gh4pkOR9oVXNj9Gf+3rEMY01EdjycfMmquCxvyEg==","signatures":[{"sig":"MEUCIDVbx7rmW2YUZOIkZLpuKiBXx5aELEVQnLxYoTDTO13aAiEAuzxGXEuGeFKnB8zQPsIVVLLnuRebQHkS1s6J/9ihbNQ=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"main":"./index.js","_from":".","_shasum":"25953238115f048c36ea9be8ef131fac68946fd1","gitHead":"10b7826a55e0c1c9fdf5a1dbd899c93bbd50916a","scripts":{},"_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"1.4.15","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}},"0.6.3":{"name":"spyder","version":"0.6.3","keywords":["scraping"],"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"_id":"spyder@0.6.3","maintainers":[{"name":"bebraw","email":"bebraw@gmail.com"}],"homepage":"https://github.com/bebraw/spyder","bugs":{"url":"https://github.com/bebraw/spyder/issues"},"bin":{"spyder":"./cli.js"},"dist":{"shasum":"b3f99bfa4b57d8e15ec814981a83cd9ecfba9603","tarball":"https://registry.npmjs.org/spyder/-/spyder-0.6.3.tgz","integrity":"sha512-70C50zVDHV/B3m2J0lKtJKSf7RwNCFZQWot4z47SfcFtqUEt1+wBPm4txdALsFHsexGkq7Xx6uPVFlYpHzXOvg==","signatures":[{"sig":"MEQCIDFlvG2Z8h5FJ7kuWsmnwBvYyW71KCNE0I7j5CbOyw/4AiBXH6aHLVPRCRAOeY6mbfiB3w6iNqIzoVgu0auOTyJ+qg==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}]},"main":"./index.js","_from":".","_shasum":"b3f99bfa4b57d8e15ec814981a83cd9ecfba9603","gitHead":"2696c6f72d8aecea134353c80f03edd765453233","scripts":{},"_npmUser":{"name":"bebraw","email":"bebraw@gmail.com"},"licenses":[{"url":"https://github.com/bebraw/spyder/blob/master/LICENSE","type":"MIT"}],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"_npmVersion":"2.0.0","description":"Indexer and scraper runner","directories":{},"dependencies":{"wait":"0.1.0","async":"0.2.10","annomath":"0.3.2","minimist":"0.0.7"},"devDependencies":{"cheerio":"0.14.0","request":"2.34.0"}}},"time":{"created":"2014-02-13T16:33:42.251Z","modified":"2026-06-02T21:58:55.603Z","0.1.0":"2014-02-13T16:33:42.251Z","0.2.0":"2014-02-20T15:52:14.251Z","0.3.0":"2014-02-24T16:53:50.881Z","0.3.1":"2014-02-24T17:04:59.363Z","0.3.2":"2014-02-25T17:08:22.368Z","0.3.3":"2014-02-28T14:59:44.769Z","0.4.0":"2014-03-02T18:21:27.193Z","0.4.1":"2014-03-03T16:33:05.016Z","0.4.2":"2014-03-24T12:56:40.635Z","0.4.3":"2014-04-03T13:20:26.025Z","0.5.0":"2014-05-04T16:19:19.228Z","0.5.1":"2014-05-06T06:07:08.087Z","0.5.2":"2014-05-19T10:01:00.070Z","0.6.0":"2014-06-18T05:16:00.764Z","0.6.1":"2014-08-01T06:10:41.325Z","0.6.2":"2014-08-01T06:26:51.798Z","0.6.3":"2014-10-05T16:02:05.999Z"},"bugs":{"url":"https://github.com/bebraw/spyder/issues"},"author":{"name":"Juho Vepsalainen","email":"bebraw@gmail.com"},"homepage":"https://github.com/bebraw/spyder","keywords":["scraping"],"repository":{"url":"https://github.com/bebraw/spyder.git","type":"git"},"description":"Indexer and scraper runner","maintainers":[{"email":"jlucasmelob558@gmail.com","name":"lucaaszsx"}],"readme":"[![build status](https://secure.travis-ci.org/bebraw/spyder.png)](http://travis-ci.org/bebraw/spyder)\n# spyder - Indexer and scraper runner\n\n`spyder` provides basic architecture for running indexers and scrapers. It comes as a CLI tool which you then provide configuration using either `-c` or `--config` parameter (`spyder -c ./config.js`).\n\nIt is also possible to provide a directory with `spyder_config.js` in it. In that case `spyder` tries to load the configuration automatically. Example: `spyder demo`.\n\nYou can also pass additional parameters to both commands. They will override the default configuration.\n\n## As a Module\n\n`spyder` can also be used as a regular Node module. It expects configuration as its parameter like this:\n\n```js\nvar spyder = require('spyder');\n\nspyder({...});\n```\n\n## config.js\n\nConsider the following `config.js` for basic configuration:\n\n```js\nmodule.exports = {\n    // workers\n    initializer: require('./init'), // optional\n    indexer: require('./indexer'),\n    scraper: require('./scraper'),\n\n    // events\n    onError: require('./error'),\n    onResult: require('./result'),\n    onFinish: require('./finish'),\n\n    // other\n    variance: 5000 // variance between scrape operations in ms\n};\n```\n\n## Workers\n\n`spyder` provides three workers into which you may attach actual functionality. `initializer` is executed once when `spyder` process is started. You may set auth keys and such there. `indexer` is run once per scraping round. `scraper` is executed per each url returned by `indexer`.\n\n### Initializer\n\n`initializer` is optional. A basic implementation could look like this:\n\n```js\nmodule.exports = function(o, cb) {\n    // do something with o now\n    // ...\n\n    cb(); // done\n};\n```\n\nThe first parameter will contain arguments passed to `spyder` process. This behavior is the same for all workers.\n\n### Indexer\n\nAn `indexer` could look like this:\n\n```js\nmodule.exports = function(o, cb) {\n    // index some page or pages here\n\n    // once finished, cb\n    cb(null, [\n        'http:// ...' // url to target to scrape\n    ]);\n};\n```\n\nRemember to return the urls you want to scrape here. In case you run into error, pass it as the first parameter to the callback.\n\n### Scraper\n\nA `scraper` could look like this:\n\n```js\nmodule.exports = function(o, url, cb) {\n    // scrape the content from url now\n\n    // once finished, cb\n    cb(null, {\n        title: 'Demo'\n    });\n};\n```\n\nThe same idea as earlier applies here. First the function receives arguments passed to spyder, then url to scrape and finally a callback to call when finished.\n\n## Events\n\nIn case an error is received, module defined at `onError` is defined. When a scraping result is received, `onResult` module is invoked. Once the whole process has finished, `onFinished` is invoked. Like above each handler receives arguments. You can for instance inject an object there at `initializer` and then use that to perform some operation. To give you an idea of what these files should look like, consider the following.\n\n`./error.js`:\n\n```js\nmodule.exports = function(o, err) {\n    // let's just log errors for now\n    // this is also the default behavior. if you don't provide a handler,\n    // spyder defaults to this\n    console.error(err);\n};\n```\n\n`./result.js`:\n\n```js\nmodule.exports = function(o, result, cb) {\n    // got some scraping result now, do something with it\n    // spyder defaults to console.log (handy during development)\n    console.log(result);\n\n    // the callback is optional and allows you to communicate possible errors\n    cb(new Error('Demo error'));\n};\n```\n\n`./finish.js`:\n\n```js\nmodule.exports = function(o) {\n    // spyder default\n    console.log('Finished');\n}\n```\n\n## Other\n\n* `variance` - Use `variance` to add arbitrary, random delay between scrape operations to make traffic look more irregular.\n\n## License\n\n`spyder` is available under MIT. See LICENSE for more details.\n","readmeFilename":"README.md"}