{"_id":"llm-scraper","_rev":"19-d959c2a9ada9301a1f02674db1cc925b","name":"llm-scraper","dist-tags":{"latest":"2.0.0"},"versions":{"1.0.0":{"name":"llm-scraper","version":"1.0.0","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.0","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"9a6374ba566b09a66ba83da613264bf219d7ecc2","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.0.tgz","fileCount":8,"integrity":"sha512-jqwy/yu9xZy8Uzkb3x91DyE3FHXBvHT2bYTWrn7ntUeEyIxPg0IeQdCPwMLsaA+hh6wbkrujObLT5bXDWx7NMg==","signatures":[{"sig":"MEUCIQDAyRhBscpoUTCSDSi9OKEEPCcPXUf4KEHvUGsIU5HQlgIgIwXfLycy9JVgCQ5hSQG5RY5pQfDViH0ry8UaBCp9UEQ=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":10796},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"09dd33b9ca5a2b20c70b3ab58ec55cec016cb966","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.0_1713632749905_0.6514486609252765","host":"s3://npm-registry-packages"}},"1.0.1":{"name":"llm-scraper","version":"1.0.1","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.1","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"1206db3675f89a532fb313f6b3fd7c51e219ffee","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.1.tgz","fileCount":9,"integrity":"sha512-Vq1iSA3PMEVsNhtO5WR5EC+93RtAusCmWAD0TEmIegADrzyu6yn4cUdR32t+kljd1XIgAIZdZkCFwuoN6yWZKg==","signatures":[{"sig":"MEUCIQDExRR5USpk0OWwaFrEDV1VcLSKs6lNiBcAnBekU1qA8gIgTmLUEnGyEYfuUeNM5kMJaNNSs24I5hT5+NB/Y4o72U8=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":13162},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"9cfd78023b2d5b7d55dce7cf16b47d61e204962f","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.1_1713644729685_0.7569205660912768","host":"s3://npm-registry-packages"}},"1.0.2":{"name":"llm-scraper","version":"1.0.2","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.2","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"e39b53419b3080c502547abef79a64c04c91b884","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.2.tgz","fileCount":10,"integrity":"sha512-4qnOZYCkSrjBECemhOUAq79TaHgWcEk++2TqB7R3wT3/d7E3Sc+/XX5ObX4LkO0ep1in9/DFaZajpPmh3KOP3Q==","signatures":[{"sig":"MEYCIQCF0ARplfq/Zk5eFyudXDb9kUwpgUOJynnHpLzwQQysdgIhALwdWg2eCCxU3O1n+MU2erpbcPDsjjzw2+mzrC8ki9zj","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":16001},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"23d0c2af9d68d731b872076ea57c3040ab4e1872","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","turndown":"^7.1.3","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.2_1713698062838_0.5938993229090697","host":"s3://npm-registry-packages"}},"1.0.3":{"name":"llm-scraper","version":"1.0.3","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.3","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"3caf62286dccbdfedfca198309606d90359ca87d","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.3.tgz","fileCount":10,"integrity":"sha512-5HBpxt+4oJzEbs2qtOj+WTRuIrdfZ4WfVrTFokEVT24AIvpXIQ2NYoixyWqgXl2hSi5Dy/lSndMW7pX7Fz0neQ==","signatures":[{"sig":"MEUCIQCurY636A9/SWAtyya5kApQ2rvRBaZKI4O9jijhLAnLrgIgcNADOb97vd3T0h5k4HPL6D4Gj11aEnN95Nk//jc4VKA=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":16302},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"e0fd1e70f00cb665a85043c7eb3b829206f6333b","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","turndown":"^7.1.3","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.3_1713703564005_0.5990112496656528","host":"s3://npm-registry-packages"}},"1.0.4":{"name":"llm-scraper","version":"1.0.4","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.4","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"3f73f36f863e0efa1073be1166066890fb5665b8","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.4.tgz","fileCount":14,"integrity":"sha512-+WHpPF0RhhsX6apjOqJeNxBTaW2+jvFqww2FIFu2BQOlACO9wPXxaBg7cs850qkjdWYsC2szzXhZA05wfBVCkg==","signatures":[{"sig":"MEUCIQCPtU1rWSLHFaXuMi+QYxbMBFafCCZY9t4/IRMa93vuYAIgA0feUoSEZh8U8eJ7IVWFBLC31wHMFJAusMudQ1tggW8=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":20529},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"6d712a99064adfa741340c5628b75a40fef2c70f","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.4_1713714735212_0.9703163892139752","host":"s3://npm-registry-packages"}},"1.0.5":{"name":"llm-scraper","version":"1.0.5","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.5","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"cb400ffb6c502698973e109470e5f6e6be5c72b2","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.5.tgz","fileCount":14,"integrity":"sha512-DLwuiYIhseuIo7XMM2SU0aoEQWqirIQ34SXZFhOosmFG504yhgEWEo+EXV10pqs8Ofbj/DXdQUySElKQlLuK1Q==","signatures":[{"sig":"MEUCIDQZjzIbtX3aDvgf2q3987kBVjP7Au2drNK5xjNudmWQAiEAoe0UO1n25ee5dXq5cNQd5U3rZthywrzkOg1Csrd4RJQ=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":21014},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"f7377480df899411f7014776df4a580edebd0b9b","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"openai":"^4.38.2","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.5_1713733923921_0.8238925754089368","host":"s3://npm-registry-packages"}},"1.0.6":{"name":"llm-scraper","version":"1.0.6","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.0.6","homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"fde561f2e4c6b46110b90d650a3211350dd66d26","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.0.6.tgz","fileCount":14,"integrity":"sha512-KeqFeYTjrIzLFZCoR5JsuwXTx0isRqrVSAepeuTCkMQwHcSX/dioPsZRdVkqqi0QctQxa4GpR3tHMy8F3b0pug==","signatures":[{"sig":"MEQCIAg7Rsd6OttACLl2wI/sryJ0tjAvfKKK1Wa4Uq3hANZGAiB5Taxt2htK+j7iwyCRz0y45KAQmfYXRDuj4na5Mctiog==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":20288},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"4d0a5d4548ab7930fe85f17e3647ac7b02f72a5a","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.0.6_1719062200233_0.5207168303450889","host":"s3://npm-registry-packages"}},"1.1.0":{"name":"llm-scraper","version":"1.1.0","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.1.0","homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"2a1c11b4044637d26fbbef935308bf949a445edc","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.1.0.tgz","fileCount":13,"integrity":"sha512-nqn/7I6+NGhYEC/q49j6QaF0S811xXSkbswDBU8tXxa0hHySZqEW/alhQjiZRkVMsY2HWb6XZJW2mMPAi4CDdw==","signatures":[{"sig":"MEQCIHjMDTzCo6lFVuEHdBhQ8hgT0A8qDibMq9WucN0xnonRAiA+XVr+T4lSWrAl7S6ysCw+7P0hEmF6WDvbPQzVvC7B0Q==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":17421},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"f5499de23ea0c8e21f29285fabe0d2a1347d311c","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.1.0_1719063382183_0.7536817284436867","host":"s3://npm-registry-packages"}},"1.1.1":{"name":"llm-scraper","version":"1.1.1","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.1.1","homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"6831436a63d7aa23f17652cc2492d35fa4b49606","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.1.1.tgz","fileCount":13,"integrity":"sha512-RBgRNmYqYqgiacUoWbWuyOJEsTR6+bY1owYjqehvcwYOfrS1RWa5gBwjqAgzNFhEpinSvsTkl5BgdPNUG2kF5Q==","signatures":[{"sig":"MEUCIC3jGQ02aS5hxMS2GlRw0jOyB5CpAGCHtWbBptG0Tu0MAiEAhAjPAj2OFmYZ1g+IPfv+xEH1jcPvUzhEccq82IFTE/g=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":17665},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"12f9ddccaa9c458d51fff9dc33ddfa8487e37c59","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.1.1_1719064321183_0.4565103982248102","host":"s3://npm-registry-packages"}},"1.1.2":{"name":"llm-scraper","version":"1.1.2","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.1.2","homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"9f73ba30ffef5481091169787505adf5919b803c","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.1.2.tgz","fileCount":14,"integrity":"sha512-jCVm6Yka4LpyFENSs/YBuJDZ4cRhY8XpN/0YcE4/wsQbpKAL7KsO2MJojGyScjaYteo9uDJIHEkNGMLJUYTZfw==","signatures":[{"sig":"MEUCIQDafX6d5ts2XFYu3YhKds9i/yaVESKrOpRu1BZPhsuHRwIgMQH/MZ0Jquc+yj78w5ETYZXWHzeun7TQ9meljcMb9uk=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":21936},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"0851f660f3d5d7313f886818748c892db6ad17ff","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.1.2_1719065708780_0.9049672011284737","host":"s3://npm-registry-packages"}},"1.1.3":{"name":"llm-scraper","version":"1.1.3","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.1.3","homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"77e76e2d65c41de02b196690f30d62e8cdf57753","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.1.3.tgz","fileCount":14,"integrity":"sha512-Z5C/usVyt7zaD5/WK1ti4UgdCjWocTX8ngfY8Ka1cSBWuqUZH3brfwjMbiqSmQ+bHFf/i+qZ9ftx+wd5TCXhJQ==","signatures":[{"sig":"MEYCIQD/PRaRMGeCDkLUWYc8vQjr8TWLTgs6UQyU4PRoPPok5gIhANsNvOF3sy1j88nJQ+xfeXdgMLsRgwHHqxkKiFHqvMr6","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":21986},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"220793f3d940840b6f381b68402793caa190306e","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.1.3_1719747066974_0.6936371436989859","host":"s3://npm-registry-packages"}},"1.1.4":{"name":"llm-scraper","version":"1.1.4","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.1.4","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"2c519334e437dda8c5e9538620f6850130b8a8b2","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.1.4.tgz","fileCount":15,"integrity":"sha512-qholuj/6Bb8RhfxHZpimcXZZSP7O7VjxO1OwSdf8AGYoSuMkRhHJaZTtAdhqRk5RSnd6fpHZIicFng0lAivAEA==","signatures":[{"sig":"MEYCIQCyMPdKHEc3CROvWIrcTGiSuZyxFy0Ykm8SN4YssbG21AIhANWbm0Gk54pBD3NYON3Ae7sUo9g+uOsxQjyaxJwx/LV/","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23828},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"0f14059de7e83de8a6d682073f204b6b80a09eb0","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.1.4_1720438966458_0.8024982487577696","host":"s3://npm-registry-packages"}},"1.2.0":{"name":"llm-scraper","version":"1.2.0","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.2.0","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"bbd3a65db2cb352080594a5b633387820f069b95","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.2.0.tgz","fileCount":15,"integrity":"sha512-e2DuOJMyAI9cHGf+qH7T6l6dRMByo/XWjThgK6DCOKWBfUs2PsrwLynhGFC7gqmUaRhYOGRgZBMOshA+znrvPA==","signatures":[{"sig":"MEUCIQC08jPuyvpZBdrOqt7swfIrhrBoJTh2SJRNxBVhqmFvcAIgVlQLH9tFdWQpTy44pGJ/iyb6iN8TMJOCTRAw2aTKZ3A=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23956},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"6af529b60b3869b393588e73a4b74c3d6dedc3a1","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.2.0_1720446389585_0.2099087666170436","host":"s3://npm-registry-packages"}},"1.2.1":{"name":"llm-scraper","version":"1.2.1","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.2.1","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"6fb17b6c2f1678b635699fa21d5249647fcf573b","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.2.1.tgz","fileCount":15,"integrity":"sha512-CZoXCNGQQXL9UvWr1PDqKqQaJCw+Uf1w9f86A+2nlMIJfCyuqu7uIbWFaTUnOhb6SRASx9qIyGQ0GQtao9cCFw==","signatures":[{"sig":"MEYCIQCihnqOfSHjq+pFsPtbHun+EavzaWffJC28/xV8DzkwJgIhAJXzux5MX/4RhcA+QjS1PhTfvRKHWjLrQTha69hD+PmV","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":23940},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"6af529b60b3869b393588e73a4b74c3d6dedc3a1","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.2.1_1720446688552_0.010013270960294252","host":"s3://npm-registry-packages"}},"1.2.2":{"name":"llm-scraper","version":"1.2.2","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.2.2","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"f8c90c0055bd64bd34d3301ce85c2c8f5aa120ec","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.2.2.tgz","fileCount":17,"integrity":"sha512-8NIekbNHc71W0im5+IEPC6naUZqco6VrNXJ+ehEI51xHLthG3QY45GDto8ixrO5EZFqykc1VvhK/AaP44tYTug==","signatures":[{"sig":"MEQCIBgqMjJIculXbVJmsqBg148FwvQZ68Vqj7QzHlEsxN+TAiAGIKvd0i4iAm/kmOklA6t0G5gsuXlXk22VgliySUrgwQ==","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":24851},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"b2db6fe6d11483903c34d004903ffe01adf1e5c1","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.2.2_1720866783770_0.2747899538884102","host":"s3://npm-registry-packages"}},"1.5.0":{"name":"llm-scraper","version":"1.5.0","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.5.0","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"ee7dce8472f44cba65e895d8ea8a7cc74a5fc2bf","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.5.0.tgz","fileCount":19,"integrity":"sha512-AyIwp+D3hIyg0BwKWbctETsCNUiAgo9FhuhsU+5hWLq2LcFiQXOQ/1ysOPmBpdYQi9R33b/W7paZslnq1ePcUw==","signatures":[{"sig":"MEUCIQCzKzbJMruHaN4fLeF98pS6MjNTzogD1O+239XDEqK8ZwIgCN+rFsMcOITK6Oi0lm0ZNh15xCxGbuoF12mWJzQ005c=","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":31931},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"61e6877e41c508db5d59d04413dac762b7c049b2","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","@ai-sdk/anthropic":"^0.0.30","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.5.0_1720878061153_0.7831889776508021","host":"s3://npm-registry-packages"}},"1.5.1":{"name":"llm-scraper","version":"1.5.1","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.5.1","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"eef0daded6920fad4a31718b944a9b9bfbe99ecd","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.5.1.tgz","fileCount":19,"integrity":"sha512-E4IvaSElOJ/bqcLcnLywU4Dm2nmIcf7UM8lAQIDSOZ3SiK5/RRSuiBACk18bbZE7wMbsMiXr3xKT6fRtfdf2Fg==","signatures":[{"sig":"MEMCIEQbNZx86MetPItYXG3k/U+/0PRgO8/DVevZ4tjY+SdBAh8i+VtEc7jmhDQsQfTcw1dYSXgF1XIHaOmipDoSCwkT","keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA"}],"unpackedSize":31997},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"eb9bd34622256a6d14d7812fe30c7d6a03b0ccd9","scripts":{"test":"echo \"Error: no test specified\" && exit 1","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.5.0","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.12.2","dependencies":{"ai":"^3.1.12","turndown":"^7.1.3","node-llama-cpp":"^2.8.9","zod-to-json-schema":"^3.22.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.22.5","playwright":"^1.43.1","typescript":"^5.4.5","@types/node":"^20.12.7","@types/react":"^18.2.79","@ai-sdk/openai":"^0.0.2","@ai-sdk/anthropic":"^0.0.30","ollama-ai-provider":"^0.10.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.5.1_1720880635277_0.13052250143347166","host":"s3://npm-registry-packages"}},"1.6.0":{"name":"llm-scraper","version":"1.6.0","keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","_id":"llm-scraper@1.6.0","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"homepage":"https://github.com/mishushakov/llm-scraper#readme","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"dist":{"shasum":"dddda5501c6f34d17cd287fec5d593c8c386a87a","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-1.6.0.tgz","fileCount":25,"integrity":"sha512-naDL/sYKFbOEf3eeogKl6GVWQFlSh+TFcWx57eT+MRRs3QnxOSbU45htfDBS00olpb5pm//VJxGvwpj3omWLag==","signatures":[{"sig":"MEQCIFEbGfYqIrpbRLCZd6yVZl7dM3OZ+oDMk5OxtoQuRA29AiAHZQCfQzfN7OKOjUs3PsrD+iqB0PKPcfAYBZkZRVnxcA==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":33983},"main":"dist/index.js","type":"module","types":"./dist/index.d.ts","gitHead":"8e4225831d2bf56b1c41759a41fa5212569f0e5c","scripts":{"test":"vitest run","build":"tsc -p tsconfig.json"},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"repository":{"url":"git+https://github.com/mishushakov/llm-scraper.git","type":"git"},"_npmVersion":"10.8.2","description":"Turn any webpage intro structured data using LLMs","directories":{},"_nodeVersion":"20.18.2","dependencies":{"ai":"^4.3.15","turndown":"^7.2.0","@ai-sdk/provider":"^1.1.3","zod-to-json-schema":"^3.24.5"},"_hasShrinkwrap":false,"devDependencies":{"zod":"^3.24.4","vitest":"^3.1.3","playwright":"^1.52.0","typescript":"^5.8.3","@types/node":"^22.15.18","@types/react":"^18.2.79","@ai-sdk/openai":"^1.3.22","ollama-ai-provider":"^1.2.0"},"_npmOperationalInternal":{"tmp":"tmp/llm-scraper_1.6.0_1747503571243_0.43399807885642816","host":"s3://npm-registry-packages-npm-production"}},"2.0.0":{"type":"module","name":"llm-scraper","version":"2.0.0","description":"Turn any webpage intro structured data using LLMs","main":"dist/index.js","scripts":{"build":"tsc -p tsconfig.json","test":"vitest run"},"repository":{"type":"git","url":"git+https://github.com/mishushakov/llm-scraper.git"},"keywords":["llm","scraper","browser","playwright","puppeteer"],"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"homepage":"https://github.com/mishushakov/llm-scraper#readme","dependencies":{"@ai-sdk/provider":"^3.0.8","ai":"^6.0.77","turndown":"^7.2.2"},"devDependencies":{"@ai-sdk/openai":"^3.0.26","@types/node":"^25.2.1","@types/react":"^19.2.13","playwright":"^1.58.2","typescript":"^5.9.3","vitest":"^4.0.18","zod":"^4.3.6"},"_id":"llm-scraper@2.0.0","gitHead":"7e6d4b01c8af2015ceb8bcc2a0fadf5807ce9612","types":"./dist/index.d.ts","_nodeVersion":"23.11.0","_npmVersion":"10.9.2","dist":{"integrity":"sha512-N6w1gmuVpXxHkt6F96UJ5t1UKwse/deLWyDm4EwbMLTdn4Dey3xoPKwcST42xvIQhxa1fKQetPGLQ2zEGFaX+g==","shasum":"5d272a42cc756f96c83cc56ebea9d77a10c7f021","tarball":"https://registry.npmjs.org/llm-scraper/-/llm-scraper-2.0.0.tgz","fileCount":25,"unpackedSize":34912,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIEPrqpCQvcS634myz4lgvULNosjypfNdrAWb7aIIX9iKAiEA25e4SXijZk1WmqnBSws7NARDJjkMZyrzQZLzoFPcgVI="}]},"_npmUser":{"name":"mishushakov","email":"mish@ushakov.co"},"directories":{},"maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/llm-scraper_2.0.0_1770484682881_0.8609034823413677"},"_hasShrinkwrap":false}},"time":{"created":"2024-04-20T17:05:49.904Z","modified":"2026-02-07T17:18:03.140Z","1.0.0":"2024-04-20T17:05:50.084Z","1.0.1":"2024-04-20T20:25:29.836Z","1.0.2":"2024-04-21T11:14:22.998Z","1.0.3":"2024-04-21T12:46:04.177Z","1.0.4":"2024-04-21T15:52:15.358Z","1.0.5":"2024-04-21T21:12:04.090Z","1.0.6":"2024-06-22T13:16:40.389Z","1.1.0":"2024-06-22T13:36:22.352Z","1.1.1":"2024-06-22T13:52:01.327Z","1.1.2":"2024-06-22T14:15:08.925Z","1.1.3":"2024-06-30T11:31:07.157Z","1.1.4":"2024-07-08T11:42:46.590Z","1.2.0":"2024-07-08T13:46:29.733Z","1.2.1":"2024-07-08T13:51:28.681Z","1.2.2":"2024-07-13T10:33:03.919Z","1.5.0":"2024-07-13T13:41:01.322Z","1.5.1":"2024-07-13T14:23:55.450Z","1.6.0":"2025-05-17T17:39:31.432Z","2.0.0":"2026-02-07T17:18:03.021Z"},"bugs":{"url":"https://github.com/mishushakov/llm-scraper/issues"},"author":{"name":"Mish Ushakov","email":"mishushakov@users.noreply.github.com"},"license":"MIT","homepage":"https://github.com/mishushakov/llm-scraper#readme","keywords":["llm","scraper","browser","playwright","puppeteer"],"repository":{"type":"git","url":"git+https://github.com/mishushakov/llm-scraper.git"},"description":"Turn any webpage intro structured data using LLMs","maintainers":[{"name":"mishushakov","email":"mish@ushakov.co"}],"readme":"# LLM Scraper\n\n<img width=\"1800\" alt=\"Screenshot 2024-04-20 at 23 11 16\" src=\"https://github.com/mishushakov/llm-scraper/assets/10400064/ab00e048-a9ff-43b6-81d5-2e58090e2e65\">\n\nLLM Scraper is a TypeScript library that allows you to extract structured data from **any** webpage using LLMs.\n\n> [!IMPORTANT]\n> **LLM Scraper was updated to version 2.0.**\n>\n> The new version comes with **Vercel AI SDK 6** support and updated examples.\n\n### Features\n\n- Supports GPT, Sonnet, Gemini, Llama, Qwen model series\n- Schemas defined with Zod or JSON Schema\n- Full type-safety with TypeScript\n- Based on Playwright framework\n- Streaming objects\n- [Code-generation](#code-generation)\n- Supports 6 formatting modes:\n  - `html` for loading pre-processed HTML\n  - `raw_html` for loading raw HTML (no processing)\n  - `markdown` for loading markdown\n  - `text` for loading extracted text (using [Readability.js](https://github.com/mozilla/readability))\n  - `image` for loading a screenshot (multi-modal only)\n  - `custom` for loading custom content (using a custom function)\n\n**Make sure to give it a star!**\n\n<img width=\"165\" alt=\"Screenshot 2024-04-20 at 22 13 32\" src=\"https://github.com/mishushakov/llm-scraper/assets/10400064/11e2a79f-a835-48c4-9f85-5c104ca7bb49\">\n\n## Getting started\n\n1. Install the required dependencies from npm:\n\n   ```\n   npm i zod playwright llm-scraper\n   ```\n\n2. Initialize your LLM:\n\n   **OpenAI**\n\n   ```\n   npm i @ai-sdk/openai\n   ```\n\n   ```js\n   import { openai } from '@ai-sdk/openai'\n\n   const llm = openai('gpt-4o')\n   ```\n\n   **Anthropic**\n\n   ```\n   npm i @ai-sdk/anthropic\n   ```\n\n   ```js\n   import { anthropic } from '@ai-sdk/anthropic'\n\n   const llm = anthropic('claude-3-5-sonnet-20240620')\n   ```\n\n   **Google**\n\n   ```\n   npm i @ai-sdk/google\n   ```\n\n   ```js\n   import { google } from '@ai-sdk/google'\n\n   const llm = google('gemini-1.5-flash')\n   ```\n\n   **Groq**\n\n   ```\n   npm i @ai-sdk/openai\n   ```\n\n   ```js\n   import { createOpenAI } from '@ai-sdk/openai'\n   const groq = createOpenAI({\n     baseURL: 'https://api.groq.com/openai/v1',\n     apiKey: process.env.GROQ_API_KEY,\n   })\n\n   const llm = groq('llama3-8b-8192')\n   ```\n\n   **Ollama**\n\n   ```\n   npm i ollama-ai-provider-v2\n   ```\n\n   ```js\n   import { ollama } from 'ollama-ai-provider-v2'\n\n   const llm = ollama('llama3')\n   ```\n\n3. Create a new scraper instance provided with the llm:\n\n   ```js\n   import LLMScraper from 'llm-scraper'\n\n   const scraper = new LLMScraper(llm)\n   ```\n\n## Example\n\nIn this example, we're extracting top stories from HackerNews:\n\n```ts\nimport { chromium } from 'playwright'\nimport { z } from 'zod'\nimport { Output } from 'ai'\nimport { openai } from '@ai-sdk/openai'\nimport LLMScraper from 'llm-scraper'\n\n// Launch a browser instance\nconst browser = await chromium.launch()\n\n// Initialize LLM provider\nconst llm = openai('gpt-4o')\n\n// Create a new LLMScraper\nconst scraper = new LLMScraper(llm)\n\n// Open new page\nconst page = await browser.newPage()\nawait page.goto('https://news.ycombinator.com')\n\n// Define schema to extract contents into\nconst schema = z.object({\n  top: z\n    .array(\n      z.object({\n        title: z.string(),\n        points: z.number(),\n        by: z.string(),\n        commentsURL: z.string(),\n      })\n    )\n    .length(5)\n    .describe('Top 5 stories on Hacker News'),\n})\n\n// Run the scraper (`output` is required)\nconst { data } = await scraper.run(page, Output.object({ schema }), {\n  format: 'html',\n})\n\n// Show the result from LLM\nconsole.log(data.top)\n\nawait page.close()\nawait browser.close()\n```\n\nOutput\n\n```js\n[\n  {\n    title: \"Palette lighting tricks on the Nintendo 64\",\n    points: 105,\n    by: \"ibobev\",\n    commentsURL: \"https://news.ycombinator.com/item?id=44014587\",\n  },\n  {\n    title: \"Push Ifs Up and Fors Down\",\n    points: 187,\n    by: \"goranmoomin\",\n    commentsURL: \"https://news.ycombinator.com/item?id=44013157\",\n  },\n  {\n    title: \"JavaScript's New Superpower: Explicit Resource Management\",\n    points: 225,\n    by: \"olalonde\",\n    commentsURL: \"https://news.ycombinator.com/item?id=44012227\",\n  },\n  {\n    title: \"\\\"We would be less confidential than Google\\\" Proton threatens to quit Switzerland\",\n    points: 65,\n    by: \"taubek\",\n    commentsURL: \"https://news.ycombinator.com/item?id=44014808\",\n  },\n  {\n    title: \"OBNC – Oberon-07 Compiler\",\n    points: 37,\n    by: \"AlexeyBrin\",\n    commentsURL: \"https://news.ycombinator.com/item?id=44013671\",\n  }\n]\n```\n\nMore examples can be found in the [examples](./examples) folder.\n\n## Streaming\n\nReplace your `run` function with `stream` to get a partial object stream (Vercel AI SDK v6+).\n\n```ts\n// Run the scraper in streaming mode\nconst { stream } = await scraper.stream(page, Output.object({ schema }))\n\n// Stream the result from LLM\nfor await (const data of stream) {\n  console.log(data.top)\n}\n```\n\n## Code-generation\n\nUsing the `generate` function you can generate re-usable playwright script that scrapes the contents according to a schema.\n\n```ts\n// Generate code and run it on the page\nconst { code } = await scraper.generate(page, Output.object({ schema }))\nconst result = await page.evaluate(code)\nconst data = schema.parse(result)\n\n// Show the parsed result\nconsole.log(data.top)\n```\n\n## Contributing\n\nAs an open-source project, we welcome contributions from the community. If you are experiencing any bugs or want to add some improvements, please feel free to open an issue or pull request.\n","readmeFilename":"README.md"}