{"_id":"@arachnodex/core","_rev":"13-6840b2322943d87968eace56a311562b","name":"@arachnodex/core","dist-tags":{"latest":"1.0.12"},"versions":{"1.0.0":{"name":"@arachnodex/core","version":"1.0.0","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.0","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"d518a06ddb385c960075accfaecee497c269a94e","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.0.tgz","fileCount":58,"integrity":"sha512-vtRwacx06rfOeug6gKhys2y/wqD6juOo6VUKLy+XxtC/n5uTYFl8Qyal+j37/gM6MENQsAER5cnl5njySn1NIQ==","signatures":[{"sig":"MEYCIQCCwQeTz46EuHKF7zpIwyR8aEz22/VmrE4FKhUJAmtxGAIhAOUZwwzJDOGiNy4FmnhjwArC0Z0qvUmZGbPMEaB4+glI","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":373293},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"4d3eab58d0a7eb494435ad9e499a1cee2bb21300","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.0"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.0_1780031413390_0.8337418886414618","host":"s3://npm-registry-packages-npm-production"}},"1.0.1":{"name":"@arachnodex/core","version":"1.0.1","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.1","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"f90af541c2fbbf3930d4137cf5c6b46334522d54","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.1.tgz","fileCount":59,"integrity":"sha512-k9rqSclgeDJw+npIex1Vzqc7q8PDjKcTICxAgpiyd7R6MXk7fHdrKc3cX4UUnptPBcv+EtVbMaopbhhCc1x9JA==","signatures":[{"sig":"MEUCIDquqJd7RRE/MkJia7XLR8hIuqgy+y5YJbMT3p/keAlBAiEAsz4LmMNkp/NqZjW67Hx0UXhgoNjfR2+VUbTymFTGHpg=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":388936},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"3b454aa834f9977fcb5cadcb0f2b4481a5750f82","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.1_1780034140751_0.7919435071525789","host":"s3://npm-registry-packages-npm-production"}},"1.0.2":{"name":"@arachnodex/core","version":"1.0.2","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.2","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"309f76e06d0a1a98709d26cdecc2908f436ddc94","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.2.tgz","fileCount":59,"integrity":"sha512-Pfgc6SyEktaL1HS9CHzLUHwlNFGu8IlWbyDpTonnQh/8F6j6Ebj1pxZa4g5dtlnBmpylRWjkMqCuwsIzz4MmEA==","signatures":[{"sig":"MEUCIGawOofbAXLB4ykEpi9OfYkiW2V0/oSh/y9uImCoKBbXAiEAnmF1xYNE+5H647IIAAj0A/AIRYhfTMCtsz9JJeC4ufA=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":389869},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"d607f2fe2fab7e2987b9d656cddf36ccdeac5fb1","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.2_1780035481777_0.16382409563948763","host":"s3://npm-registry-packages-npm-production"}},"1.0.3":{"name":"@arachnodex/core","version":"1.0.3","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.3","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"6088908ad224149312a33d0697193854b2898fea","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.3.tgz","fileCount":59,"integrity":"sha512-AL7eQIDBYJNrMDRbRH2KKTM65GHZeFbw1q5i0Cl+jLe4OLrZl64V9Oom+l7kFZlKC8Puh3BS+PB4xNgQead7hA==","signatures":[{"sig":"MEUCIQDGB1TNyHzbx3Yu99xfl25EVS3RbuR3bU6dhThd1aAlLgIgfnpi/1FZQ6JMNMIWRGLcTCeKvKHUPIIzYT6Vff9Zitk=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":392231},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"d607f2fe2fab7e2987b9d656cddf36ccdeac5fb1","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.3_1780074176876_0.387288627499649","host":"s3://npm-registry-packages-npm-production"}},"1.0.4":{"name":"@arachnodex/core","version":"1.0.4","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.4","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"30602643e3a8317fef1c18929cfb3348817d22ba","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.4.tgz","fileCount":59,"integrity":"sha512-X3shMtxVJfGmUCPCZtDXPuqHNOOl/FiCHwKjTKgYw9lgabhfMTKzb59waMaKVSgagSuJCVco2AKsdStDFMBrVg==","signatures":[{"sig":"MEUCIQC+gjf3ht/XVvYqY98sXgdJ6NZQeGk3BiyWprowDLtrYAIgfhXOScatUqexvOu14KHH9XUwgrYzSpBaD6Ra0nMsbgQ=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":396694},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"f182a55ffac726141405b352ad31d4db987cdc83","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.4_1780451996282_0.20819040312821402","host":"s3://npm-registry-packages-npm-production"}},"1.0.5":{"name":"@arachnodex/core","version":"1.0.5","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.5","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"ca97d8a456de2cbf1ae2d0b00bd749393d6c4225","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.5.tgz","fileCount":59,"integrity":"sha512-J67qTaPFNZolIuP+ErREzjg5ePbw7Z+z48o/HbnIf/QeT02zfw07Qu/CSGc6m+MZvT1UeUPYuLuKJBtISztdGg==","signatures":[{"sig":"MEQCICjUF1gHs4eyQLvQMDK2X8GUUhHXDAaK8WpY4MXQmOTIAiAmWbWpXrC2X+mHC9JC/jwemn4ecZFFh2fzCdFWsqhEsg==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":396943},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"366882216108b29e466e9fe7c9e0c81ddf3b7062","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.5_1780526377811_0.9715896717720569","host":"s3://npm-registry-packages-npm-production"}},"1.0.6":{"name":"@arachnodex/core","version":"1.0.6","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.6","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"04e4f69bd8edbe39bf28590ff911f19aa2035e57","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.6.tgz","fileCount":59,"integrity":"sha512-J3xEjA4+tYHnO2EvMffjkCJXWxu8D85twjbS32o9EcTmTg1BvINvoJB+ZjC4AQoU5zIL8PcDLKDNzznSmoHp/g==","signatures":[{"sig":"MEUCIB9g25VMkHxkHCN/ErA4akx/lCMi2t+WYt5aY7JyOGkJAiEAtpg+/pF1RDG0xvkNx8glUC3NHvnz4QhWgKg87sYJY8o=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":397116},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"ec64588bb4a0ec3c22e2ea7d073b167ab121e7f3","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.6_1780606339276_0.526639992334692","host":"s3://npm-registry-packages-npm-production"}},"1.0.7":{"name":"@arachnodex/core","version":"1.0.7","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.7","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"b4c3fb63a5b64c40f26af04dd9bae411c8892f6d","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.7.tgz","fileCount":59,"integrity":"sha512-qEN5VGwMdMJDq9yr/CrGhJ86HI7dTqUi6MhtdLuV/cTjbTYRXTswgxw4Zu+fk1Je6gI7WFGqmwZOa64r3yInfQ==","signatures":[{"sig":"MEQCIHMh6B2X8qx6e4jWM2CQ9W/GAlqURk5qs5MWifyA6bI5AiBUfs4bhbCX2TmqzoSaKYMtISfGjRKy8hVdDQn9+2x52Q==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":405578},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"b0e3b30f375340d5ad0bf99deb84d976a51a7bc2","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.7_1780682619134_0.5504383046962871","host":"s3://npm-registry-packages-npm-production"}},"1.0.8":{"name":"@arachnodex/core","version":"1.0.8","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.8","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"4d4dcd241206c2c87b4b97190fed7543471235b4","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.8.tgz","fileCount":59,"integrity":"sha512-WIxF6meHQB5QQKZ05QWVAdlZsGjA9bMVmvprv8lqE4tJqFnSuxAyOu/SbKnRd1+ZohsYTx39zZ/8qBLNVqq9DQ==","signatures":[{"sig":"MEUCIEY4NXBbfyZeQoHDBpzwDRHxBtwxVt/Lh5hPfpw2P2UuAiEAmGxfB9XvcX2TquKyaBhCHehpReZAKGoQxAyitRICiw0=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":406368},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"d58d41f13c38355d2bb3c35d1b8c6612d1506fa2","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.8_1781300437339_0.45342541975012884","host":"s3://npm-registry-packages-npm-production"}},"1.0.9":{"name":"@arachnodex/core","version":"1.0.9","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.9","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"25eceb6ec410e8be60e2b66eaa9c21e72b335401","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.9.tgz","fileCount":59,"integrity":"sha512-3nENbod6YmhGPKCR8zn0ZhACBUVyKi0x92YaPW0i1cU55vMB8kr16PDO9Q7Jzfv/CHD1bGOvJtOn9HTYI5PYmA==","signatures":[{"sig":"MEQCICPJCOrCvYaV4S/g44AQl1HSasX/4RBRzRxpnBXWdL99AiABvEkby8+24198G860HMF+ZKnM7NSqQCZA+TNqsN5Qyw==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":407204},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"0e7ce7f80e6454376dee1297aaa6f510dd9b827f","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"echo \"Error: no test specified\" && exit 1","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.13.0","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^6.10.1","deepmerge-ts":"^1.1.5","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^6.4.21","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.9_1784824889003_0.8537998548839665","host":"s3://npm-registry-packages-npm-production"}},"1.0.10":{"name":"@arachnodex/core","version":"1.0.10","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.10","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"60a59c32264f8c313fdaa061258910da9159f875","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.10.tgz","fileCount":61,"integrity":"sha512-vwqK9cmeDeoSNeDh+36ZLJb9roZZslg+rL1/jRjxodripc7xKEUc9d/AF/UkZQkGxaMZPdLZbvePj1KKupuWdQ==","signatures":[{"sig":"MEQCIAefkGXhxVW3MTxDU4oRACaQXlKJotVLUN35dERN8/30AiAoseoQ2j9zB/o0AhuiVkGEqHk08ZXYrzLOgawdrWro8g==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@arachnodex%2fcore@1.0.10","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":423098},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"56d075dd0cd42082f51c6aa5a85d4ebbe373178c","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"node --test --conditions=development --import tsx test/*.test.ts","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"GitHub Actions","email":"npm-oidc-no-reply@github.com","trustedPublisher":{"id":"github","oidcConfigId":"oidc:168323f1-72f6-4ba3-9af7-e9b6d2fd4771"}},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.19.1","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.20.0","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^9.0.6","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^8.0.1","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.10_1788284404688_0.0819878057458594","host":"s3://npm-registry-packages-npm-production"}},"1.0.11":{"name":"@arachnodex/core","version":"1.0.11","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/core@1.0.11","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"bin":{"arachnodex":"bin/index.js"},"dist":{"shasum":"3f1894b4a2ad89db5200d7432df2f6bb63258126","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.11.tgz","fileCount":61,"integrity":"sha512-g3NCb5QyiOQCVUd+3z4YaNtqeI49KuCzwW5t6vPuZFXCfnFDBrat4X/xYvMUrmIe++AcyQ/y2Lx0+/qDdN+cWg==","signatures":[{"sig":"MEUCIC7E0OmEbWGVcbFCz732vO7tVxIjEvyaoqKAmxX6TbnXAiEArsy8ff8eGOzttdfxPsy6uOJ8+FKXXQk8+dGqylI0w9s=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@arachnodex%2fcore@1.0.11","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"unpackedSize":423950},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"gitHead":"a4e140dc6dad837b0f2ffae0f64936051e3ab47b","scripts":{"fix":"eslint ./src/**/*.ts --fix","lint":"eslint ./src/*.ts ./src/**/*.ts","test":"node --test --conditions=development --import tsx test/*.test.ts","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","crawl":"node bin/index.js","prepack":"node ../../scripts/sync-core-readme.cjs","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","crawl:src":"node --conditions=development --import tsx src/index.ts","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"GitHub Actions","email":"npm-oidc-no-reply@github.com","trustedPublisher":{"id":"github","oidcConfigId":"oidc:168323f1-72f6-4ba3-9af7-e9b6d2fd4771"}},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/core"},"_npmVersion":"11.19.1","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.20.0","chalk":"^5.6.2","jsdom":"^22.1.0","fs-extra":"^11.3.2","handlebars":"^4.7.8","nodemailer":"^9.0.6","eventemitter3":"^4.0.7","@arachnodex/bot-protection-heuristics":"^1.0.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.6","eslint":"^8.57.1","esbuild":"^0.25.0","ts-node":"^10.9.2","chokidar":"^4.0.3","typescript":"^5.6.3","@types/node":"^22","@types/jsdom":"^27.0.0","chokidar-cli":"^3.0.0","@types/fs-extra":"^11.0.4","@types/nodemailer":"^8.0.1","@typescript-eslint/parser":"^8.46.2","@typescript-eslint/eslint-plugin":"^8.46.2"},"_npmOperationalInternal":{"tmp":"tmp/core_1.0.11_1788297250619_0.44125001166864886","host":"s3://npm-registry-packages-npm-production"}},"1.0.12":{"name":"@arachnodex/core","version":"1.0.12","description":"Arachnodex core crawler runtime, shared APIs, and CLI.","main":"bin/index.js","types":"types/index.d.ts","bin":{"arachnodex":"bin/index.js"},"type":"module","exports":{".":{"types":"./types/index.d.ts","development":"./src/index.ts","default":"./bin/index.js"},"./README.md":{"default":"./README.md"},"./config/default.example.json":{"default":"./config/default.example.json"}},"scripts":{"test":"node --test --conditions=development --import tsx test/*.test.ts","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\"","build:types":"tsc -p tsconfig.types.json","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --banner:js='#!/usr/bin/env node' --outfile=bin/index.js && node -e \"require('node:fs').chmodSync('bin/index.js', 0o755)\" && npm run build:types","prepack":"node ../../scripts/sync-core-readme.cjs","lint":"eslint ./src/*.ts ./src/**/*.ts","fix":"eslint ./src/**/*.ts --fix","lint:watch":"chokidar src/*.ts src/jobs/*.ts -c \"npm run lint\" --initial --verbose","crawl":"node bin/index.js","crawl:src":"node --conditions=development --import tsx src/index.ts"},"repository":{"type":"git","url":"git+https://github.com/Arachnodex/arachnodex.git","directory":"packages/core"},"keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"publishConfig":{"access":"public"},"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"homepage":"https://github.com/Arachnodex/arachnodex#readme","dependencies":{"@arachnodex/bot-protection-heuristics":"^1.0.2","axios":"^1.20.0","chalk":"^5.6.2","eventemitter3":"^4.0.7","fs-extra":"^11.3.2","handlebars":"^4.7.8","jsdom":"^22.1.0","nodemailer":"^9.0.6"},"devDependencies":{"@types/fs-extra":"^11.0.4","@types/jsdom":"^27.0.0","@types/node":"^22","@types/nodemailer":"^8.0.1","@typescript-eslint/eslint-plugin":"^8.46.2","@typescript-eslint/parser":"^8.46.2","chokidar":"^4.0.3","chokidar-cli":"^3.0.0","esbuild":"^0.25.0","eslint":"^8.57.1","ts-node":"^10.9.2","tsx":"^4.20.6","typescript":"^5.6.3"},"engines":{"node":">=22.13.0 <25"},"gitHead":"26dcf5eac8a87bde5b3b274fba3a6f82776e0bfd","_id":"@arachnodex/core@1.0.12","_nodeVersion":"24.16.0","_npmVersion":"11.19.1","dist":{"integrity":"sha512-q0GH2lyyGvfb4YkXKY/m19boWD1Mca5AycIoIsJw3QpozlTWiWcNMNLl5mTg41oH42ZMDnD+xv31/UwOBJWPjA==","shasum":"40d04f0dbe6f30bdeb347790c7b9e7d02524971a","tarball":"https://registry.npmjs.org/@arachnodex/core/-/core-1.0.12.tgz","fileCount":61,"unpackedSize":430884,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@arachnodex%2fcore@1.0.12","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCICXFfwDbt7h5ulVVVvij4ArCqsw9hqD6TICbeHk+dVs7AiB0BaMbaVGY+gRZXexV3536nM5sd8I8H3MjzWueiTLbdw=="}]},"_npmUser":{"name":"GitHub Actions","email":"npm-oidc-no-reply@github.com","trustedPublisher":{"id":"github","oidcConfigId":"oidc:168323f1-72f6-4ba3-9af7-e9b6d2fd4771"}},"directories":{},"maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/core_1.0.12_1788304763402_0.9311044897092091"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-29T05:10:13.186Z","modified":"2026-09-01T23:19:23.949Z","1.0.0":"2026-05-29T05:10:13.607Z","1.0.1":"2026-05-29T05:55:40.985Z","1.0.2":"2026-05-29T06:18:01.993Z","1.0.3":"2026-05-29T17:02:57.165Z","1.0.4":"2026-06-03T01:59:56.489Z","1.0.5":"2026-06-03T22:39:37.995Z","1.0.6":"2026-06-04T20:52:19.470Z","1.0.7":"2026-06-05T18:03:39.340Z","1.0.8":"2026-06-12T21:40:37.523Z","1.0.9":"2026-07-23T16:41:29.136Z","1.0.10":"2026-09-01T17:40:04.884Z","1.0.11":"2026-09-01T21:14:10.820Z","1.0.12":"2026-09-01T23:19:23.614Z"},"bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","homepage":"https://github.com/Arachnodex/arachnodex#readme","keywords":["arachnodex","crawler","cli","sitemap generator","404 finder","bad link checker","website spider"],"repository":{"type":"git","url":"git+https://github.com/Arachnodex/arachnodex.git","directory":"packages/core"},"description":"Arachnodex core crawler runtime, shared APIs, and CLI.","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"readme":"# Arachnodex\n\nArachnodex is a modular Node.js web crawler framework. It spiders a configured site, parses page data, and runs one or more installed jobs during the crawl.\n\nThe create workflow installs the core crawler, sitemap, link issue reporting, non-fingerprinted assets reporting, CSP reporting, and a small bot protection heuristics package that can be updated independently.\n\n## Requirements\n\n- Node.js `22.13.0` or newer, including Node 24.\n- npm `11.13.0` or compatible.\n\nThe repo includes an `.nvmrc` for contributors who use nvm. CI checks Node 22 and Node 24 so changes stay compatible with the supported range.\n\n## Custom Jobs\n\nArachnodex can load your own job packages in addition to the official jobs. A job is an npm package with a default class export. Official shorthand such as `-j sitemap` resolves to `@arachnodex/job-sitemap`, while third-party scoped packages can be loaded by their full package name.\n\nCustom jobs can be published to npm, installed from a private registry, or installed from a local filesystem path while you are developing private code.\n\nFor active job development, Arachnodex also supports a TypeScript source runner so you can iterate without rebuilding `bin/index.js` after every edit.\n\nRead [CUSTOM-JOBS.md](CUSTOM-JOBS.md) for job package structure, lifecycle hooks, command switches, config files, source-mode development, and install options.\n\nFor a list of third-party custom jobs, check out the [Third-Party Job Registry](JOB-REGISTRY.md).\n\n## Install\n\nThe recommended path is the create command:\n\n```sh\nnpm create @arachnodex my-crawl-project\ncd my-crawl-project\nnpm run crawl:default\n```\n\nThat initializes a runnable project with local `README.md`, `package.json`, and `config/` files, installs the core crawler and official jobs, and adds pass-through `crawl` / `crawl:src` scripts plus default starter scripts.\n\nYou can skip automatic install if you want to inspect or edit files first:\n\n```sh\nnpm create @arachnodex my-crawl-project -- --no-install\ncd my-crawl-project\nnpm install\n```\n\nYou can also install the CLI globally:\n\n```sh\nnpm install -g @arachnodex/core\nnpm install -g @arachnodex/job-sitemap @arachnodex/job-link-issues @arachnodex/job-nfa-report @arachnodex/job-csp-report\narachnodex -c default -j sitemap -j link-issues -j nfa-report -j csp-report\n```\n\nThe global install works for the core CLI, but core does not include job packages. Install the jobs you want alongside it so shorthand names such as `sitemap` and `link-issues` can resolve.\n\nFor project work, prefer the create command or a local project install so config, job versions, and output files live with the site audit project.\n\nFor a minimal manual local install, add core and only the jobs you need:\n\n```sh\nnpm install @arachnodex/core\nnpm install @arachnodex/job-sitemap\nnpm exec -- arachnodex -c default -j sitemap\n```\n\n## Package Boundaries\n\n`@arachnodex/core` provides the crawler runtime, shared APIs, and the `arachnodex` CLI. It does not bundle official jobs.\n\nJob handles such as `sitemap`, `link-issues`, `nfa-report`, and `csp-report` are shorthand import names. They resolve to `@arachnodex/job-sitemap`, `@arachnodex/job-link-issues`, `@arachnodex/job-nfa-report`, and `@arachnodex/job-csp-report`, but those packages must be installed in the current local project or in the same global environment as the CLI.\n\n`npm create @arachnodex` installs the default official jobs for generated projects. Manual and global installs should add whichever official or third-party job packages they plan to run.\n\n## Basic Usage\n\nArachnodex reads JSON config files from `config/`. The default crawler config is:\n\n```text\nconfig/default.json\n```\n\nA minimal config looks like this:\n\n```json\n{\n  \"siteName\": \"Example Site\",\n  \"domain\": \"example.com\",\n  \"baseUrl\": \"https://www.example.com\",\n  \"numThreads\": 10,\n  \"mail\": {\n    \"disabled\": true\n  }\n}\n```\n\nCrawler HTTPS certificate verification is enabled by default. If you intentionally need to crawl a staging or client site with an invalid certificate, opt out in config:\n\n```json\n{\n  \"requestTls\": {\n    \"rejectUnauthorized\": false\n  }\n}\n```\n\nProjects created with `npm create @arachnodex` include a default starter script:\n\n```sh\nnpm run crawl:default\n```\n\nThey also include a pass-through `crawl` script for custom runs. Put crawler arguments after `--` so npm forwards them:\n\n```sh\nnpm run crawl -- -c default -j sitemap\n```\n\nGenerated projects also include a source-mode runner for custom job development:\n\n```sh\nnpm run crawl:src -- -c default -j sitemap -j link-issues -j nfa-report -j csp-report\n```\n\nIn this monorepo, use the root `crawl-dev` script for the same source-mode workflow:\n\n```sh\nnpm run crawl-dev -- -j link-issues -n -e -p\n```\n\n`crawl-dev` wraps `npm --workspace @arachnodex/core run crawl:src --`, so anything after `--` is passed through to the crawler. The underlying `crawl:src` script uses `tsx` and Node's `development` export condition to run core and compatible job packages from TypeScript source instead of their built `bin/` files.\n\nFor local installs, run custom commands through npm so it can find the local `node_modules/.bin/arachnodex` executable:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap\n```\n\nBare `arachnodex ...` commands are intended for global installs.\n\nRun one job:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap\n```\n\nRun multiple jobs in one crawl:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap -j link-issues\n```\n\nRun all default official jobs:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap -j link-issues -j nfa-report\n```\n\nPass switches to a specific job by placing them after that job name and before the next `-j`:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap -j link-issues -e -n\n```\n\nIn that example, `-e` and `-n` belong to `link-issues`.\n\nUse a non-default crawler config:\n\n```sh\nnpm exec -- arachnodex -c staging -j sitemap\n```\n\nThat loads:\n\n```text\nconfig/staging.json\n```\n\nUse a job-specific config by placing `-c` after the job name:\n\n```sh\nnpm exec -- arachnodex -c default -j link-issues -c link-issues\n```\n\nThat loads the crawler config from `config/default.json` and the link issue job config from `config/link-issues.json`.\n\n## Core Config Settings\n\nCore crawler config files live in `config/`. The default run loads:\n\n```text\nconfig/default.json\n```\n\nProjects created with `npm create @arachnodex` start from the core example config and copy it into:\n\n```text\nconfig/default.example.json\nconfig/default.json\n```\n\nFull example:\n\n```json\n{\n  \"siteName\": \"Example Site\",\n  \"domain\": \"example.com\",\n  \"baseUrl\": \"https://www.example.com\",\n  \"pathPrefix\": \"\",\n  \"entryFile\": \"\",\n  \"dontResetUrls\": false,\n  \"numThreads\": 10,\n  \"requestDelayMs\": 0,\n  \"requestTimeoutMs\": 30000,\n  \"requestTimeoutMaxRetries\": 3,\n  \"requestHead\": {\n    \"enabled\": true,\n    \"failureWarningIgnorePatterns\": []\n  },\n  \"requestTls\": {\n    \"rejectUnauthorized\": true\n  },\n  \"muteResponseStatus\": false,\n  \"muteAll\": false,\n  \"disableColorOutput\": false,\n  \"urlCantContain\": [],\n  \"urlMustContain\": [],\n  \"treatHashAsUniquePage\": false,\n  \"mail\": {\n    \"disabled\": true,\n    \"defaultSubject\": \"Arachnodex {{domain}} Report [{{jobs}}]\",\n    \"developerRecipients\": [],\n    \"reportRecipients\": [],\n    \"errorRecipients\": [],\n    \"from\": {\n      \"Arachnodex Spider\": \"noreply@example.com\"\n    },\n    \"replyTo\": [],\n    \"transport\": {\n      \"host\": \"127.0.0.1\",\n      \"port\": \"1025\",\n      \"secure\": false\n    }\n  }\n}\n```\n\n### Site Settings\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `siteName` | string | `\"\"` | Human-readable site name used in reports. |\n| `domain` | string | `\"\"` | Canonical hostname without `www.`. For example, use `example.com`, not `www.example.com`. |\n| `baseUrl` | string | `\"\"` | Root crawl URL. Must be a valid `http://` or `https://` URL with no path, query string, or hash. The hostname must match `domain`, allowing only an optional leading `www.`. |\n| `pathPrefix` | string | `\"\"` | Optional path prefix for crawls that should stay under a subdirectory. When set, Arachnodex also requires discovered URLs to contain the configured domain plus this prefix. |\n| `entryFile` | string | `\"\"` | Optional entry path appended after `baseUrl` and `pathPrefix` for the first URL queued by the crawler. |\n| `dontResetUrls` | boolean | `false` | Leave internal URLs on their discovered protocol and host instead of normalizing them back to `baseUrl`. Most projects should keep this `false`. |\n\n### Crawl Settings\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `numThreads` | number | `10` | Maximum number of concurrent crawler workers. Values lower than `1` are treated as `1`. |\n| `requestDelayMs` | number | `0` | Delay in milliseconds before crawler requests. |\n| `requestTimeoutMs` | number | `30000` | Per-request timeout in milliseconds. |\n| `requestTimeoutMaxRetries` | number | `3` | Maximum retry attempts for timeout failures. |\n| `requestHead.enabled` | boolean | `true` | Probe internal URLs with HEAD before GET. Set to `false` to skip HEAD and request every internal URL with GET immediately. |\n| `requestHead.failureWarningIgnorePatterns` | string[] | `[]` | Regular expressions matched against the full normalized internal URL. Matching URLs still use fallback GET after a failed HEAD, but the Link Issues job omits the `internal-head-failed` warning. |\n| `requestTls.rejectUnauthorized` | boolean | `true` | Verify HTTPS certificates for crawler requests and job requests that use the core request TLS setting. Set to `false` only when intentionally crawling a site with an invalid certificate. |\n| `treatHashAsUniquePage` | boolean | `false` | Keep URL fragments as part of the crawled URL identity. By default, fragments are stripped so `/page#one` and `/page#two` are treated as the same page. |\n\nInternal HEAD responses below 200 or at least 400, plus transport failures that\ndo not return a response, fall back to GET. The GET result determines whether\nthe URL is available, while the Link Issues job separately reports the failed\nHEAD as an `internal-head-failed` warning. Redirect responses remain normal\nredirect findings rather than HEAD failures.\n\nSuppress the HEAD warning for known URL exceptions with regular expressions:\n\n```json\n{\n  \"requestHead\": {\n    \"enabled\": true,\n    \"failureWarningIgnorePatterns\": [\n      \"^https://www\\\\.example\\\\.com/legacy/\",\n      \"^https://www\\\\.example\\\\.com/api/status$\"\n    ]\n  }\n}\n```\n\nUse `\".*\"` as the only pattern to suppress these warnings site-wide while\nretaining HEAD probes and GET fallback behavior. To avoid sending internal HEAD\nrequests entirely, configure:\n\n```json\n{\n  \"requestHead\": {\n    \"enabled\": false,\n    \"failureWarningIgnorePatterns\": []\n  }\n}\n```\n\nWith HEAD disabled, the crawler goes directly to GET and no HEAD-failure\nwarnings are generated.\n\n### Filtering Settings\n\n`urlCantContain` and `urlMustContain` are arrays of regular expression strings.\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `urlCantContain` | string[] | `[]` | Reject discovered URLs that match any listed pattern. |\n| `urlMustContain` | string[] | `[]` | Reject discovered URLs unless every listed pattern matches. |\n\n### Output Settings\n\nThese values can be set in config or overridden by core switches.\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `muteResponseStatus` | boolean | `false` | Mute crawler response status output. Job output and errors still print. |\n| `muteAll` | boolean | `false` | Mute all non-error output, including job output. |\n| `disableColorOutput` | boolean | `false` | Disable ANSI color output. |\n\n### Mail Settings\n\nMail is disabled by default. Enable it by setting `mail.disabled` to `false` and configuring recipients plus transport settings.\n\nRecipient lists accept email strings or name-to-email maps:\n\n```json\n[\n  \"rick@example.com\",\n  {\"Jane Developer\": \"jane@example.com\"}\n]\n```\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `mail.disabled` | boolean | `true` | Disable regular and error report emails. |\n| `mail.defaultSubject` | string | `\"Arachnodex {{domain}} Report [{{jobs}}]\"` | Subject template for report emails. Supports `{{domain}}` and `{{jobs}}`. |\n| `mail.developerRecipients` | `(string \\| object)[]` | `[]` | Developer recipients that can receive crawler reports. |\n| `mail.reportRecipients` | `(string \\| object)[]` | `[]` | Recipients for regular completion reports. |\n| `mail.errorRecipients` | `(string \\| object)[]` | `[]` | Recipients for accumulated error reports. |\n| `mail.from` | `string \\| object` | `{}` | Sender email string or name-to-email map. |\n| `mail.replyTo` | `(string \\| object)[]` | `[]` | Reply-to email strings or recipient maps. |\n\nMail transport settings are passed to Nodemailer.\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `mail.transport.host` | string | `\"\"` | SMTP host. |\n| `mail.transport.port` | number or string | `465` | SMTP port. |\n| `mail.transport.secure` | boolean | `true` | Use TLS for the SMTP connection. Local Mailpit/MailHog setups commonly use `false`. |\n| `mail.transport.auth.username` | string | `\"\"` | Optional SMTP username. |\n| `mail.transport.auth.password` | string | `\"\"` | Optional SMTP password. |\n| `mail.transport.tls.rejectUnauthorized` | boolean | `true` | Verify SMTP TLS certificates when TLS is used. |\n\n## Core Switches\n\nCore switches may be used before the first job:\n\n| Switch | Description |\n| --- | --- |\n| `-c <config-name>`, `--config=<config-name>` | Load `config/<config-name>.json`. Defaults to `default`; the `.json` suffix is optional. |\n| `-j <job-name\\|package>`, `--job=<job-name\\|package>` | Run an installed job package. `-j sitemap` loads `@arachnodex/job-sitemap`; `-j @scope/job-name` loads that exact scoped package; `-j npm:package-name` loads an exact unscoped package. |\n| `-h`, `--help` | Display help. Use after a job name for that job's help. |\n| `-m`, `--mute` | Mute crawler response status output by overriding `muteResponseStatus` for this run. Job output and errors still print. Alias: `--mute-status`. |\n| `-q`, `--quiet` | Mute all non-error output by overriding `muteAll` for this run, including job output. Legacy aliases: `-mm`, `--mute-all`. |\n| `-nc`, `--no-color` | Disable ANSI color output. |\n| `-nm`, `--no-mail` | Disable regular and error email reports for the run. |\n| `-t <count>`, `--threads=<count>` | Set the maximum worker thread count. |\n| `-v`, `--verbose` | Show summary crawl statistics at the end. |\n| `-vv` | Show full URL lists in crawl statistics. |\n| `-vvv` | Show sorted full URL lists in crawl statistics. |\n| `-p`, `--profile` | Print profiler milestones. |\n| `--test-report-email` | Render and send a test report email without crawling. |\n\n## Official Jobs\n\nOfficial jobs are regular job packages. They install by default through the create workflow, and they can also be installed or updated individually.\n\n### Sitemap\n\n`@arachnodex/job-sitemap` writes an XML sitemap from crawlable internal URLs found during a crawl. It can include canonical HTML pages and optionally include document URLs such as PDFs.\n\nRead the [Sitemap job README](packages/job-sitemap/README.md) for install notes, usage examples, switches, and full job config settings.\n\n### Link Issues\n\n`@arachnodex/job-link-issues` reports broken, malformed, non-canonical, insecure, placeholder, redirect, fragment, optional external-link issues, and optional asset-link issues for scripts, stylesheets, images, embeds, and nested same-site CSS/JS references. External checks use the bot protection heuristics package to avoid false positives from common WAF, CAPTCHA, and browser-challenge responses. Reports and copy/paste prompts include bounded source snippets when available, and query-string-only canonical mismatches are notice-level findings that can be suppressed with the job config.\n\nRead the [Link Issues job README](packages/job-link-issues/README.md) for install notes, usage examples, switches, finding severities, bot-protection behavior, and full job config settings.\n\n### NFA Report\n\n`@arachnodex/job-nfa-report` reports asset, media, and document references that are missing filename fingerprints or approved query-string cache-bust values. It scans common page markup references by default and can optionally scan same-site CSS/JS bodies for nested asset URLs.\n\nRead the [NFA Report job README](packages/job-nfa-report/README.md) for install notes, usage examples, switches, fingerprint rules, nested scanning behavior, ignore patterns, and full job config settings.\n\n### CSP Report (Beta)\n\n`@arachnodex/job-csp-report` reports observed content dependencies and generates Content Security Policy header directives for Apache, nginx, lighttpd, or raw header output. It scans crawled HTML plus nested same-site CSS/JS by default, emits report-only and enforce header variants, and reports inline usage or risky sources as hardening items.\n\nThis job is in beta and has not been fully tested. Review all generated directives carefully before deploying them.\n\nRead the [CSP Report job README](packages/job-csp-report/README.md) for install notes, switches, output formats, inline handling, nested scanning behavior, and full job config settings.\n\n## Updating a Created Project\n\nUpdate an existing project from inside its directory:\n\n```sh\nnpm create @arachnodex@latest -- --update\n```\n\nUpdate a project at another path by passing the directory explicitly:\n\n```sh\nnpm create @arachnodex@latest -- --update ../my-crawl-project\n```\n\nUpdate mode installs the current official Arachnodex packages, refreshes the generated `README.md` and `config/*.example.json` reference files, and preserves the existing project name, version, scripts, custom dependencies, and runtime `config/*.json` files. Commit or back up the project first if its generated README or example files contain local edits.\n\nTo refresh the generated files and dependency ranges without installing packages, add `--no-install` and run the printed install command later:\n\n```sh\nnpm create @arachnodex@latest -- --update --no-install\n```\n\n## Updating Individual Packages\n\nArachnodex is designed as a set of independently published npm packages. You can update the core, jobs, or shared heuristic data separately when new versions are available.\n\nUpdate the core:\n\n```sh\nnpm install @arachnodex/core@latest\n```\n\nUpdate one job:\n\n```sh\nnpm install @arachnodex/job-link-issues@latest\nnpm install @arachnodex/job-nfa-report@latest\nnpm install @arachnodex/job-sitemap@latest\nnpm install @arachnodex/job-csp-report@latest\n```\n\nUpdate bot protection heuristics:\n\n```sh\nnpm install @arachnodex/bot-protection-heuristics@latest\n```\n\nThe job packages use `@arachnodex/core` as a peer dependency so npm can warn when a job expects a different core range. The bot protection heuristics package is a required dependency of core and is re-exported from `@arachnodex/core` for jobs that need it.\n\n## Bot Protection Heuristics\n\nPackage:\n\n```text\n@arachnodex/bot-protection-heuristics\n```\n\nThis package contains marker lists used to detect common bot protection, WAF, CAPTCHA, and challenge/interstitial responses.\n\nThe official `link-issues` job uses these markers during external-link audits. If an external URL returns a response matching one of the bot-protection markers, the job reports a notice-level `external-bot-protection` finding instead of reporting a broken-link error. Unmatched network failures, DNS failures, timeouts, and ordinary error responses still report as external-link problems. That distinction matters most for CDNs, WAF-protected sites, and services that reject crawler-style HEAD requests but still work in a browser.\n\nKeeping the markers separate lets Arachnodex update bot-protection detection without requiring a full core crawler or job release. Updating `@arachnodex/bot-protection-heuristics` can improve how `link-issues` classifies those external responses.\n\nThe package exports:\n\n```ts\nimport {\n  botProtectionHeuristics,\n  type BotProtectionHeuristics\n} from \"@arachnodex/bot-protection-heuristics\";\n```\n\nMost users do not need to import it directly. Jobs can also consume the core re-export:\n\n```ts\nimport {botProtectionHeuristics} from \"@arachnodex/core\";\n```\n\n## Contributing\n\nContributions are welcome. This repo is a monorepo, so focused package-specific pull requests are easiest to review.\n\nRead [CONTRIBUTING.md](CONTRIBUTING.md) for the fork and pull request flow, package boundaries, build expectations, and required checks.\n","readmeFilename":"README.md"}