{"_id":"@arachnodex/job-sitemap","_rev":"4-2d9ada6f765528071c6743ae4b9cadf4","name":"@arachnodex/job-sitemap","dist-tags":{"latest":"1.0.3"},"versions":{"1.0.0":{"name":"@arachnodex/job-sitemap","version":"1.0.0","keywords":["arachnodex","crawler","sitemap","sitemap generator","seo"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/job-sitemap@1.0.0","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"dist":{"shasum":"2379d50a17e59e6e56b3d7781f89898bbb15fa37","tarball":"https://registry.npmjs.org/@arachnodex/job-sitemap/-/job-sitemap-1.0.0.tgz","fileCount":11,"integrity":"sha512-sYnDn8hYGF1oUhA/efeDzE1podmECzAlLMX+mzT5T87z+xY22y7rWqDWTdHJ043iZsImUkvXy/sIRtSBcfxuHg==","signatures":[{"sig":"MEUCIGPM1e/6Ns0ze/ztBjvB7prI6XLP417qQWZrNPjg1FWoAiEApjg2g33GoL4Xm0sgyxZlDmjcsK0Nif9J8OzmA3LwiwE=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":41441},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./config/sitemap.example.json":"./config/sitemap.example.json"},"gitHead":"4d3eab58d0a7eb494435ad9e499a1cee2bb21300","scripts":{"build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --outfile=bin/index.js && npm run build:types","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/job-sitemap"},"_npmVersion":"11.13.0","description":"Arachnodex job for generating crawl sitemaps.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","fs-extra":"^11.3.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"esbuild":"^0.25.0","typescript":"^5.6.3","@types/node":"^22","@types/fs-extra":"^11.0.4","@arachnodex/core":"^1.0.0"},"peerDependencies":{"@arachnodex/core":"^1.0.0"},"_npmOperationalInternal":{"tmp":"tmp/job-sitemap_1.0.0_1780031421463_0.04187538227481591","host":"s3://npm-registry-packages-npm-production"}},"1.0.1":{"name":"@arachnodex/job-sitemap","version":"1.0.1","keywords":["arachnodex","crawler","sitemap","sitemap generator","seo"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/job-sitemap@1.0.1","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"dist":{"shasum":"f0bb56a3522f65820e341ec551189b5ae7b5d54d","tarball":"https://registry.npmjs.org/@arachnodex/job-sitemap/-/job-sitemap-1.0.1.tgz","fileCount":11,"integrity":"sha512-DySdzvjM9bsJEHpDHDi7UWT9rAZR3L8zhuYSM6+ypnyNPgSHXgSb3TL6BAUIDmx2uTwlfuEw3925x06F48mjuA==","signatures":[{"sig":"MEUCIQDEomqWq+TegnLjjUrxnM6hwOAeUWvL8owf/O4icXteIAIgMftcwRbWTfNnnTARGr0BP6gYSLfpLHagxWiDrw0A040=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":41441},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./config/sitemap.example.json":"./config/sitemap.example.json"},"gitHead":"3b454aa834f9977fcb5cadcb0f2b4481a5750f82","scripts":{"build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --outfile=bin/index.js && npm run build:types","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/job-sitemap"},"_npmVersion":"11.13.0","description":"Arachnodex job for generating crawl sitemaps.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","fs-extra":"^11.3.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"esbuild":"^0.25.0","typescript":"^5.6.3","@types/node":"^22","@types/fs-extra":"^11.0.4","@arachnodex/core":"^1.0.1"},"peerDependencies":{"@arachnodex/core":"^1.0.1"},"_npmOperationalInternal":{"tmp":"tmp/job-sitemap_1.0.1_1780034142421_0.16000084273731918","host":"s3://npm-registry-packages-npm-production"}},"1.0.2":{"name":"@arachnodex/job-sitemap","version":"1.0.2","keywords":["arachnodex","crawler","sitemap","sitemap generator","seo"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","_id":"@arachnodex/job-sitemap@1.0.2","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"homepage":"https://github.com/Arachnodex/arachnodex#readme","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"dist":{"shasum":"fd48d1b21b4dd7a34414d4c3cfaec2204e06f944","tarball":"https://registry.npmjs.org/@arachnodex/job-sitemap/-/job-sitemap-1.0.2.tgz","fileCount":11,"integrity":"sha512-wMzYq9T9Ng+Wcz3fS33QVx/PSe+dYQ+vTtASf+x5yD0pQCnG/15ga55Smu/pwVnQSjil39Q/vjJMlMFx2XZq4g==","signatures":[{"sig":"MEQCIE8dvGlMRsRUoGbtG/OyidmybtyrVRs9Zscltj53p5JDAiA/XfU8DWi+jRcYWuzXxcmBWs4ms6U1R2WcIX9rLj56dQ==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":41445},"main":"bin/index.js","type":"module","types":"types/index.d.ts","engines":{"node":">=22.13.0 <25"},"exports":{".":{"types":"./types/index.d.ts","default":"./bin/index.js","development":"./src/index.ts"},"./config/sitemap.example.json":"./config/sitemap.example.json"},"gitHead":"53c650ccf0a97f825b0c84bad88e15d023e850ad","scripts":{"build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --outfile=bin/index.js && npm run build:types","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","build:types":"tsc -p tsconfig.types.json","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\""},"_npmUser":{"name":"rickkukiela","email":"rick@belniakmedia.com"},"repository":{"url":"git+https://github.com/Arachnodex/arachnodex.git","type":"git","directory":"packages/job-sitemap"},"_npmVersion":"11.13.0","description":"Arachnodex job for generating crawl sitemaps.","directories":{},"_nodeVersion":"24.16.0","dependencies":{"axios":"^1.12.2","fs-extra":"^11.3.2"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"esbuild":"^0.25.0","typescript":"^5.6.3","@types/node":"^22","@types/fs-extra":"^11.0.4","@arachnodex/core":"^1.0.1"},"peerDependencies":{"@arachnodex/core":"^1.0.1"},"_npmOperationalInternal":{"tmp":"tmp/job-sitemap_1.0.2_1781641219252_0.8091913373822321","host":"s3://npm-registry-packages-npm-production"}},"1.0.3":{"name":"@arachnodex/job-sitemap","version":"1.0.3","description":"Arachnodex job for generating crawl sitemaps.","type":"module","main":"bin/index.js","types":"types/index.d.ts","exports":{".":{"types":"./types/index.d.ts","development":"./src/index.ts","default":"./bin/index.js"},"./config/sitemap.example.json":"./config/sitemap.example.json"},"repository":{"type":"git","url":"git+https://github.com/Arachnodex/arachnodex.git","directory":"packages/job-sitemap"},"keywords":["arachnodex","crawler","sitemap","sitemap generator","seo"],"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"homepage":"https://github.com/Arachnodex/arachnodex#readme","publishConfig":{"access":"public"},"scripts":{"test":"node --conditions=development --import tsx test/*.test.ts","clean:bin":"node -e \"const fs=require('node:fs'); fs.rmSync('bin',{recursive:true,force:true}); fs.mkdirSync('bin',{recursive:true});\"","clean:types":"node -e \"const fs=require('node:fs'); fs.rmSync('types',{recursive:true,force:true});\"","build:types":"tsc -p tsconfig.types.json","build":"npm run clean:bin && npm run clean:types && esbuild src/index.ts --bundle --platform=node --format=esm --target=node22 --minify --packages=external --outfile=bin/index.js && npm run build:types"},"dependencies":{"axios":"^1.20.0","fs-extra":"^11.3.2"},"peerDependencies":{"@arachnodex/core":"^1.0.10"},"devDependencies":{"@arachnodex/core":"^1.0.10","@types/fs-extra":"^11.0.4","@types/node":"^22","esbuild":"^0.25.0","typescript":"^5.6.3"},"engines":{"node":">=22.13.0 <25"},"gitHead":"56d075dd0cd42082f51c6aa5a85d4ebbe373178c","_id":"@arachnodex/job-sitemap@1.0.3","_nodeVersion":"24.16.0","_npmVersion":"11.19.1","dist":{"integrity":"sha512-LCto2xySTVWyeBjxedsWyhSvFjQXOa5+04nBW+TIPIQsJlvnxy5o3/e3U49DG3Gfw15guomPqkwPw4mY68bqww==","shasum":"040e5ff78ea1d5a0676ca685eb32bd7b92fc353e","tarball":"https://registry.npmjs.org/@arachnodex/job-sitemap/-/job-sitemap-1.0.3.tgz","fileCount":11,"unpackedSize":45312,"attestations":{"url":"https://registry.npmjs.org/-/npm/v1/attestations/@arachnodex%2fjob-sitemap@1.0.3","provenance":{"predicateType":"https://slsa.dev/provenance/v1"}},"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEYCIQDSA4Jb960adYL/RlNKYAes7SPyc0glm7rnu+1NlH1l6wIhAO2KhvF+CeSaGIH6EWZ3nn112xCNDZovChUAjkcLmSBN"}]},"_npmUser":{"name":"GitHub Actions","email":"npm-oidc-no-reply@github.com","trustedPublisher":{"id":"github","oidcConfigId":"oidc:30c1641b-9fc3-47e6-8661-1f86b6c3d285"}},"directories":{},"maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/job-sitemap_1.0.3_1788284427212_0.37410911908500566"},"_hasShrinkwrap":false}},"time":{"created":"2026-05-29T05:10:21.313Z","modified":"2026-09-01T17:40:27.666Z","1.0.0":"2026-05-29T05:10:21.617Z","1.0.1":"2026-05-29T05:55:42.552Z","1.0.2":"2026-06-16T20:20:19.439Z","1.0.3":"2026-09-01T17:40:27.346Z"},"bugs":{"url":"https://github.com/Arachnodex/arachnodex/issues"},"author":{"name":"Rick Kukiela","email":"rick@belniakmedia.com"},"license":"Apache-2.0","homepage":"https://github.com/Arachnodex/arachnodex#readme","keywords":["arachnodex","crawler","sitemap","sitemap generator","seo"],"repository":{"type":"git","url":"git+https://github.com/Arachnodex/arachnodex.git","directory":"packages/job-sitemap"},"description":"Arachnodex job for generating crawl sitemaps.","maintainers":[{"name":"rickkukiela","email":"rick@belniakmedia.com"}],"readme":"# @arachnodex/job-sitemap\n\nThe Sitemap job creates an XML sitemap from crawlable internal URLs discovered while Arachnodex spiders the configured site.\n\nIt can include normal HTML pages, optionally include document URLs such as PDFs, and optionally skip URLs that point at a different canonical target.\n\n## Install\n\nProjects created with `npm create @arachnodex` include this job by default. For a manual install, add it beside `@arachnodex/core`:\n\n```sh\nnpm install @arachnodex/job-sitemap\n```\n\nThe package uses `@arachnodex/core` as a peer dependency, so it should be installed in the same project as the crawler.\n\n## Usage\n\nRun the job with the default crawler config:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap\n```\n\nUse a job-specific config by placing `-c` after the job name:\n\n```sh\nnpm exec -- arachnodex -c default -j sitemap -c sitemap\n```\n\nThat loads the crawler config from `config/default.json` and the Sitemap job config from `config/sitemap.json`.\n\n## Config File\n\nThe package example config is available at:\n\n```text\nconfig/sitemap.example.json\n```\n\nA generated Arachnodex project copies this to:\n\n```text\nconfig/sitemap.json\n```\n\nFor a manual install, copy the example into your Arachnodex project's `config/` directory as `sitemap.json` before running the job.\n\nDefault config:\n\n```json\n{\n  \"includeOnlyCanonical\": true,\n  \"includeDocs\": true,\n  \"emailReportEnabled\": true,\n  \"outputFile\": \"../web/sitemap.xml\",\n  \"includeDocPattern\": \"((x-)?pdf)|(ms-?excel)|(vnd.)|(ms-?word)|(ms-?powerpoint)|(ms-?access)|(download)\"\n}\n```\n\n## Settings\n\n| Setting | Type | Default | Description |\n| --- | --- | --- | --- |\n| `includeOnlyCanonical` | boolean | `true` | Only include HTML page URLs when the page has no canonical URL or its canonical URL matches the current crawled URL. Set to `false` to include crawled page URLs even when they point at another canonical target. |\n| `includeDocs` | boolean | `true` | Include matching non-HTML document URLs discovered from successful response headers. |\n| `emailReportEnabled` | boolean | `true` | Include the Sitemap job summary in Arachnodex report emails. |\n| `outputFile` | string | `\"../web/sitemap.xml\"` | Path for the generated sitemap file, resolved from the directory where the crawler is run. The default assumes the Arachnodex project sits beside a website document root at `../web`. |\n| `includeDocPattern` | string | `\"((x-)?pdf)|(ms-?excel)|(vnd.)|(ms-?word)|(ms-?powerpoint)|(ms-?access)|(download)\"` | Regular expression used against the authoritative GET `content-type` when `includeDocs` is enabled, with matching HEAD values retained as a compatibility fallback. Matching URLs are written once as document entries. |\n\n## Output Path\n\n`outputFile` may point outside the Arachnodex project directory as long as the Node process has filesystem permission to write there. A common layout is:\n\n```text\nsite-audit/\nweb/\n```\n\nWith that layout, running Arachnodex from `site-audit/` and leaving `outputFile` as `../web/sitemap.xml` writes the public sitemap beside the audit project.\n\nThe job overwrites the configured output file at the end of each successful crawl. Temporary working files are run-specific and are cleaned up after the final sitemap is written.\n\n## Switches\n\n| Switch | Description |\n| --- | --- |\n| `-V`, `--version` | Print the Sitemap job version and exit without crawling. |\n","readmeFilename":"README.md"}