{"_id":"@bsbofmusic/cdper-runtime-reddit","_rev":"3-32231e191cf34cff4546f00b698a4e4b","name":"@bsbofmusic/cdper-runtime-reddit","dist-tags":{"latest":"1.1.3"},"versions":{"1.1.0":{"name":"@bsbofmusic/cdper-runtime-reddit","version":"1.1.0","keywords":["reddit","crawler","cdp","puppeteer","jtbd","openclaw"],"license":"MIT","_id":"@bsbofmusic/cdper-runtime-reddit@1.1.0","maintainers":[{"name":"bsbofmusic","email":"359633048@qq.com"}],"homepage":"https://github.com/bsbofmusic/openclaw/tree/main/cdp-reddit","bugs":{"url":"https://github.com/bsbofmusic/openclaw/issues"},"bin":{"cdp-reddit-jtbd":"cdp-reddit-jtbd.js","cdp-reddit-dedup":"cdp-reddit-dedup.js","cdp-reddit-export":"cdp-reddit-export.js","cdp-reddit-report":"cdp-reddit-report-gen.js","cdp-reddit-collect":"cdp-reddit-collect.js","cdp-reddit-keyword":"cdp-reddit-keyword.js","cdp-reddit-watcher":"cdp-reddit-watcher.sh","reddit-research-run":"reddit-research-run.js","cdp-reddit-freshness":"cdp-reddit-freshness.js","cdp-reddit-scheduler":"cdp-reddit-scheduler.js","reddit-research-plan":"reddit-research-plan.js","reddit-research-answer":"reddit-research-answer.js","reddit-research-export":"reddit-research-export.js","cdp-reddit-backfill-post":"cdp-reddit-backfill-post.js","cdp-reddit-jtbd-autonomous":"cdp-reddit-jtbd-autonomous.js"},"dist":{"shasum":"e42cddfda3685a2f4d473e0ddeef491edfe65171","tarball":"https://registry.npmjs.org/@bsbofmusic/cdper-runtime-reddit/-/cdper-runtime-reddit-1.1.0.tgz","fileCount":40,"integrity":"sha512-VGdKERWCYNAI403jCEH4APsA8aLoV9SdL5176ryas8IIowno2+rQrWmbB6LCntTaZmQh7pz5cD7FI+ZfJY0P/w==","signatures":[{"sig":"MEYCIQDOdhYs8wSVAwGOMQyiMGai+zW/ZlV8iS5mW+meCZ+40AIhAKcsaErlq2bI47AATuOFG7cgFalb2az/CxyWDNV2iCq5","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":256097},"main":"cdp-reddit-collect.js","engines":{"node":">=18.0.0"},"scripts":{"run":"node reddit-research-run.js","plan":"node reddit-research-plan.js","test":"echo \"No automated test suite yet; run the smoke checklist in README.md before release\" && exit 0","answer":"node reddit-research-answer.js","report":"node cdp-reddit-report-gen.js","collect":"node cdp-reddit-collect.js","watcher":"bash cdp-reddit-watcher.sh","schedule":"node cdp-reddit-scheduler.js","export:nl":"node reddit-research-export.js","smoke:install":"node -e \"const fs=require('fs'); const pkg=require('./package.json'); const bins=Object.values(pkg.bin); const miss=bins.filter(f=>!fs.existsSync(f)); if(miss.length){console.error('Missing bin targets:', miss.join(', ')); process.exit(1)} console.log('Install smoke OK:', bins.length, 'bin targets present')\"","watcher:example":"CDP_REDDIT_WATCHER_PROJECTS=example_project bash cdp-reddit-watcher.sh"},"_npmUser":{"name":"bsbofmusic","email":"359633048@qq.com"},"openclaw":{"cron":{"backup":{"note":"Placeholder metadata only; no backup script is shipped in this package","enabled":false,"schedule":"0 3 * * *"},"watcher":{"note":"Optional local cron suggestion; no installer registers this automatically","enabled":false,"schedule":"*/5 * * * *"},"jtbd-autonomous":{"note":"Optional local cron suggestion; enable only after validating the annotation flow","enabled":false,"schedule":"0 20 * * *"}},"entry":"cdp-reddit-collect.js","plugin":true,"runtime":"cdp","configEnv":["CDP_REDDIT_CDP_WS","CDP_REDDIT_DB_CONTAINER","CDP_REDDIT_DB_USER","CDP_REDDIT_DB_NAME","CDP_REDDIT_DEFAULT_SUBREDDIT","CDP_REDDIT_MAX_CDP_SESSIONS","CDP_REDDIT_WORKDIR","CDP_REDDIT_REPORT_FILE","CDP_REDDIT_WATCHER_STATE_FILE","CDP_REDDIT_WATCHER_MAX_IDLE_ROUNDS","CDP_REDDIT_WATCHER_MINUTES","CDP_REDDIT_WATCHER_PROJECTS"]},"repository":{"url":"git+https://github.com/bsbofmusic/openclaw.git","type":"git"},"_npmVersion":"11.9.0","description":"Reddit research runtime for cdper: collect, analyze, and export structured Reddit data","directories":{},"_nodeVersion":"24.14.0","dependencies":{"duckdb":"^1.1.3","exceljs":"^4.4.0","puppeteer-core":"^24.7.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"optionalDependencies":{"pg":">=8.0.0"},"_npmOperationalInternal":{"tmp":"tmp/cdper-runtime-reddit_1.1.0_1774506183492_0.5380164344097462","host":"s3://npm-registry-packages-npm-production"}},"1.1.1":{"name":"@bsbofmusic/cdper-runtime-reddit","version":"1.1.1","keywords":["reddit","crawler","cdp","puppeteer","jtbd","openclaw"],"license":"MIT","_id":"@bsbofmusic/cdper-runtime-reddit@1.1.1","maintainers":[{"name":"bsbofmusic","email":"359633048@qq.com"}],"homepage":"https://github.com/bsbofmusic/openclaw/tree/main/cdp-reddit","bugs":{"url":"https://github.com/bsbofmusic/openclaw/issues"},"bin":{"cdp-reddit-jtbd":"cdp-reddit-jtbd.js","cdp-reddit-dedup":"cdp-reddit-dedup.js","cdp-reddit-export":"cdp-reddit-export.js","cdp-reddit-report":"cdp-reddit-report-gen.js","cdp-reddit-collect":"cdp-reddit-collect.js","cdp-reddit-keyword":"cdp-reddit-keyword.js","cdp-reddit-watcher":"cdp-reddit-watcher.sh","reddit-research-run":"bin/reddit-research-run.cjs","cdp-reddit-freshness":"cdp-reddit-freshness.js","cdp-reddit-scheduler":"cdp-reddit-scheduler.js","reddit-research-plan":"bin/reddit-research-plan.cjs","reddit-research-answer":"bin/reddit-research-answer.cjs","reddit-research-export":"bin/reddit-research-export.cjs","cdp-reddit-backfill-post":"cdp-reddit-backfill-post.js","cdp-reddit-jtbd-autonomous":"cdp-reddit-jtbd-autonomous.js"},"dist":{"shasum":"0af77c2f395bfeb6e7e986af67b5286044b2dedb","tarball":"https://registry.npmjs.org/@bsbofmusic/cdper-runtime-reddit/-/cdper-runtime-reddit-1.1.1.tgz","fileCount":44,"integrity":"sha512-lUvxTYDfUeoRnGjEnxMPfF1R0z6jLfujIMpEYwnxa8OhCiQhk1y3V/nEWIl3mTx3vfdh8UIsAwfe4YuTk5sRsg==","signatures":[{"sig":"MEUCIQDyubXZzZ4UGq9yBsQjsnXX9yNsjc22/K/SZWEKBOnvyQIgcvHw2Wc89Na3VQLFVWyJpw396wHqOffhOQ0G9FodvpY=","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":256373},"main":"cdp-reddit-collect.js","engines":{"node":">=18.0.0"},"scripts":{"run":"node reddit-research-run.js","plan":"node reddit-research-plan.js","test":"echo \"No automated test suite yet; run the smoke checklist in README.md before release\" && exit 0","answer":"node reddit-research-answer.js","report":"node cdp-reddit-report-gen.js","collect":"node cdp-reddit-collect.js","watcher":"bash cdp-reddit-watcher.sh","schedule":"node cdp-reddit-scheduler.js","export:nl":"node reddit-research-export.js","smoke:install":"node -e \"const fs=require('fs'); const pkg=require('./package.json'); const bins=Object.values(pkg.bin); const miss=bins.filter(f=>!fs.existsSync(f)); if(miss.length){console.error('Missing bin targets:', miss.join(', ')); process.exit(1)} console.log('Install smoke OK:', bins.length, 'bin targets present')\"","watcher:example":"CDP_REDDIT_WATCHER_PROJECTS=example_project bash cdp-reddit-watcher.sh"},"_npmUser":{"name":"bsbofmusic","email":"359633048@qq.com"},"openclaw":{"cron":{"backup":{"note":"Placeholder metadata only; no backup script is shipped in this package","enabled":false,"schedule":"0 3 * * *"},"watcher":{"note":"Optional local cron suggestion; no installer registers this automatically","enabled":false,"schedule":"*/5 * * * *"},"jtbd-autonomous":{"note":"Optional local cron suggestion; enable only after validating the annotation flow","enabled":false,"schedule":"0 20 * * *"}},"entry":"cdp-reddit-collect.js","plugin":true,"runtime":"cdp","configEnv":["CDP_REDDIT_CDP_WS","CDP_REDDIT_DB_CONTAINER","CDP_REDDIT_DB_USER","CDP_REDDIT_DB_NAME","CDP_REDDIT_DEFAULT_SUBREDDIT","CDP_REDDIT_MAX_CDP_SESSIONS","CDP_REDDIT_WORKDIR","CDP_REDDIT_REPORT_FILE","CDP_REDDIT_WATCHER_STATE_FILE","CDP_REDDIT_WATCHER_MAX_IDLE_ROUNDS","CDP_REDDIT_WATCHER_MINUTES","CDP_REDDIT_WATCHER_PROJECTS"]},"repository":{"url":"git+https://github.com/bsbofmusic/openclaw.git","type":"git"},"_npmVersion":"11.9.0","description":"Reddit research runtime for cdper: collect, analyze, and export structured Reddit data","directories":{},"_nodeVersion":"24.14.0","dependencies":{"duckdb":"^1.1.3","exceljs":"^4.4.0","puppeteer-core":"^24.7.1"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"optionalDependencies":{"pg":">=8.0.0"},"_npmOperationalInternal":{"tmp":"tmp/cdper-runtime-reddit_1.1.1_1774508000095_0.37352539017573605","host":"s3://npm-registry-packages-npm-production"}},"1.1.3":{"name":"@bsbofmusic/cdper-runtime-reddit","version":"1.1.3","description":"Reddit research runtime for cdper: collect, analyze, and export structured Reddit data","keywords":["reddit","crawler","cdp","puppeteer","jtbd","openclaw"],"license":"MIT","publishConfig":{"access":"public"},"homepage":"https://github.com/bsbofmusic/openclaw/tree/main/cdp-reddit","repository":{"type":"git","url":"git+https://github.com/bsbofmusic/openclaw.git"},"engines":{"node":">=18.0.0"},"main":"cdp-reddit-collect.js","bin":{"reddit-research-plan":"bin/reddit-research-plan.cjs","reddit-research-run":"bin/reddit-research-run.cjs","reddit-research-answer":"bin/reddit-research-answer.cjs","reddit-research-export":"bin/reddit-research-export.cjs","cdp-reddit-collect":"cdp-reddit-collect.js","cdp-reddit-scheduler":"cdp-reddit-scheduler.js","cdp-reddit-watcher":"cdp-reddit-watcher.sh","cdp-reddit-keyword":"cdp-reddit-keyword.js","cdp-reddit-jtbd":"cdp-reddit-jtbd.js","cdp-reddit-jtbd-autonomous":"cdp-reddit-jtbd-autonomous.js","cdp-reddit-export":"cdp-reddit-export.js","cdp-reddit-freshness":"cdp-reddit-freshness.js","cdp-reddit-dedup":"cdp-reddit-dedup.js","cdp-reddit-report":"cdp-reddit-report-gen.js","cdp-reddit-backfill-post":"cdp-reddit-backfill-post.js"},"scripts":{"test":"echo \"No automated test suite yet; run the smoke checklist in README.md before release\" && exit 0","smoke:install":"node -e \"const fs=require('fs'); const pkg=require('./package.json'); const bins=Object.values(pkg.bin); const miss=bins.filter(f=>!fs.existsSync(f)); if(miss.length){console.error('Missing bin targets:', miss.join(', ')); process.exit(1)} console.log('Install smoke OK:', bins.length, 'bin targets present')\"","plan":"node reddit-research-plan.js","run":"node reddit-research-run.js","answer":"node reddit-research-answer.js","export:nl":"node reddit-research-export.js","collect":"node cdp-reddit-collect.js","schedule":"node cdp-reddit-scheduler.js","watcher":"bash cdp-reddit-watcher.sh","watcher:example":"CDP_REDDIT_WATCHER_PROJECTS=example_project bash cdp-reddit-watcher.sh","report":"node cdp-reddit-report-gen.js"},"dependencies":{"fflate":"^0.8.2","puppeteer-core":"^24.7.1"},"optionalDependencies":{"pg":">=8.0.0"},"openclaw":{"plugin":true,"runtime":"cdp","entry":"cdp-reddit-collect.js","configEnv":["CDP_REDDIT_CDP_WS","CDP_REDDIT_DB_CONTAINER","CDP_REDDIT_DB_USER","CDP_REDDIT_DB_NAME","CDP_REDDIT_DEFAULT_SUBREDDIT","CDP_REDDIT_MAX_CDP_SESSIONS","CDP_REDDIT_WORKDIR","CDP_REDDIT_REPORT_FILE","CDP_REDDIT_WATCHER_STATE_FILE","CDP_REDDIT_WATCHER_MAX_IDLE_ROUNDS","CDP_REDDIT_WATCHER_MINUTES","CDP_REDDIT_WATCHER_PROJECTS"],"cron":{"watcher":{"schedule":"*/5 * * * *","enabled":false,"note":"Optional local cron suggestion; no installer registers this automatically"},"jtbd-autonomous":{"schedule":"0 20 * * *","enabled":false,"note":"Optional local cron suggestion; enable only after validating the annotation flow"},"backup":{"schedule":"0 3 * * *","enabled":false,"note":"Placeholder metadata only; no backup script is shipped in this package"}}},"_id":"@bsbofmusic/cdper-runtime-reddit@1.1.3","bugs":{"url":"https://github.com/bsbofmusic/openclaw/issues"},"_nodeVersion":"24.14.0","_npmVersion":"11.9.0","dist":{"integrity":"sha512-Echi5obFwykvOGbvPz+HQv2jn2bEGe/20+6gwLDK+6Xk4jgqIBqOuyWZ2NUCyICtWHq4sEtSMjaTkgzAbTCz0w==","shasum":"4c6a142fbcf56cfd1a2f38247a1d46eb87fac4e4","tarball":"https://registry.npmjs.org/@bsbofmusic/cdper-runtime-reddit/-/cdper-runtime-reddit-1.1.3.tgz","fileCount":44,"unpackedSize":247759,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIF71q2mfAD2XLNFKD6OSUkdGlNM5G3EaeWaoH8MaG8KrAiAjx9eyiw4LUIe7gQy+g/zzY533B8aDeI/ZqwvVeE13bw=="}]},"_npmUser":{"name":"bsbofmusic","email":"359633048@qq.com"},"directories":{},"maintainers":[{"name":"bsbofmusic","email":"359633048@qq.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/cdper-runtime-reddit_1.1.3_1774515811301_0.8897575674030676"},"_hasShrinkwrap":false}},"time":{"created":"2026-03-26T06:23:03.440Z","modified":"2026-03-26T09:03:31.578Z","1.1.0":"2026-03-26T06:23:03.645Z","1.1.1":"2026-03-26T06:53:20.238Z","1.1.3":"2026-03-26T09:03:31.461Z"},"bugs":{"url":"https://github.com/bsbofmusic/openclaw/issues"},"license":"MIT","homepage":"https://github.com/bsbofmusic/openclaw/tree/main/cdp-reddit","keywords":["reddit","crawler","cdp","puppeteer","jtbd","openclaw"],"repository":{"type":"git","url":"git+https://github.com/bsbofmusic/openclaw.git"},"description":"Reddit research runtime for cdper: collect, analyze, and export structured Reddit data","maintainers":[{"name":"bsbofmusic","email":"359633048@qq.com"}],"readme":"# @bsbofmusic/cdper-runtime-reddit\n\n> Reddit research toolkit for authenticated CDP browser sessions.\n\n**Version:** 8.4.0 | **Runtime:** Node.js ≥18 | **Browser:** Puppeteer-compatible CDP endpoint\n\n---\n\n## Release posture\n\nThis package is usable for maintainer-operated deployments, but it is not a fully generalized product package yet.\n\nWhat is verified:\n- Live collection through a reachable authenticated CDP browser session\n- Database writes through `docker exec -i <container> psql ...`\n- Queue-based scheduling and npm bin packaging presence\n- A real smoke run on 2026-03-23 for project `piercing_pillow`\n\nWhat is not fully verified:\n- Automated tests are not shipped\n- Cross-environment install validation beyond the maintainer environment\n- Consistent `run_id` workflows across every downstream report/export/dedup path\n- Watcher / cron behavior as a polished end-user product feature\n\nDo not market this release as “production-ready for all environments”.\n\n---\n\n## Overview\n\n`@bsbofmusic/cdp-reddit` collects Reddit posts/comments through an authenticated CDP browser session, stores results in PostgreSQL, and supports a research workflow of collection → annotation → export.\n\nCurrent scope in this package:\n- Collection with nested comment traversal and Reddit-native `depth`\n- Queue-based scheduling with configurable CDP concurrency\n- Rule-based / semi-automatic JTBD annotation helpers\n- CSV / JSON export and data-quality utilities\n\nImportant current limitations:\n- `run_id` is written by collector / scheduler and schema support exists, but downstream tooling is still not fully normalized around `run_id`\n- Some utilities already share `cdp-reddit-config.js`; others still use older inline defaults and are not fully productized\n- Watcher / cron examples are operational recipes, not auto-installed package features\n\n---\n\n## Installation\n\n```bash\nnpm install @bsbofmusic/cdp-reddit puppeteer\n```\n\nThis package expects:\n- a reachable CDP browser WebSocket endpoint\n- a PostgreSQL instance accessible via `docker exec -i <container> psql ...`\n- an existing schema compatible with the shipped migration files\n\n---\n\n## CLI usage\n\nAll binaries are exposed through npm `bin` entries and can also be run with `node` inside the package directory.\n\n```bash\n# Register a new project\ncdp-reddit-collect \\\n  --register \\\n  --project=my_project \\\n  --label=\"Dog toy JTBD research\" \\\n  --keywords=\"dog toy,best chew toy\" \\\n  --sub=dogtoys \\\n  --threshold=15\n\n# Collect with stored project config\ncdp-reddit-collect --project=my_project\n\n# Collect with explicit version / run id\ncdp-reddit-collect \\\n  --project=my_project \\\n  --version=v1_research \\\n  --run-id=my_project_20260323_181500_a7f2 \\\n  --keywords='[\"dog toy\",\"chew toy\"]' \\\n  --threshold=15\n\n# Enqueue through the scheduler\ncdp-reddit-scheduler --enqueue --project=my_project --version=v1\n\n# Queue status\ncdp-reddit-scheduler --status\n\n# JTBD tagging\ncdp-reddit-jtbd --project=my_project --min-score=15\ncdp-reddit-jtbd-autonomous --project=my_project --batch-size=30\n\n# Export\ncdp-reddit-export --project=my_project --format=csv --min-score=10\ncdp-reddit-export --project=my_project --format=json --jtbd-only\n\n# Quality / maintenance\ncdp-reddit-freshness --project=my_project --threshold=10\ncdp-reddit-backfill-post --project=my_project\ncdp-reddit-dedup --project=my_project\ncdp-reddit-report --project=my_project --stats-only\n```\n\nIf you prefer direct `node` execution, run the corresponding `*.js` file in this directory.\n\n---\n\n## Configuration\n\n### Environment variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `CDP_REDDIT_CDP_WS` | none | CDP browser WebSocket endpoint; must be set for real collection |\n| `CDP_REDDIT_DB_CONTAINER` | `memos-postgres` | PostgreSQL Docker container name |\n| `CDP_REDDIT_DB_USER` | `memos` | DB username |\n| `CDP_REDDIT_DB_NAME` | `memos` | Database name |\n| `CDP_REDDIT_DEFAULT_SUBREDDIT` | `piercing` | Default subreddit when project config is absent |\n| `CDP_REDDIT_MAX_CDP_SESSIONS` | `2` | Scheduler concurrency cap |\n| `CDP_REDDIT_WORKDIR` | package dir | Scheduler working directory |\n| `CDP_REDDIT_REPORT_FILE` | `cdp-reddit-report.md` | Report append target |\n| `CDP_REDDIT_WATCHER_STATE_FILE` | `/tmp/.cdp_watcher_state` | Watcher state file |\n| `CDP_REDDIT_WATCHER_MAX_IDLE_ROUNDS` | `2` | Watcher completion threshold |\n| `CDP_REDDIT_WATCHER_MINUTES` | `30` | Watcher idle alert threshold |\n| `CDP_REDDIT_WATCHER_PROJECTS` | empty | Optional comma-separated project list for watcher summary output |\n\nNotes:\n- `cdp-reddit-collect.js`, `cdp-reddit-scheduler.js`, and `cdp-reddit-report-gen.js` already consume shared config from `cdp-reddit-config.js`\n- Other scripts still contain inline defaults and should be reviewed before broader packaging\n- Any built-in default CDP endpoint in source should be overridden in real deployments\n\n### Database connection\n\nCurrent implementation shells out to:\n\n```bash\ndocker exec -i <container> psql -U <user> -d <db>\n```\n\nThis package does not yet provide a direct TCP PostgreSQL client path for all scripts.\n\n---\n\n## Data model\n\n### Semantic layers\n\n| Field | Scope | Description |\n|-------|-------|-------------|\n| `project` | Research boundary | Data isolation key |\n| `version` | Batch label | Human-readable strategy / keyword-set label per run |\n| `data_source` | Compatibility layer | Currently equals `version`; stored in DB `data_source` column |\n| `run_id` | Single execution instance | Unique execution id written by collector / scheduler |\n\n### Current `run_id` status\n\n- ✅ `reddit_crawl_log.run_id` exists and is written by the collector\n- ✅ `reddit_comments.run_id` exists in schema and is written by the collector\n- ✅ `reddit_posts.run_id` exists in schema and is written by the collector\n- ⚠️ Report / export / dedup paths have `--run-id` support, but maintainers should still verify the exact output path they rely on before external release claims\n\n### Key tables\n\n| Table | Status |\n|-------|--------|\n| `reddit_comments` | Main collected comments table |\n| `reddit_posts` | Post metadata table |\n| `crawl_projects` | Registered projects and defaults |\n| `reddit_crawl_log` | Per-run execution log; authoritative source for current `run_id` metrics |\n\nSee `CHANGELOG_PHASE1.md` and `migrations/2026-03-23-add-run-id.sql` for the current schema transition notes.\n\n---\n\n## Included scripts\n\n| Script | Purpose |\n|--------|---------|\n| `cdp-reddit-collect.js` | Main collector |\n| `cdp-reddit-scheduler.js` | Queue / concurrency scheduler |\n| `cdp-reddit-watcher.sh` | Optional watchdog script for local ops use |\n| `cdp-reddit-keyword.js` | Keyword pre-analysis |\n| `cdp-reddit-jtbd.js` | Rule-based JTBD tagging |\n| `cdp-reddit-jtbd-autonomous.js` | Batch JTBD tagging |\n| `cdp-reddit-export.js` | CSV / JSON export |\n| `cdp-reddit-freshness.js` | Freshness detection |\n| `cdp-reddit-dedup.js` | Deduplication / version cleanup |\n| `cdp-reddit-report-gen.js` | Data quality report generator |\n| `cdp-reddit-backfill-post.js` | Historical post metadata backfill |\n\n---\n\n## Smoke checklist\n\nUse this before publishing or tagging a release.\n\n### 1) Install/package smoke\n\n```bash\nnpm install\nnpm run smoke:install\n```\n\nConfirms the declared npm bin targets exist.\n\n### 2) Environment smoke\n\nPrepare `.env` from `.env.example` and set a real CDP endpoint.\n\nMinimum required values:\n- `CDP_REDDIT_CDP_WS`\n- `CDP_REDDIT_DB_CONTAINER`\n- `CDP_REDDIT_DB_USER`\n- `CDP_REDDIT_DB_NAME`\n\n### 3) Live collector smoke\n\n```bash\ncd /path/to/cdp-reddit\nset -a\nsource ./.env\nset +a\n\nnode cdp-reddit-collect.js \\\n  --project=smoke_project \\\n  --version=smoke_v1 \\\n  --run-id=smoke_project_$(date +%Y%m%d_%H%M%S) \\\n  --keywords='[\"test keyword\"]' \\\n  --threshold=1\n```\n\nVerify all of the following manually:\n- CDP bridge is reachable\n- browser is actually ready, not just the bridge process\n- collector exits successfully\n- `reddit_crawl_log` contains the expected `run_id`\n- inserted rows exist in `reddit_comments` and/or `reddit_posts`\n\n### 4) Scheduler/report smoke\n\n```bash\nnode cdp-reddit-scheduler.js --status\nnode cdp-reddit-report-gen.js --project=smoke_project --stats-only\n```\n\n### 5) Optional watcher smoke\n\n```bash\nCDP_REDDIT_WATCHER_PROJECTS=\"smoke_project\" bash cdp-reddit-watcher.sh\n```\n\nTreat watcher output as ops assistance, not as release-grade verification by itself.\n\n---\n\n## Release steps\n\n1. Confirm package metadata in `package.json`\n2. Confirm `.env.example` matches the current env contract\n3. Run the smoke checklist above in the target environment\n4. Confirm required migrations are present and applied\n5. Review `CHANGELOG_PHASE1.md` so release notes match current facts\n6. Publish only after repository/homepage fields are filled with real values\n\nExample publish flow:\n\n```bash\nnpm pack\nnpm publish\n```\n\nOnly publish after maintainer review of metadata, access, and release notes.\n\n---\n\n## Operational notes\n\n### Watcher\n\n`cdp-reddit-watcher.sh` is included as an ops helper script. It is shipped as a CLI entry, but it should still be treated as environment-specific operations tooling rather than a polished general-purpose watcher.\n\nCurrent behavior to know before publishing:\n- it reads whole-table counts from `reddit_comments`\n- it can optionally print per-project summaries via `CDP_REDDIT_WATCHER_PROJECTS=\"proj_a,proj_b\"`\n- when no watcher project list is provided, it falls back to whole-table totals only\n- it does not prove cron is installed or that a deployment is self-managing\n\nExample:\n\n```bash\nCDP_REDDIT_WATCHER_PROJECTS=\"my_project,backup_project\" bash cdp-reddit-watcher.sh\n```\n\n### Cron metadata\n\n`package.json` contains cron metadata as suggestions only. They are not proof that cron jobs are installed in the current environment.\n\n### Live smoke result\n\nValidated on 2026-03-23 with a real `piercing_pillow` smoke run:\n- CDP bridge reachable but browser initially not ready\n- Browser successfully woken via `/control/start?mode=advanced&profile=Default`\n- Collector completed with `run_id = smoke_piercing_pillow_20260323_f6`\n- `reddit_crawl_log.comments_new = 3`\n- `reddit_comments` inserted rows for that run = `3`\n\n### Productization gaps\n\nBefore promoting this as a broadly reusable npm package, validate:\n- repository / homepage metadata\n- whether the default CDP endpoint should be removed from source defaults\n- whether watcher behavior should stay shipped, be generalized, or be documented as example ops tooling only\n\n---\n\n## Known limitations\n\n- No automated test suite is shipped yet\n- Some scripts still rely on environment-specific assumptions or legacy defaults\n- Direct PostgreSQL connectivity is not standardized across scripts\n- Release claims should stay limited to maintainer-verified workflows unless broader validation is completed\n\n---\n\n## Files\n\n- `README.md` — package overview, smoke checklist, and release posture\n- `package.json` — npm metadata and CLI entry points\n- `.env.example` — deploy-time environment template\n- `CHANGELOG_PHASE1.md` — current implementation status for Phase 1\n- `migrations/2026-03-23-add-run-id.sql` — current shipped migration\n","readmeFilename":"README.md"}