![\"npm](\"https://img.shields.io/npm/v/sxng-cli?style=flat-square&color=cb3837\")
\n \n \n ![\"npm](\"https://img.shields.io/npm/dm/sxng-cli?style=flat-square&color=cb3837\")
\n \n \n ![\"license\"](\"https://img.shields.io/github/license/hkwuks/sxng-cli?style=flat-square&color=green\")
\n \n ![\"node](\"https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen?style=flat-square\"){kind=icon}
\n\n A powerful command-line interface for SearXNG
\n Privacy-respecting web search from your terminal\n
\n \n
` | Language code (e.g., `en`, `zh`, `ja`) |\n| `--time ` | Time range: `day`, `week`, `month`, `year`, `all` |\n| `-f, --format ` | Output format: `md` (default) or `json` |\n| `--queries ` | Multi-query with RRF fusion (e.g., `q1,q2,q3`) |\n| `--session ` | Session directory or `new` for deep search |\n| `--owner ` | Session owner identifier |\n| `--desc ` | Session description |\n| `--redundancy ` | Query redundancy check: `warn`, `adjust`, `skip` |\n| `--quality` | Assess result quality (requires --session) |\n| `--approve-file ` | Approve `{id, revision}` selections from this session's `agent-inputs` directory |\n| `--skip-file ` / `--unskip-file ` | Skip or restore `{id, revision}` selections from this session's `agent-inputs` directory |\n| `--threshold-override ` | Override quality thresholds (JSON) |\n| `--merge ` | Merge new results with previous search JSON |\n\n### Examples\n\n```bash\n# Basic search (outputs Markdown by default)\nsxng \"machine learning\"\n\n# Output as JSON\nsxng --format json \"machine learning\"\n\n# Search with specific engines\nsxng --engines google,duckduckgo \"privacy tools\"\n\n# Search IT and Science categories\nsxng --categories it,science \"kubernetes tutorial\"\n\n# Limit results and filter by time\nsxng --limit 5 --time week \"latest AI news\"\n\n# Multi-query search with RRF fusion\nsxng --queries \"tokio tutorial,rust async basics,async-std guide\"\n\n# List available engines (fetched from server)\nsxng --engines-list\n```\n\n## ⚙️ Configuration\n\nConfiguration priority (highest to lowest):\n1. Environment variables\n2. Local config file (`./sxng.config.json`)\n3. Global config file (`~/sxng-cli/sxng.config.json`)\n4. Default values\n\n### Environment Variables\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `SEARXNG_BASE_URL` | SearXNG server URL | *(required)* |\n| `SEARXNG_DEFAULT_ENGINE` | Default search engine | *(none)* |\n| `SEARXNG_ALLOWED_ENGINES` | Comma-separated allowed engines | *(all)* |\n| `SEARXNG_DEFAULT_LIMIT` | Default result limit | `10` |\n| `SEARXNG_DEFAULT_FORMAT` | Default output format (`md` or `json`) | `md` |\n| `SEARXNG_USE_PROXY` | Use proxy (`true`/`false`) | `false` |\n| `SEARXNG_PROXY_URL` | Proxy URL | *(none)* |\n| `SEARXNG_TIMEOUT` | Request timeout in ms | `30000` |\n| `SEARXNG_REDUNDANCY_THRESHOLD` | Word-level Jaccard threshold for redundancy | `0.7` |\n| `SEARXNG_REDUNDANCY_BIGRAM_THRESHOLD` | Bigram-level Jaccard threshold (short queries) | `0.5` |\n| `OLLAMA_API_KEY` | [Ollama web search](https://ollama.com) API key (optional backend) | *(none)* |\n\n### Config File\n\nCreate `sxng.config.json`:\n\n```json\n{\n \"baseUrl\": \"http://localhost:8080\",\n \"defaultEngine\": \"\",\n \"allowedEngines\": [],\n \"defaultLimit\": 10,\n \"defaultFormat\": \"md\",\n \"useProxy\": false,\n \"proxyUrl\": \"\",\n \"timeout\": 30000,\n \"ollamaApiKey\": \"\",\n \"redundancyThreshold\": 0.7,\n \"redundancyBigramThreshold\": 0.5\n}\n```\n\n## 🧠 Deep Search\n\nDeep search enables multi-round iterative research with session accumulation, quality assessment, agent-controlled graph injection, and recovery strategies.\n\n### Workflow\n\n```\nSearch → Extract → Quality Assess → Approve → Build Graph → (Loop or Explore)\n ↓\n (L2/L3) Claim—Evidence—Review → Final Output\n```\n\n### Quick Example\n\n```bash\n# 1. Create a session and search\nsxng --session new --owner \"agent-1\" --desc \"Rust async study\" \"rust async ecosystem\"\n# Outputs session path, e.g. .sxng/sessions/ds_1712345678_abcdef\n\n# 2. Extract content from results\nsxng extract --session ds_1712345678_abcdef\n\n# 3. Check extraction output: stats.success and session.updated identify\n# results that received source text; failed URLs remain pending.\n\n# 4. Assess quality and see each result's verified state\nsxng --session ds_1712345678_abcdef --quality\n\n# 5. Copy selected {id, revision} objects from quality output into\n# .sxng/sessions/ds_1712345678_abcdef/agent-inputs/approve.json, then approve them.\nsxng --session ds_1712345678_abcdef --quality --approve-file .\\.sxng\\sessions\\ds_1712345678_abcdef\\agent-inputs\\approve.json\n\n# 6. Preprocess session content for entity discovery and provenance\nsxng graph-preprocess ds_1712345678_abcdef\n\n# 7. Put complex graph JSON in this session's Agent scratch directory, then add semantic edges\nsxng graph-add ds_1712345678_abcdef --data-file .\\.sxng\\sessions\\ds_1712345678_abcdef\\agent-inputs\\graph-data.json\n\n# 8. Continue research with redundancy check\nsxng --session ds_1712345678_abcdef --queries \"tokio vs async-std,benchmark 2026\" --redundancy warn\n```\n\n### Quality and Evidence Semantics\n\n- Extracted web content is deduplicated by normalized URL, then by full-text character 5-gram Jaccard similarity. Query redundancy uses a separate word-level or character-bigram Jaccard check.\n- `--quality` assesses the newest recorded round against earlier approved results. A URL already seen in an earlier round is non-novel, even when it reappears in the newest round.\n- Quality is a diagnostic, not a fact verifier. A search discovery can be approved only after `extract` writes a non-empty body and extraction timestamp. An explicitly imported external body must provide non-empty `content` and an `extractor`; absent `extractedAt` is recorded as its import time.\n- Claim policy treats two normalized publisher domains as two sources. It does not infer corporate ownership, editorial relationships, or syndication across domains.\n\n### Session Management\n\n| Command | Description |\n|---------|-------------|\n| `sxng --session new` | Create new auto-named session |\n| `sxng --session ` | Use session by name |\n| `sxng session-list` | List all sessions with stats |\n| `sxng session-delete ` | Delete specific session |\n| `sxng session-delete --older ` | Delete old sessions |\n\n**Session Path Resolution:**\n- Pure name (e.g., `my-session`) → `.sxng/sessions/my-session`\n- Full path (e.g., `/custom/path/session`) → used as-is\n- `new` → auto-generate unique name under `.sxng/sessions/`\n\n### Knowledge Graph\n\nTwo layers:\n\n**Structural** (auto-built):\n| Prefix | Type | Example |\n|--------|------|---------|\n| `q:` | Query node | `q:rust_async` |\n| `r:` | Result node | `r:https://example.com/page` |\n| `d:` | Domain node | `d:github_com` |\n\n**Semantic** (via `graph-add`):\n| Prefix | Type | Example |\n|--------|------|---------|\n| `e:` | Entity node | `e:tokio` |\n| `p:` | Path node | `p:chain_001` |\n\nGraph navigation commands: `graph-search` (discover entities), `graph-explore` (view relations), `graph-drill` (follow specific relations), `graph-traverse` (traverse reasoning paths).\n\n`graph-preprocess` returns `resultProvenance` (`id`, `revision`, `url`, `title`, `approval`) for every extracted body. For each semantic entity or edge, copy one or more currently approved `id` values into `sourceResultIds`. This keeps semantic facts traceable to the exact approved bodies rather than a search round.\n\n### External Search Results\n\nResults from other search tools (Tavily, Exa, etc.) can be injected into any active session via `results-add`, following the same pipeline as native sxng results:\n\n```powershell\n# Import search discovery; its excerpts remain summaries and require extraction.\nsxng results-add --kind search --tool exa --query \"async runtime\" `\n --data-file .\\.sxng\\sessions\\\\agent-inputs\\exa-search.json\n```\n\nSearch imports become `pending` and require `extract --session`; inspect `stats.success`, `stats.failed`, and `session.updated`, then approve only entries whose quality output reports `verified: true`. An external tool that already returned a body must instead use `--kind extracted` with non-empty `content` and `extractor`; it becomes pending approval without another extraction. A failed extraction remains pending and cannot enter the graph. `tool` and `query` record the discovery provenance.\n\n### Structured JSON Input\n\nAll structured writes use a UTF-8 JSON file under `.sxng/sessions//agent-inputs/`. This avoids PowerShell quoting and Windows command-line length limits, keeps concurrent sessions isolated, and gives each command one inspectable input artifact. The CLI rejects inline JSON and files outside the owning session.\n\n| Command | Required file option |\n|---|---|\n| External search or extracted bodies | `results-add --data-file ` |\n| Graph entities/edges | `graph-add --data-file ` |\n| Approval, skip, restore selections | `--approve-file` / `--skip-file` / `--unskip-file ` |\n| One/batch Claim | `claim-add --claim-file` / `--claims-file ` |\n| Evidence verification | `evidence-verify --evidence-file ` |\n\nThe file must be UTF-8 (a UTF-8 BOM is accepted). Missing, malformed, cross-session, or out-of-directory input fails before state changes. The CLI never deletes Agent input files automatically; `session-delete` removes the entire session only when explicitly requested.\n\n### Claim—Evidence—Review Pipeline (L2/L3 Only)\n\nAfter search, extraction, and graph building, the Agent can run the claim audit pipeline to verify individual statements before final output. CLI does the deterministic work (evidence anchoring, hash verification, Jaccard matching, policy rules); the Agent does all semantic reasoning (claim extraction, stance judgement, final output decisions).\n\n**Workflow:**\n\n```\nAgent writes draft → claim-add (auto evidence-search) → \nevidence-verify (with --complete auto-policy) → Review → Agent adjusts output\n```\n\n**Quick Example:**\n\n```powershell\n# 1. Put claims from your draft in the session's UTF-8 Agent input directory, then submit it.\nsxng claim-add my-session --claims-file .\\.sxng\\sessions\\my-session\\agent-inputs\\claims.json\n# → returns claims with IDs + auto-discovered evidence candidates\n\n# 2. Put {resultId, quote, charStart, charEnd} in the session's UTF-8 Agent input directory.\nsxng evidence-verify my-session --claim-id \"cl_001\" \\\n --evidence-file .\\.sxng\\sessions\\my-session\\agent-inputs\\evidence.json \\\n --stance support --reason \"Official docs confirm\" --complete\n# → returns evidence + verdict + review (decision: approved / needsReview)\n\n# 3. Check final reviews\nsxng review-list my-session\n```\n\n**Key Principles:**\n- **Deterministic checks**: CLI validates `resultId` against approved extracted results, UTF-16 offsets within range, and the SHA256 hash of the quoted source span\n- **No embedded LLM**: All semantic work (claim extraction, stance) is done by the Agent outside the CLI\n- **Policy rules**: 7 rules (singleRefute → highRiskInsufficient → dualSourceSupport → dualSourceMixed → singleSource → allInsufficient → fallback) — pure rule engine, no model calls\n- **`--complete` flag**: Evidence + verdict + auto-policy in one call\n\n| Commands | Purpose |\n|----------|---------|\n| `claim-add` | Submit claims (auto-triggers evidence search) |\n| `evidence-verify` | Confirm evidence + submit stance + optional auto-policy |\n| `policy-aggregate` | Manual re-aggregation |\n| `claim-list` / `evidence-list` / `verdict-list` / `review-list` | Query pipeline state |\n\nThe pipeline is **only available for L2/L3 deep search sessions** — L1 simple searches have no session or approved results pool.\n\n### Local Document Search\n\n`doc-index` and `doc-search` enable BM25 full-text search over local documents, with results flowing directly into the session pipeline:\n\n```bash\n# Index (auto-triggered by doc-search)\nsxng doc-index ./docs\n\n# Search and inject into session\nsxng doc-search \"query\" --path ./docs\n```\n\n**How it works:** auto-indexing on first use (Orama BM25, title×3 / headings×2 / content×1), each matched chunk is injected as an `extracted` result with `extractor: \"local-index\"`, then follows `--quality` → `--approve-file` → `graph-add`. Use `graph-preprocess` before adding new entities so approved IDs become `sourceResultIds`. Local searches do **not** increment the round counter.\n\n**Quality note:** Pure local search yields `sourceDiversity: 1`. Combine with web results for adequate diversity.\n\n### Known Limitations\n\n- Session result and graph mutations use a per-session lock plus atomic replacement; Claim, Evidence, and Review files still have no cross-file transaction.\n- URL extraction is not an SSRF security boundary; use only trusted public URLs.\n- Local document scanning has no defined symbolic-link boundary or cycle policy.\n- Index rebuilds overwrite the current persistence files; an interruption or disk failure can require re-indexing.\n\nSee [PRD-005 design improvements](docs/prds/PRD-005-design-improvements.md) for the deferred remediation plan.\n\n## 💡 Projects Using SXNG CLI\n\nSXNG CLI's deep search workflow (session accumulation, quality assessment, knowledge graph, claim–evidence–review) is used by projects that need persistent, multi-round research capabilities:\n\n- **[1052 OS](https://github.com/1052666/1052-OS)** — A personal AI operating system. Its `search-pack` integrates persistent research sessions, quality assessment, content extraction, and a claim–evidence–review pipeline, all powered by the same deep search concepts that drive SXNG CLI.\n\n> Open a PR to add your project here.\n\n## 🔗 Links\n\n- **GitHub:** https://github.com/hkwuks/sxng-cli\n- **npm:** https://www.npmjs.com/package/sxng-cli\n- **SearXNG:** https://github.com/searxng/searxng\n\n## 📄 License\n\nMIT\n","readmeFilename":"README.md"}