{"_id":"@arc-lang/arc-search","name":"@arc-lang/arc-search","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@arc-lang/arc-search","version":"0.1.0","description":"Zero-dependency, SQLite FTS5-powered full-text search for Arc apps. BM25 ranking, prefix matching, highlighted excerpts.","main":"src/index.js","keywords":["arc","search","fts5","sqlite","full-text","bm25","algolia","meilisearch"],"license":"MIT","repository":{"type":"git","url":"git+https://github.com/arc-language/arc-web.git","directory":"packages/arc-search"},"homepage":"https://arc-language.dev/docs/search","bugs":{"url":"https://github.com/arc-language/arc-web/issues"},"engines":{"node":">=18.0.0"},"peerDependencies":{"@arc-lang/arc":">=0.2.0"},"peerDependenciesMeta":{"@arc-lang/arc":{"optional":true}},"publishConfig":{"access":"public"},"gitHead":"2258f6699081c9c3a9892bd9fe217cba8c6796f2","_id":"@arc-lang/arc-search@0.1.0","_nodeVersion":"24.15.0","_npmVersion":"11.12.1","dist":{"integrity":"sha512-mWaqpVOgm9xaro2yvqMWyP/LB7djYUzYXuxi/qBIspu41IQ+5yfSFQQKu1rOKoIrBaErIbHSV41gEMRzbLKn/Q==","shasum":"e2da8fd8c60a135462c51e30f26063f5a9b69d0e","tarball":"https://registry.npmjs.org/@arc-lang/arc-search/-/arc-search-0.1.0.tgz","fileCount":7,"unpackedSize":25399,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIFubtJ8j1k/2acwplS4QVR6Iw0QaO9axFAhQck9Bs/h8AiEAmsxcnZ9m3WalYo8TeAB2rZ3OQp1dhvfaohUshDBgpGY="}]},"_npmUser":{"name":"kobecuppens","email":"kobecuppens@hotmail.com"},"directories":{},"maintainers":[{"name":"kobecuppens","email":"kobecuppens@hotmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/arc-search_0.1.0_1780407181833_0.747643975830693"},"_hasShrinkwrap":false}},"time":{"created":"2026-06-02T13:33:01.643Z","0.1.0":"2026-06-02T13:33:02.005Z","modified":"2026-06-02T13:33:02.264Z"},"maintainers":[{"name":"kobecuppens","email":"kobecuppens@hotmail.com"}],"description":"Zero-dependency, SQLite FTS5-powered full-text search for Arc apps. BM25 ranking, prefix matching, highlighted excerpts.","homepage":"https://arc-language.dev/docs/search","keywords":["arc","search","fts5","sqlite","full-text","bm25","algolia","meilisearch"],"repository":{"type":"git","url":"git+https://github.com/arc-language/arc-web.git","directory":"packages/arc-search"},"bugs":{"url":"https://github.com/arc-language/arc-web/issues"},"license":"MIT","readme":"# arc-search\n\nZero-dependency, SQLite FTS5-powered full-text search for Arc apps. BM25 ranking, prefix matching, highlighted excerpts, and an inline search widget — all with no external services, no API keys, and no added dependencies.\n\n## Features\n\n- **FTS5 full-text search** — SQLite's built-in BM25 ranking engine. Sub-millisecond queries on 100k+ documents\n- **Prefix matching** — results appear as you type, no full word needed\n- **Highlighted excerpts** — match context with `<mark>` tags, powered by `fts5_snippet()`\n- **Type filtering** — index multiple content types, filter results by type\n- **`ArcSearchBar` widget** — drop-in inline search bar, ~4KB, zero runtime dependencies\n- **arc-cms integration** — upgrades the admin CMS search to FTS5 automatically when installed\n- **REST API** — index and query documents over HTTP from any language or tool\n- **Lazy init** — tables are created on first use, no migration step required\n\n## Features\n\n- **FTS5 full-text search** — SQLite's built-in BM25 ranking. Sub-millisecond queries on 100k+ documents\n- **`@searchable` decorator** — mark model fields; create/update/delete auto-sync the index. Zero boilerplate\n- **Bulk indexing** — batch up to 5000 documents in one transaction (`POST /arc-search/api/docs/bulk`)\n- **Prefix matching** — results appear as you type, no full word required\n- **Highlighted excerpts** — match context with `<mark>` tags via `fts5_snippet()`\n- **Type filtering in SQL** — O(1) predicate, no JS-side overfetch\n- **Trigram tokenizer** — opt-in typo/substring tolerance (`arc.config.json`)\n- **`ArcSearchBar` widget** — drop-in inline search bar, ~4KB, zero runtime dependencies\n- **AbortController** — cancels in-flight requests on new keystroke; no stale results\n- **arc-cms integration** — upgrades admin search to FTS5 automatically when installed\n- **Lazy FTS5 init** — tables created on first use; no migration step required\n\n## Install\n\n```bash\nnpm install @arc-lang/arc-search\n```\n\nAdd to `arc.config.json`:\n\n```json\n{\n  \"packages\": [\"@arc-lang/arc-search\"]\n}\n```\n\n## Quick start\n\n### 1. Add the widget to any page\n\n```arc\nimport ArcSearchBar from \"@arc-search/widgets/ArcSearchBar.arc\"\n\npage \"Search\"\n  ArcSearchBar(placeholder=\"Search docs...\")\n```\n\n### 2. Mark fields as searchable (auto-indexing)\n\nAdd `@searchable` to model fields. The compiler injects FTS5 index calls into every `create`, `update`, and `delete` — zero manual wiring:\n\n```arc\nmodel Post\n  @id let id: Int = autoincrement()\n  @searchable let title: String        // first @searchable field = title (boosted)\n  @searchable let body: String         // remaining fields = body content\n  let slug: String\n  let publishedAt: DateTime = now()\n```\n\nConfigure the result URL pattern in `arc.config.json`:\n\n```json\n{\n  \"packages\": [\"arc-search\"],\n  \"search\": {\n    \"models\": {\n      \"Post\": { \"url\": \"/blog/{slug}\" }\n    }\n  }\n}\n```\n\nThat's it. Every `db.posts.create(...)`, `db.posts.update(...)`, and `db.posts.delete(...)` automatically maintains the search index.\n\n### 3. (Alternative) Index manually via HTTP\n\nIf you can't use `@searchable` (e.g. external data), call the index endpoint:\n\n```arc\n@server fn createPost(title: String, body: String, slug: String) -> Any\n  const post = db.posts.create({ title, body, slug })\n  fetch(\"/arc-search/api/docs\", {\n    method: \"POST\",\n    headers: { \"Content-Type\": \"application/json\" },\n    body: JSON.stringify({\n      type: \"posts\",\n      ref:  String(post.id),\n      title: post.title,\n      body:  post.body,\n      url:   \"/blog/\" + post.slug\n    })\n  })\n  return post\n```\n\n### 4. Bulk index existing data\n\nUse the bulk endpoint to reindex large datasets in one transaction:\n\n```javascript\nfetch('/arc-search/api/docs/bulk', {\n  method: 'POST',\n  headers: { 'Content-Type': 'application/json' },\n  body: JSON.stringify({\n    docs: posts.map(p => ({\n      type: 'posts', ref: String(p.id),\n      title: p.title, body: p.body,\n      url: '/blog/' + p.slug\n    }))\n  })\n})\n// → { ok: true, created: 412, updated: 0, skipped: 0, total: 412 }\n```\n\nOr index directly with raw SQL in a server route:\n\n```arc\ndb.run(\"CREATE TABLE IF NOT EXISTS arc_search_docs (id INTEGER PRIMARY KEY AUTOINCREMENT, type TEXT NOT NULL, ref TEXT NOT NULL, title TEXT NOT NULL DEFAULT '', body TEXT NOT NULL DEFAULT '', url TEXT NOT NULL DEFAULT '', meta TEXT NOT NULL DEFAULT '{}', UNIQUE(type, ref))\", [])\ndb.run(\"INSERT OR IGNORE INTO arc_search_docs(type,ref,title,body,url) VALUES(?,?,?,?,?)\", [\"posts\", String(post.id), post.title, post.body, \"/blog/\"+post.slug])\n```\n\n### 3. Search\n\n```\nGET /arc-search/api/search?q=arc+framework&limit=10\n```\n\n```json\n{\n  \"results\": [\n    {\n      \"type\": \"posts\",\n      \"ref\": \"42\",\n      \"title\": \"Getting started with Arc\",\n      \"excerpt\": \"...build your first <mark>Arc</mark> <mark>framework</mark> app in minutes...\",\n      \"url\": \"/blog/getting-started\",\n      \"meta\": \"{}\",\n      \"score\": -1.234\n    }\n  ],\n  \"query\": \"arc framework\",\n  \"total\": 1\n}\n```\n\n## `ArcSearchBar` widget\n\n```arc\nArcSearchBar(\n  placeholder=\"Search...\"   // input placeholder text\n  types=\"posts,docs\"        // comma-sep type filter (optional, default: all)\n  minChars=2                // minimum chars before querying (default: 2)\n  limit=8                   // max results to show (default: 8, max: 50)\n  endpoint=\"/arc-search/api/search\"  // API endpoint (default)\n)\n```\n\nThe widget is self-contained: one `@raw` block with scoped CSS and ~600B of vanilla JS. No framework, no bundler, no external fonts. It uses CSS custom properties from your Arc theme automatically.\n\n## REST API\n\n### `GET /arc-search/api/search`\n\n| Param | Type | Description |\n|-------|------|-------------|\n| `q` | string | Search query (min 2 chars) |\n| `types` | string | Comma-separated type filter, e.g. `posts,docs` |\n| `limit` | number | Max results, 1–50 (default: 10) |\n\nReturns `{ results[], query, total }`. Each result: `{ type, ref, title, excerpt, url, meta, score }`.\n\n---\n\n### `POST /arc-search/api/docs`\n\nIndex or update a document.\n\n```json\n{\n  \"type\":  \"posts\",\n  \"ref\":   \"42\",\n  \"title\": \"Getting started with Arc\",\n  \"body\":  \"Full text content to index...\",\n  \"url\":   \"/blog/getting-started\",\n  \"meta\":  { \"author\": \"Kobe\", \"tags\": [\"arc\", \"web\"] }\n}\n```\n\n| Field | Required | Description |\n|-------|----------|-------------|\n| `type` | ✅ | Content type identifier (e.g. `posts`, `products`) |\n| `ref` | ✅ | Unique ID within the type |\n| `title` | ✅ | Primary display title, boosted in ranking |\n| `body` | — | Full text content to index |\n| `url` | — | Navigation URL shown in results |\n| `meta` | — | Any JSON — returned in results but not indexed |\n\nCalling this for an existing `(type, ref)` pair updates the document. Returns `{ ok: true, action: \"created\" | \"updated\" }`.\n\n---\n\n### `POST /arc-search/api/docs/bulk`\n\nUpsert up to 5000 documents in one SQLite transaction. Accepts the same fields as the single-doc endpoint, wrapped in a `docs` array.\n\n```json\n{ \"docs\": [{ \"type\": \"...\", \"ref\": \"...\", \"title\": \"...\", \"body\": \"...\", \"url\": \"...\" }, ...] }\n```\n\nReturns `{ ok, created, updated, skipped, total }`.\n\n---\n\n### `DELETE /arc-search/api/docs/:type/:ref`\n\nRemove a document from the index.\n\n```\nDELETE /arc-search/api/docs/posts/42\n```\n\nReturns `{ ok: true, action: \"deleted\" | \"not_found\" }`.\n\n## Typo tolerance (trigram tokenizer)\n\nBy default arc-search uses SQLite's `unicode61` tokenizer — exact matching with accent insensitivity. For typo-tolerant substring search (like Algolia), switch to `trigram`:\n\n```json\n{\n  \"search\": {\n    \"tokenizer\": \"trigram\"\n  }\n}\n```\n\n| Tokenizer | Typo tolerance | Index size | Best for |\n|-----------|---------------|-----------|---------|\n| `unicode61 remove_diacritics 1` | ❌ exact | 1× | dashboards, admin tools, dev docs |\n| `trigram` | ✅ substring + 1-char typos | ~3× | e-commerce, public search |\n\n> **Note:** Changing the tokenizer requires dropping and recreating the FTS5 table (`DROP TABLE arc_search_fts`) and reindexing. The `arc_search_docs` table is untouched.\n\n## arc-cms integration\n\nWhen `arc-search` is listed in `packages`, the CMS admin search automatically upgrades from client-side string filtering to FTS5-powered search. No configuration needed.\n\nTo keep the index fresh as editors create and update content, call the docs endpoint from your CMS server routes:\n\n```arc\n// After db.pages.create(...) or db.pages.update(...)\nfetch(\"/arc-search/api/docs\", {\n  method: \"POST\",\n  headers: { \"Content-Type\": \"application/json\" },\n  body: JSON.stringify({ type: \"pages\", ref: String(page.id), title: page.title, body: page.metaDescription ?? \"\", url: \"/admin/pages/\" + page.id })\n})\n```\n\nOr run a one-time reindex with the bulk endpoint pattern — query all records and POST each one.\n\n## How it works\n\narc-search uses two SQLite tables:\n\n```\narc_search_docs    — regular table storing all document fields\narc_search_fts     — FTS5 virtual table (external-content, backed by arc_search_docs)\n```\n\nThe FTS5 table uses `unicode61 remove_diacritics 1` tokenization for accent-insensitive matching and is kept in sync via explicit insert/delete operations on mutation. Queries use SQLite's native `bm25()` function for ranking and `snippet()` for highlighted excerpts — both run in the same SQLite process as your app with no network overhead.\n\nBoth tables are created lazily on first use (`CREATE TABLE IF NOT EXISTS`). No `arc db migrate` step is needed.\n\n## Performance\n\n| Documents | Query time | Memory |\n|-----------|-----------|--------|\n| 1,000 | < 0.1ms | ~500KB |\n| 10,000 | < 0.5ms | ~5MB |\n| 100,000 | < 2ms | ~50MB |\n\nBenchmarked on an M2 MacBook Pro with SQLite 3.43. FTS5 index size is approximately 10–20% of raw document size.\n\n## Security\n\nThe `/arc-search/api/docs` endpoint (POST and DELETE) has no built-in authentication. Protect it at the infrastructure level or add an auth guard in your Arc config:\n\n```arc\n// site/arc-search/server/docs.arc — override the package route\n@route @auth(admin) POST \"/arc-search/api/docs\" -> Response\n  // ... your custom handler, or just forward to the package implementation\n```\n\nOr set `ARC_SEARCH_KEY` in your environment and validate it in a middleware.\n\n## License\n\nMIT\n","readmeFilename":"README.md","_rev":"1-16d985982b27e35e92474807e4c27aa0"}