![\"OKDB](\"https://raw.githubusercontent.com/kedemd/okdb/main/banner.jpg\")
\n\n\n Getting Started ·\n Roles & Deployment ·\n Querying ·\n Indexes ·\n Full-Text Search ·\n Embeddings ·\n Sync ·\n Functions ·\n Pipelines ·\n HTTP API\n
\n\n---\n\n**OKDB** is an embedded, document-centric database for Node.js. No separate server. No network hop. Open a path, write data.\n\nIt wraps [LMDB](https://www.symas.com/lmdb) — one of the fastest memory-mapped key-value stores ever built — and layers on top: typed collections, composite secondary indexes, full-text search, vector embeddings, multi-node sync, a durable job queue, file storage, point-in-time time-machine queries, reactive materialized views, and a built-in HTTP API. All in one process. All ACID.\n\n```js\nconst OKDB = require('@kedem/okdb');\n\nconst db = new OKDB('./mydb');\nawait db.open();\n\nawait db.ensureType('users', {\n indexes: [['role'], ['email']],\n});\n\nawait db.put('users', 'alice', { name: 'Alice', role: 'admin', email: 'alice@example.com' });\n\nconst alice = db.get('users', 'alice'); // synchronous — instant\nconst admins = db.query('users', { role: 'admin' }); // MongoDB-style filter\n```\n\n---\n\n## Why OKDB?\n\nMost Node.js apps reach for a separate database server — Postgres, MongoDB, Redis — even when the workload fits comfortably in a single process. OKDB eliminates that infrastructure layer without sacrificing capability.\n\n| | OKDB | SQLite | MongoDB | Redis |\n| ------------------------ | :--: | :----: | :-----: | :---: |\n| Embedded (no server) | ✅ | ✅ | ❌ | ❌ |\n| Document model | ✅ | ❌ | ✅ | ❌ |\n| Composite indexes | ✅ | ✅ | ✅ | ❌ |\n| Full-text search | ✅ | ⚠️ | ✅ | ❌ |\n| Vector / semantic search | ✅ | ⚠️ | ✅ | ❌ |\n| Multi-node sync | ✅ | ❌ | ✅ | ✅ |\n| Durable job queue | ✅ | ❌ | ❌ | ✅ |\n| Synchronous reads | ✅ | ❌ | ❌ | ❌ |\n| ACID transactions | ✅ | ✅ | ✅ | ❌ |\n| Point-in-time queries | ✅ | ❌ | ⚠️ | ❌ |\n| Materialized views | ✅ | ⚠️ | ✅ | ❌ |\n\n---\n\n## Features\n\n### ⚡ Synchronous reads\n\nLMDB maps the database file directly into process memory. Reads are pointer dereferences — no `await`, no round-trip, no copy.\n\n### 🔒 ACID transactions\n\nInherited from LMDB's copy-on-write B+ trees with MVCC. Partial writes are impossible. Crash-safe by default.\n\n### 🗂️ Typed collections + composite indexes\n\nOrganise records into named types (like tables or collections). Declare composite secondary indexes and OKDB maintains them automatically inside every write transaction.\n\n```js\nawait db.ensureType('orders', {\n indexes: [\n ['status'],\n ['customerId'],\n ['customerId', 'createdAt'], // composite — fast filtered range scans\n ],\n});\n\n// Range scan by composite index\nfor (const { key, value } of db.byIndex('orders', ['customerId', 'createdAt'], {\n start: ['cust-1', 0],\n end: ['cust-1', Date.now()],\n})) {\n console.log(value);\n}\n```\n\n### 🔍 Full-text search\n\nBuilt-in inverted-index FTS backed by a dedicated LMDB environment. No Elasticsearch. No external process.\n\n```js\nawait db.fts.register('articles', 'content', { fields: ['title', 'body'] });\nawait db.fts.ready('articles', 'content');\n\nconst results = await db.fts.search('articles', 'content', 'hello world');\n```\n\n### 🤖 Vector embeddings + semantic search\n\nFirst-class embedding pipeline. Point it at a type and a field, connect an embedder (Ollama, OpenAI, or custom), and OKDB handles indexing automatically. Nearest-neighbour search uses an in-process HNSW graph.\n\n```js\nawait db.embeddings.createPipeline('articles', {\n source_type: 'articles',\n field: 'body',\n dims: 1024,\n embedder: { type: 'ollama', model: 'mxbai-embed-large', url: 'http://localhost:11434' },\n});\n\nconst similar = await db.embeddings.search('articles', queryVector, { limit: 10 });\n```\n\n### 🌍 Geo queries\n\nBuilt-in geospatial index for radius and bounding-box queries. See [docs/indexes.md](docs/indexes.md).\n\n### 🔄 Multi-node sync\n\nPeer-to-peer last-write-wins replication over HTTP. No coordinator. No Raft. Nodes exchange deltas; HLC timestamps resolve conflicts.\n\n```js\nconst db = new OKDB('./node1', {\n sync: { address: 'http://192.168.1.10:8080' },\n});\nawait db.open();\ndb.http.listen(8080);\n\n// Create an admin token first, then join a peer with it\nawait db.sync.join('http://192.168.1.11:8080', { token: 'cluster-bearer-token' });\n```\n\n### 📬 Durable job queue\n\nCrash-safe job queue stored in LMDB. Workers pull jobs, process them, and acknowledge — all without Redis.\n\n### 📁 File / blob storage\n\nSHA-256 content-addressable binary blob storage, integrated with the sync system. See [docs/files.md](docs/files.md).\n\n### 🧩 Sandboxed custom functions\n\nStore JavaScript functions in the database, validate them, and execute them in pooled sandbox runners.\n\n- all functions execute with env-local facades: `ctx.env`, `ctx.env.queue`, `ctx.env.files`\n- mark a function `unsafe: true` to additionally receive `ctx.okdb` for cross-env access\n- scripts use the contract `[async] (ctx) => {}`\n- `require`, `import`, and `console` are rejected\n- logging flows through `ctx.log(...)`\n\n```js\nawait db.env('analytics').functions.create({\n name: 'countEvents',\n source: '(ctx) => ({ count: ctx.env.getCount(\"events\") })',\n});\n\nconst run = await db.env('analytics').functions.run('countEvents');\nconsole.log(run.result.value.count);\n```\n\n### 🔌 Built-in HTTP API + Admin UI\n\nZero-config REST API and a browser-based admin UI for data exploration, index management, and real-time monitoring.\n\n```js\ndb.http.listen(8080);\n// REST API: http://localhost:8080/api/type/users\n// Admin UI: http://localhost:8080/admin/\n```\n\n### 🧠 MCP endpoint (Inspector-ready example)\n\nOKDB also exposes an MCP server over SSE + POST:\n\n- `GET /mcp/sse`\n- `POST /mcp/messages`\n\nFor a local working example, run:\n\n```bash\nnode examples/mcp-inspector.js\n```\n\nThen in MCP Inspector choose the SSE transport and connect to:\n\n- `http://localhost:8787/mcp/sse`\n\nThe server responds with browser CORS headers for common Inspector origins, accepts preflight `OPTIONS` requests, emits the POST endpoint URL over the SSE `endpoint` event, and exposes the MCP session id via the `mcp-session-id` response header. After connecting, try:\n\n- `tools/list`\n- `tools/call` with generated tools such as:\n - `system_info`\n - `system_runtime`\n - `queue_list_jobs`\n - `processor_list`\n\nOKDB now derives MCP tools primarily from route operation metadata instead of keeping a separate hand-maintained MCP tool list. Routes marked with canonical metadata and `mcp.include !== false` can participate in MCP generation, while streaming/SSE/raw transport routes stay excluded by default.\n\nIf you run Inspector from a different origin, allow that origin in `src/features/mcp/okdb-mcp.js`.\n\n### ⚙️ Reactive processors\n\nComposable change handlers — like database triggers, written in JavaScript. React to writes on any type, in the same transaction or asynchronously.\n\n### ⏱️ Time-machine\n\nFull edit history for every record. Ask what any document looked like at any point in the past — OKDB stores diffs automatically and reconstructs the state on demand.\n\n```js\n// Enable once; tracking starts from current clock\nawait db.timeMachine.enable();\n\n// Point-in-time read — synchronous, no await\nconst snapshot = db.timeMachine.getStateAt('orders', 'ord-42', historicClock);\n\n// Full history for a single record\nconst { diffs, head } = db.timeMachine.getHistory('orders', 'ord-42');\n\n// All changes to a type within a clock range — useful for auditing\nfor (const change of db.timeMachine.getChanges('orders', fromClock, toClock)) {\n console.log(change.key, change.diff);\n}\n```\n\n### 📊 Materialized views\n\nReactive aggregations that update automatically on every write. Declare a view with built-in reducers (`$count`, `$sum`, `$avg`, `$min`, `$max`, `$countBy`) or supply your own, and OKDB keeps the result in sync.\n\n```js\nconst env = db.env('default');\n\nawait env.views.create('orderStats', {\n type: 'orders',\n filter: { status: 'completed' },\n reduce: {\n total: { $sum: 'amount' },\n count: { $count: true },\n avgValue: { $avg: 'amount' },\n byRegion: { $countBy: 'region' },\n },\n});\n\n// Read the live result — synchronous, O(1)\nconst stats = env.views.get('orderStats');\n// → {\n// total: { value: 142500 },\n// count: { value: 312 },\n// avgValue: { value: 456.7 },\n// byRegion: { EU: { value: 140 }, US: { value: 172 } }\n// }\n```\n\nSee [docs/views.md](./docs/views.md) for the full reducer reference, `map`, `$ref` sub-aggregations, lifecycle management, and HTTP/MCP surface.\n\n### 🔑 Authentication & authorization\n\nMulti-mode security layer built into the HTTP API. No separate auth service needed.\n\n- **Token / session** — bearer tokens with configurable TTL\n- **OIDC / OAuth2** — discovery-based, JWKS validation, email/domain allowlists\n- **IP allowlists** — restrict access by network range\n\n---\n\n## Installation\n\n```bash\nnpm install okdb\n```\n\n> **Runtime:** Node.js only. Requires a native LMDB build (handled automatically by the `lmdb` package via node-gyp).\n\n---\n\n## Quick start\n\n```js\n'use strict';\nconst OKDB = require('@kedem/okdb');\n\nasync function main() {\n const db = new OKDB('./demo');\n await db.open();\n\n await db.ensureType('products', {\n indexes: [['category'], ['price'], ['category', 'price']],\n });\n\n await db.put('products', 'p1', { name: 'Widget', category: 'tools', price: 9.99 });\n await db.put('products', 'p2', { name: 'Gadget', category: 'tools', price: 19.99 });\n await db.put('products', 'p3', { name: 'Doohickey', category: 'misc', price: 4.99 });\n\n // MongoDB-style filter\n for (const { key, value } of db.query('products', { category: 'tools' })) {\n console.log(`${key}: ${value.name} — $${value.price}`);\n }\n\n // Index range scan\n for (const { key, value } of db.byIndex('products', ['price'], { start: [0], end: [10] })) {\n console.log(`${value.name} costs $${value.price}`);\n }\n\n await db.close();\n}\n\nmain().catch(console.error);\n```\n\n---\n\n## Documentation\n\n| Topic | |\n| ------------------------------------------ | ------------------------------------------------------------------- |\n| [Getting Started](docs/getting-started.md) | Installation, opening a database, first type, writes & reads |\n| [Querying](docs/querying.md) | Key reads, range scans, index scans, MongoDB filters |\n| [Indexes](docs/indexes.md) | Composite indexes, unique indexes, geo indexes |\n| [Transactions](docs/transactions.md) | Multi-write atomic transactions |\n| [Full-Text Search](docs/fts.md) | Registering FTS indexes, tokenizer config, search |\n| [Embeddings](docs/embeddings.md) | Vector pipeline, embedders, HNSW search |\n| [Sync](docs/sync.md) | Cluster setup, joining peers, conflict resolution |\n| [Queue](docs/queue.md) | Durable job queue, workers, retry |\n| [Files](docs/files.md) | Blob storage, upload, download, sync |\n| [Custom Functions](docs/functions.md) | Stored sandboxed JS functions, env/global execution, HTTP routes |\n| [Pipelines](docs/pipelines.md) | Composable data-flow orchestration with queue and processor drivers |\n| [HTTP API](docs/http-api.md) | REST endpoints, auth, admin UI |\n| [Auth](docs/auth-permissions.md) | Token, OIDC, IP allowlist authentication |\n| [Plugins](docs/plugins.md) | Extending OKDB with custom features |\n| [Migration](docs/migration.md) | Schema evolution, data migration |\n| [Architecture](OKDB-ARCHITECTURE.md) | Deep-dive technical reference |\n\n---\n\n## Use cases\n\n- **Local-first and offline-capable apps** — the database travels with the user\n- **Desktop / Electron apps** — no bundled server process\n- **Edge and IoT services** — minimal footprint, full capability\n- **Data pipelines** — fast ingest, durable queue, automatic vector indexing\n- **AI-powered apps** — embedding storage and semantic search are first-class\n- **Audit trails and compliance** — time-machine records every change; reconstruct any past state on demand\n- **Analytics dashboards** — materialized views keep aggregates live without separate query jobs\n\n---\n\n## License\n\nSee [LICENSE](LICENSE) for details.\n","readmeFilename":"README.md"}