{"_id":"@cosmiclasagnadev/zmem","_rev":"2-f4bee63b526d7a3f85b2533a45bacfb9","name":"@cosmiclasagnadev/zmem","dist-tags":{"latest":"0.1.1"},"versions":{"0.1.0":{"name":"@cosmiclasagnadev/zmem","version":"0.1.0","keywords":["memory","mcp","vector-search","fts","sqlite","zvec","rag","local-first"],"license":"MIT","_id":"@cosmiclasagnadev/zmem@0.1.0","maintainers":[{"name":"aoponcedeleon","email":"allenpdl75@gmail.com"}],"homepage":"https://github.com/cosmiclasagnadev/zmem#readme","bugs":{"url":"https://github.com/cosmiclasagnadev/zmem/issues"},"bin":{"zmem":"dist/index.js"},"dist":{"shasum":"7473847fd1c50eb59b2c5b7468912aa08c6a97eb","tarball":"https://registry.npmjs.org/@cosmiclasagnadev/zmem/-/zmem-0.1.0.tgz","fileCount":153,"integrity":"sha512-581x5xMBbk23NE9Fxonl3Vo8d+A2s90LpLYLoAt9Pt+qxjwfYdIR2FuTm8FmCi7SKKoJE0tl5KkOdV68iK6i7w==","signatures":[{"sig":"MEQCIEnQfuX0Ez9/KGh2P1FolIUJlQjjHSGt4hMvYE3BQ7+rAiAioGBp3svMPDfdWi40Y7Zp3AdkUGBG+gtUfg/bxNJqUA==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":304465},"type":"module","engines":{"node":">=22"},"gitHead":"0c3bb172ea9c964c036248d996e2d5b9971fe762","private":false,"scripts":{"dev":"tsx src/index.ts","test":"tsx --test test/**/*.test.ts","build":"tsc -p tsconfig.build.json","smoke":"npm run build && tsx scripts/smoke.ts","start":"node dist/index.js","prepack":"npm run build","smoke:mcp":"npm run build && tsx scripts/smoke.ts --only=mcp","typecheck":"tsc --noEmit","smoke:core":"npm run build && tsx scripts/smoke.ts --only=core","test:lexical":"tsx --test test/lexical.sanitization.test.ts","smoke:verbose":"ZMEM_MCP_VERBOSE=true npm run smoke"},"_npmUser":{"name":"aoponcedeleon","email":"allenpdl75@gmail.com"},"repository":{"url":"git+https://github.com/cosmiclasagnadev/zmem.git","type":"git"},"_npmVersion":"10.4.0","description":"Local-first hybrid memory with zvec + FTS + MCP","directories":{},"_nodeVersion":"22.18.0","dependencies":{"zod":"^3.25.76","p-map":"^7.0.4","dotenv":"^16.6.1","fast-glob":"^3.3.3","@zvec/zvec":"^0.2.0","gray-matter":"^4.0.3","cli-progress":"^3.12.0","gpt-tokenizer":"^3.4.0","better-sqlite3":"^11.8.1","node-llama-cpp":"^3.0.0","@modelcontextprotocol/sdk":"^1.17.4"},"publishConfig":{"access":"public"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.20.3","typescript":"^5.8.3","@types/node":"^22.15.30","@types/better-sqlite3":"^7.6.13"},"_npmOperationalInternal":{"tmp":"tmp/zmem_0.1.0_1772122282242_0.9257554811438402","host":"s3://npm-registry-packages-npm-production"}},"0.1.1":{"name":"@cosmiclasagnadev/zmem","version":"0.1.1","private":false,"type":"module","description":"Local-first hybrid memory with zvec + FTS + MCP","license":"MIT","repository":{"type":"git","url":"git+https://github.com/cosmiclasagnadev/zmem.git"},"bugs":{"url":"https://github.com/cosmiclasagnadev/zmem/issues"},"homepage":"https://github.com/cosmiclasagnadev/zmem#readme","publishConfig":{"access":"public"},"keywords":["memory","mcp","vector-search","fts","sqlite","zvec","rag","local-first"],"engines":{"node":">=22"},"bin":{"zmem":"dist/index.js"},"scripts":{"dev":"tsx src/index.ts","build":"tsc -p tsconfig.build.json","start":"node dist/index.js","typecheck":"tsc --noEmit","test":"tsx --test test/**/*.test.ts","test:lexical":"tsx --test test/lexical.sanitization.test.ts","smoke":"npm run build && tsx scripts/smoke.ts","smoke:core":"npm run build && tsx scripts/smoke.ts --only=core","smoke:mcp":"npm run build && tsx scripts/smoke.ts --only=mcp","smoke:verbose":"ZMEM_MCP_VERBOSE=true npm run smoke","prepack":"npm run build"},"dependencies":{"@modelcontextprotocol/sdk":"^1.17.4","@zvec/zvec":"^0.2.0","better-sqlite3":"^11.8.1","cli-progress":"^3.12.0","dotenv":"^16.6.1","fast-glob":"^3.3.3","gpt-tokenizer":"^3.4.0","gray-matter":"^4.0.3","node-llama-cpp":"^3.0.0","p-map":"^7.0.4","zod":"^3.25.76"},"devDependencies":{"@types/better-sqlite3":"^7.6.13","@types/node":"^22.15.30","tsx":"^4.20.3","typescript":"^5.8.3"},"_id":"@cosmiclasagnadev/zmem@0.1.1","gitHead":"e30e9f3d23fcbe918992feccf703173e5851e6b6","_nodeVersion":"22.18.0","_npmVersion":"10.4.0","dist":{"integrity":"sha512-4GOOvV6q8pDSacXh4L6Oj4ygZBjSi4562jBHV2oWEzdk+XUHGlnbFJlcwyjJfZiXzXEoQQDlAYeUkPuy3fASRA==","shasum":"61290f9ae4736465e95e5a49f67001516f5dcf0c","tarball":"https://registry.npmjs.org/@cosmiclasagnadev/zmem/-/zmem-0.1.1.tgz","fileCount":209,"unpackedSize":626632,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEQCIDrPOQFoFbNPsZz23wpKwMi1FB+HGbJpmiX5yxDtaxYtAiA7SLUofHYOrQJ/gZ9n3+Cc33y7HC7/o2oIryRSBjA4fA=="}]},"_npmUser":{"name":"aoponcedeleon","email":"allenpdl75@gmail.com"},"directories":{},"maintainers":[{"name":"aoponcedeleon","email":"allenpdl75@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/zmem_0.1.1_1773656779507_0.23472093827236407"},"_hasShrinkwrap":false}},"time":{"created":"2026-02-26T16:11:22.109Z","modified":"2026-03-16T10:26:19.790Z","0.1.0":"2026-02-26T16:11:22.379Z","0.1.1":"2026-03-16T10:26:19.645Z"},"bugs":{"url":"https://github.com/cosmiclasagnadev/zmem/issues"},"license":"MIT","homepage":"https://github.com/cosmiclasagnadev/zmem#readme","keywords":["memory","mcp","vector-search","fts","sqlite","zvec","rag","local-first"],"repository":{"type":"git","url":"git+https://github.com/cosmiclasagnadev/zmem.git"},"description":"Local-first hybrid memory with zvec + FTS + MCP","maintainers":[{"name":"aoponcedeleon","email":"allenpdl75@gmail.com"}],"readme":"# zmem\n\nLocal-first hybrid memory for engineering workflows.\n\n`zmem` is a small POC/experiment inspired by QMD-style document search, built to explore whether we can get strong practical recall by combining:\n\n- dense retrieval via `@zvec/zvec`\n- lexical retrieval via SQLite FTS5/BM25\n- a simple fusion pipeline (RRF)\n- a local MCP server for coding-agent integration\n- graph data for relationships between memories (wip)\n\noverall it is a local-first, agent-oriented memory substrate for engineering decisions and evolving project context\n\n## Quickstart\n\nGet `zmem` running locally in a few minutes.\n\n### 1. Install\n\n```bash\nnpm install -g @cosmiclasagnadev/zmem\nzmem help\n```\n\nOr from this repo:\n\n```bash\nnpm install\nnpm run build\nnpm install -g .\n```\n\n### 2. Initialize a workspace\n\nCreate or update a starter config for the workspace you want to index:\n\n```bash\nzmem init --workspace=default --root=/absolute/path/to/your/project --yes\n```\n\nThis creates a config and sets up zmem to use user-scoped storage by default.\n\n### 3. Check setup\n\nValidate config, storage, and local model settings:\n\n```bash\nzmem doctor --workspace=default\nzmem models status\n```\n\n### 4. Prepare local query-expansion models\n\nzmem uses local-first query expansion by default.\n\nSee the exact model pull commands:\n\n```bash\nzmem models pull\n```\n\nThis will print commands for the default local models:\n\n- primary: `hf:mradermacher/qmd-query-expansion-qwen3.5-2B-GGUF:Q4_K_M`\n- fallback: `hf:mradermacher/qmd-query-expansion-qwen3.5-2B-GGUF:Q4_K_S`\n\nRun the printed `node-llama-cpp pull` commands, or let zmem download on first query.\n\n### 5. Ingest your docs\n\n```bash\nzmem ingest /absolute/path/to/your/project --workspace=default\n```\n\n### 6. Query your memory\n\n```bash\nzmem query \"database migration\" --workspace=default\nzmem query \"\" --mode=recent --workspace=default\nzmem query \"\" --mode=typed --types=decision,preference --workspace=default\n```\n\n### 7. Use with MCP / OpenCode\n\nStart the MCP server:\n\n```bash\nzmem mcp --workspace=default\n```\n\nThen point your MCP client, such as OpenCode, at the `zmem` command.\n\n### First-run notes\n\n- Default storage is outside your repo:\n  - macOS/Linux: `~/.local/share/zmem/workspaces/<workspace-slug>/`\n  - Windows: `%APPDATA%/zmem/workspaces/<workspace-slug>/`\n- Query expansion is enabled by default for hybrid retrieval.\n- If the local query-expansion model is unavailable, zmem warns once and falls back to deterministic expansion.\n- You can inspect the fully resolved config with:\n\n```bash\nzmem config show --workspace=default\n```\n\nReferences:\n- [QMD](https://github.com/tobi/qmd)\n- [ZVec](https://github.com/alibaba/zvec)\n\n## What this does differently\n- explicit memory types and lifecycle (pending, active, archived, deleted) plus supersedes_id\n  - if no explicit type is defined, it defaults to 'fact'\n  - if no explicit tags are defined, it defaults to '[]' \n- local-first query expansion is enabled by default for hybrid retrieval, with deterministic fallback when the local model is unavailable\n- uses `@zvec/zvec` for dense retrieval when doing vector search\n\n## Architecture (high level)\n\n- **Language/runtime:** TypeScript + Node.js\n- **Vectors:** `@zvec/zvec`\n- **Lexical:** `better-sqlite3` + FTS5/BM25\n- **Embeddings:** `node-llama-cpp` by default, with explicit provider hooks for Gemini and offline mock testing\n- **Validation:** `zod`\n- **Agent integration:** `@modelcontextprotocol/sdk`\n\nHybrid retrieval flow (follows qmd's style closely):\n\n1. lexical search (FTS/BM25)\n2. vector search (zvec)\n3. reciprocal rank fusion (RRF)\n4. bounded query expansion\n5. heuristic memory-item reranking\n\n### Indexing pipeline\n\n```mermaid\nflowchart LR\n    A[Markdown/Text Sources] --> B[Ingestion Pipeline]\n    B --> C[Parse + Chunk]\n    C --> D[(SQLite + FTS5)]\n    C --> E[Generate Embeddings]\n    E --> F[(zvec Vector Index)]\n```\n\n### Retrieval pipeline\n\n```mermaid\nflowchart LR\n    Q[User Query] --> M{Mode}\n\n    M -->|lexical| L[FTS/BM25 Search]\n    M -->|vector| V[Embedding + zvec Search]\n    M -->|hybrid| H[FTS + Vector in parallel]\n\n    L --> O[Top Results]\n    V --> O\n    H --> R[RRF Fusion]\n    R --> O\n\n    O --> C1[CLI Output]\n    O --> C2[MCP Tools]\n```\n\nAt runtime, the same core API powers both CLI commands and MCP tools, so behavior stays consistent across direct terminal use and agent workflows.\n\n## Project layout\n\n- `src/ingest/` - discovery, parsing, chunking, orchestration\n- `src/search/` - lexical/vector retrieval + fusion\n- `src/core/` - shared API (`save`, `recall`, `get`, `list`, `delete`, `reindex`, `status`)\n- `src/mcp/` - MCP server + tool registration\n- `src/db/` - SQLite handling + migrations\n- `config.example.json` - sample configuration\n\n## Getting started\n\n### 1) Install\n\n```bash\nnpm install\n```\n\n### 2) Configure\n\nCreate your local config file:\n\n```bash\ncp config.example.json config.json\n```\n\nThen update:\n\n- `workspaces[].root` to a real absolute path\n- any desired `patterns` for ingestion\n- storage paths (`storage.dbPath`, `storage.zvecPath`) if needed\n- `ai.embedding.provider` if you want to switch from local `llamacpp` to `gemini`\n- `ai.embedding.apiKey` / `ZMD_EMBED_API_KEY` when using `gemini`\n- `storage.baseDir` only if you want to override the default XDG-style storage root\n- `ai.queryExpansion.*` if you want to tune or disable default local-first query expansion\n\nIf `config.json` is missing, `zmem` falls back to defaults.\n\nDefault storage is outside your repo:\n\n- macOS/Linux: `~/.local/share/zmem/workspaces/<workspace-slug>/`\n- Windows: `%APPDATA%/zmem/workspaces/<workspace-slug>/`\n\nWithin each workspace directory, zmem stores:\n\n- `memory.db`\n- `vectors/`\n\nAdvanced overrides still work through `storage.baseDir`, `storage.dbPath`, `storage.zvecPath`, `ZMEM_STORAGE_BASE_DIR`, `ZMEM_DB_PATH`, and `ZMEM_ZVEC_PATH`.\n\nQuery expansion can also be tuned with env vars such as:\n\n- `ZMEM_QUERY_EXPANSION_ENABLED`\n- `ZMEM_QUERY_EXPANSION_PROVIDER`\n- `ZMEM_QUERY_EXPANSION_MODEL`\n- `ZMEM_QUERY_EXPANSION_FALLBACK_MODEL`\n- `ZMEM_QUERY_EXPANSION_MAX_EXPANSIONS`\n\n### 3) Ingest documents\n\n```bash\nnpm run dev -- ingest ./docs --workspace=default\n```\n\n### 4) Query memory\n\n```bash\nnpm run dev -- query \"database decisions\" --mode=hybrid --workspace=default\n```\n\n### 5) Check status\n\n```bash\nnpm run dev -- status --workspace=default\n```\n\n## CLI usage\n\nRun help:\n\n```bash\nnpm run dev -- help\n```\n\nPrimary commands:\n\n- `ingest <path> [--workspace=<name>] [--logs=true|false]`\n- `query <query> [--workspace=<name>] [--mode=hybrid|lexical|vector|recent|important|typed] [--scopes=a,b] [--types=a,b] [--expansion-mode=off|deterministic|llm] [--logs=true|false]`\n- `status [--workspace=<name>] [--logs=true|false]`\n- `init [--config=./config.json] [--workspace=<name>] [--root=<path>] [--storage-base-dir=<path>] [--enable-query-expansion=true|false] [--yes]`\n- `doctor [--config=./config.json] [--workspace=<name>]`\n- `config show [--config=./config.json] [--workspace=<name>]`\n- `models <status|check|pull> [--config=./config.json]`\n- `mcp [--config=./config.json] [--workspace=<name>] [--verbose=true|false]`\n\nExamples:\n\n```bash\nnpm run dev -- ingest ./test-docs/search --workspace=default\nnpm run dev -- query \"sqlite\" --mode=lexical --workspace=default\nnpm run dev -- query \"vector embeddings\" --mode=vector --workspace=default\nnpm run dev -- init --workspace=default --root=/absolute/path/to/repo --yes\nnpm run dev -- doctor --workspace=default\nnpm run dev -- models pull\n```\n\n## Local query expansion\n\nzmem now defaults to local-first query expansion for hybrid retrieval.\n\n- primary model: `hf:mradermacher/qmd-query-expansion-qwen3.5-2B-GGUF:Q4_K_M`\n- fallback model: `hf:mradermacher/qmd-query-expansion-qwen3.5-2B-GGUF:Q4_K_S`\n- expansion is skipped when lexical exact-match signal is already strong\n- if the local model is unavailable, zmem warns once and falls back to deterministic expansion\n\nTo inspect or prepare local models:\n\n```bash\nnpm run dev -- models status\nnpm run dev -- models pull\n```\n\n## MCP usage\n\nStart the MCP stdio server:\n\n```bash\nnpm run dev -- mcp --workspace=default\n```\n\nImplemented tools:\n\n- `memory_query`\n- `memory_search`\n- `memory_get`\n- `memory_list`\n- `memory_save`\n- `memory_delete`\n- `memory_status`\n- `memory_link`\n- `memory_neighbors`\n- `memory_edge_update`\n\nOptional admin tool:\n\n- `memory_reindex` (enabled with `ZMEM_ENABLE_REINDEX_TOOL=true`)\n\nVerbose MCP logs:\n\n```bash\nZMEM_MCP_VERBOSE=true npm run dev -- mcp --workspace=default\n```\n\n## Local development\n\nUseful scripts:\n\n- `npm run dev` - run CLI via `tsx`\n- `npm run build` - build TypeScript to `dist/`\n- `npm start` - run built CLI\n- `npm run typecheck` - type-check without emitting\n- `npm test` - run tests\n- `npm run smoke` - build + smoke script\n\nTypical dev loop:\n\n```bash\nnpm run typecheck\nnpm test\nnpm run smoke\n```\n\n## Environment variables\n\n- `ZMD_EMBED_MODEL` - override embedding model\n- `ZMD_EMBED_DIMENSIONS` - override embedding dimensions\n- `ZMD_EMBED_PROVIDER` - override embedding provider (`llamacpp`, `openai`, `ollama`, `gemini`, `mock`)\n- `ZMD_EMBED_API_KEY` - embedding API key override for remote providers such as Gemini\n- `ZMD_EMBED_BASE_URL` - optional embedding API base URL override\n- `ZMD_EMBED_TASK_TYPE` - optional Gemini task type override such as `RETRIEVAL_DOCUMENT`\n- `ZMEM_STORAGE_BASE_DIR` - override the XDG-style storage root\n- `ZMEM_DB_PATH` - override the resolved database path directly\n- `ZMEM_ZVEC_PATH` - override the resolved vector storage path directly\n- `ZMEM_WORKSPACE` - default workspace for MCP resolution\n- `ZMEM_MCP_VERBOSE=true` - verbose MCP logs to stderr\n- `ZMEM_ENABLE_REINDEX_TOOL=true` - expose `memory_reindex` MCP tool\n\n## Roadmap\n- [ ] graph traversal poc \n- [ ] Rust implementation and comparison with metrics\n- [ ] policy/compliance/audit/retention layer\n- [ ] added unit tests for error paths \n- [ ] improve batching controls and recall latency metrics\n- [ ] more integration ergonomics and ideas\n- [ ] reranking improvements (position-aware blend)\n- [ ] query expansion strategies\n- [ ] deeper retrieval tuning and eval harnesses\n","readmeFilename":"README.md"}