\r\n Deciding whether to adopt it? → · \r\n Want the mechanism walked through? → · \r\n Docs site → · \r\n Are you an LLM? Read llms.txt →\r\n
\r\n\r\n---\r\n\r\n## What it does, measured\r\n\r\nMeasured against a frozen copy of a **real work store — 29,853 engrams, 11,262 of them\r\nretrievable**, which is the honest denominator for a retrieval score. Not synthetic data, and\r\nnot a corpus we can publish: it is a real store with real colleagues and real decisions in it,\r\nso it is gitignored and so is everything derived from it.\r\n\r\nWhat you *can* run: **`npm run bench:public`** scores the same runner through the same\r\nground-truth derivation against a synthetic corpus you rebuild yourself. It verifies the method,\r\nnot the figures — a synthetic corpus scores lower. Current numbers, stamped with the commit that\r\nproduced them, are in [`docs/benchmarks-current.md`](docs/benchmarks-current.md); every claim\r\nbelow, what produced it, and **who can reproduce it** is in [`docs/claims.md`](docs/claims.md),\r\nincluding the ones we think are weakest.\r\n\r\n| | |\r\n|---|---|\r\n| **Finds the right memory** | In the top five for **98.7%** of identifier queries and **98.7%** of topic queries — and **first** for **93.3%** and **96.0%**. A recall returns three by default, so the top-five figure is what the agent gets; the first-result figure is what the ranker is held to. **Baselines on the same questions:** plain keyword search (BM25) puts the right memory first for 95.7% / 91.6% — ahead of AWM on identifier lookups, behind it on topic questions. These fixtures are keyword-friendly by construction. **On natural-language questions** (194, phrased the way a person asks) AWM puts the right memory first for **47.9%**, ahead of fused BM25 + vector search (45.4%) and plain BM25 (42.8%) on the same questions — and **52.6%** once memories carry the questions they answer (new in 0.22). Over the raw store, with superseded and retracted memories still in it, BM25 falls to 87.7% / 79.8% on the keyword fixtures and returns an outdated memory first 8–15% of the time; AWM's lifecycle is what it adds there |\r\n| **Stays silent when it should** | With the opt-in calibrated gate (`AWM_ABSTAIN_GATE=calibrated`), withholds **58 of 73** questions about facts never stored, while withholding **1 of 712** correct answers to keyword-shaped questions. Natural-language questions get their own model once a quarter of the store's memories carry questions: it withholds **10 of 97** correct natural answers (was 21 of 88 in 0.21), at the price of answering more questions whose fact was deleted. The gate learns its noise floor from the store itself; [how it works and what it costs](docs/abstention.md). Off (the default), AWM answers almost every question with its nearest match. |\r\n| **Costs the same at any scale** | Scoped recall stays flat as the store grows — published as **~630 tokens**, measured by the harness at ~1,518 ([claims §3](docs/claims.md) explains the gap). Carrying the store instead: ~1.3M tokens |\r\n| **Keeps secrets out** | On by default: credentials are masked before store, index, embedding or log. A read-only scan of the real store found them in ~70 memories written before this existed — exactly what it now stops. With the opt-in vault, a migration of a copy of that store sealed 40 distinct values and left **0** anywhere in the file, with recall unchanged (same first result for 8 of 8 test questions) |\r\n| **Lets a cheap model punch up** | Small model + AWM beat a frontier model on a 15-task domain workload: **14/15 vs 7/15**, at ~1/40th the cost |\r\n\r\nTwo independently built fixtures agree within a point. A warm recall takes under half a second\r\non the 30k-memory store (p50 375–438 ms, [benchmarks](docs/benchmarks-current.md)); smaller stores\r\nare faster. Those figures are for memories written the way\r\n[`docs/writing-memories.md`](docs/writing-memories.md) says to — the rules hold for any model\r\nthat writes to the store, and the write is where recall quality is decided.\r\n\r\n