---
name: mongodb
description: "Use when modeling MongoDB documents (embed vs reference, the 16MB cap, bucket/subset/extended-reference patterns), choosing or fixing indexes (compound order via the ESR rule, partial/TTL/multikey, reading explain), writing aggregation pipelines that stay index-eligible ($match-first, $lookup cost, $group memory + allowDiskUse), running multi-document transactions with retry, or operating/securing a deployment (replica set, read/write concern, Atlas M0/Flex/dedicated tiers, Vector Search, Queryable Encryption). Triggers: 'slow Mongo query', 'add an index', 'COLLSCAN', 'embed or reference', 'aggregation pipeline', '$lookup', 'transaction keeps retrying', 'modela esta colección', 'mi consulta hace un full scan'. MongoDB 8.2, driver-agnostic (mongosh syntax). NOT relational schema/SQL/EXPLAIN ANALYZE (that is postgresdb)."
tags: [mongodb, nosql, database, aggregation, indexing]
recommends: [secure-coding, harness]
profiles: []
origin: risco
---

# MongoDB — modeling, indexing, aggregation, transactions, ops

Engine-level MongoDB 8.2 guidance: model documents for the queries you actually run, pick the
index the planner will use, write aggregation pipelines that stay index-eligible, run
multi-document transactions with correct retry, and operate/secure a deployment. Driver-agnostic —
every example is `mongosh` shell syntax that maps 1:1 to the official drivers (Node, Python, Go,
Java, Rust). This skill owns the **server's query and the index it picks**, not any ODM's API.

## When to use / When NOT to use

**When to use:**

- Document modeling: embed vs reference, the 16 MB cap, one-to-many/many-to-many, the
  subset/extended-reference/bucket/computed/outlier patterns, taming unbounded array growth.
- Index decisions: single-field, compound (the ESR ordering rule), multikey, partial, TTL, text,
  wildcard, `2dsphere`; and when an index is NOT worth it.
- Any query that is slow or scans too much; reading `explain("executionStats")`.
- Aggregation pipelines: stage order so `$match`/`$sort` hit an index, `$lookup` cost, `$unwind`
  explosion, `$group`/`$sort` memory limits and `allowDiskUse`, `$merge`/`$out`, faceting.
- Multi-document transactions: sessions, `withTransaction` retry semantics, read/write concern.
- Operating/securing: replica set, read preference, write concern, Atlas tier choice, Atlas Search
  & Vector Search, Queryable Encryption, role-based access, connection-pool knobs.

**When NOT to use:**

- Relational schema / SQL / `EXPLAIN ANALYZE` → [`postgresdb`](../postgresdb/SKILL.md). Different
  engine, planner, and concurrency model.
- ODM/driver API ergonomics (Mongoose pre-save hooks, the Node driver's `bulkWrite` return shape,
  `updateMany`'s result object) → that tool's own docs. This skill owns the server query and the
  index the server picks, not the JS object the driver hands back.
- App-layer caching as a product (Redis-in-front-of-reads).
- Cloud-console click-paths — we give the shell command / connection string, not the Atlas UI tour.
- Picking a vector store *across engines* (Pinecone vs Weaviate). Atlas Vector Search *inside* Mongo
  is in scope; cross-engine selection is not.

Deep dives: [data-modeling](references/data-modeling.md) (embed/reference tree, all six patterns,
16 MB math, polymorphic & schema versioning) · [aggregation](references/aggregation.md) (per-stage
index eligibility, `$lookup` variants, `$facet`, window fns, `$merge`/`$out`, reading pipeline
`explain`) · [transactions-and-ops](references/transactions-and-ops.md) (retry wrappers, concern
semantics, Atlas tiers, Vector Search, Queryable Encryption, RBAC, pooling, change streams).

## Non-negotiables

1. **Design for the queries you run, not the shape of your data.** The schema is the set of
   documents that make your common reads single-document and index-eligible.
2. **Never let an array grow unbounded inside a document.** It walks toward the 16 MB cap, bloats
   every read of the parent, and kills update performance — reference or bucket it.
3. **The hard ceiling is 16 MB per document.** If a one-to-many can exceed it, you reference; there
   is no `TOAST`-style overflow here.
4. **A single-document write is already atomic.** Reach for a multi-document transaction *only* when
   two or more documents must change together — otherwise you are paying for nothing.
5. **Every transaction retries on the `TransientTransactionError` label** (and commit retries on
   `UnknownTransactionCommitResult`). `withTransaction` does both for you; a hand-rolled loop must.
6. **Index by ESR: Equality fields, then the Sort field, then Range fields.** This order lets one
   compound index serve the filter, the sort, and the range without an in-memory sort.
7. **Read `explain("executionStats")` before and after adding an index** — confirm `IXSCAN`, not
   `COLLSCAN`, and `totalKeysExamined ≈ nReturned`. Or it didn't happen.
8. **`w:"majority"` for money and state transitions**, read concern `"majority"`/`"snapshot"` when a
   read must reflect a durable write. `w:1` can be rolled back on a primary failover.
9. **Money is `Decimal128` (`NumberDecimal("...")`), never a JS `double`.** Binary floats drift;
   `0.1 + 0.2 !== 0.3` in your ledger.
10. **Never store a secret in plaintext.** Use Queryable Encryption / client-side field-level
    encryption; never commit a `mongodb://user:pass@` literal.

## Decision rules

### Embed or reference

| Relationship | Choose | Why |
| --- | --- | --- |
| Read together, small, bounded (address on a user) | embed | one read, no `$lookup`, atomic update |
| One-to-few, bounded (≤ a few dozen, won't grow) | embed | stays well under 16 MB |
| One-to-many, growth not bounded (comments on a post) | reference | array would chase the 16 MB cap |
| Many-to-many (students↔courses) | reference (array of ids on the lighter side) | shared, independently mutated |
| Child shared across parents | reference | one source of truth, no duplication drift |
| Child independently and frequently mutated | reference | avoid rewriting a big parent per child edit |
| High-cardinality / huge child set | reference (+ optional subset embed) | keep the hot read small |

### Which schema pattern

| Symptom | Pattern | What it does |
| --- | --- | --- |
| List view reads 3 fields of a heavy doc | **subset** | embed only the hot fields, reference the rest |
| `$lookup` on every read just to show a name/price | **extended reference** | copy the few joined fields you display |
| Unbounded time-ordered events (readings, logs) | **bucket** | group N events per doc by time window |
| Same `count`/`sum` recomputed on every read | **computed** | store the rollup, update it on write |
| 1% of docs break the shape (a few mega-children) | **outlier** | flag them, overflow into linked docs |
| One collection holds several entity shapes | **polymorphic** | a `type` discriminator + shared `_id` space |

Full Bad→Good documents for each in [data-modeling](references/data-modeling.md).

### Which index type

| Access pattern | Index | Note |
| --- | --- | --- |
| `=` on one field | single-field | also covers the field's sort |
| filter + sort + range together | compound, ordered **ESR** | one index serves all three |
| query into an array field | multikey (automatic on an array key) | one multikey field per compound index |
| query only a subset of docs (`status:"active"`) | partial (`partialFilterExpression`) | smaller, cheaper to maintain |
| auto-expire docs after a time | TTL (`expireAfterSeconds` on a Date) | single-field only; deletes in background |
| language-aware text search | text **or** Atlas Search | Atlas Search is far richer; text is legacy |
| unpredictable / many query shapes on subdocs | wildcard (`"$**"`) | last resort; never beats a targeted index |
| geospatial proximity / within | `2dsphere` | GeoJSON `Point`/`Polygon` |
| vector similarity (8.2, Community+) | Atlas/Vector Search index | see transactions-and-ops ref |

### When NOT to add an index

- Low-cardinality field (a boolean, a 3-value status) — the planner skips it; `COLLSCAN` wins.
- Tiny collection — a collection scan reads one or two pages; the index is pure write tax.
- A field already the **left prefix** of an existing compound index — redundant.
- Write-heavy field rarely filtered — every index is paid on every insert/update.
- "Just in case" indexes — an unused index costs writes and RAM, returns nothing.

## Copy-paste patterns

Every fence is `mongosh` syntax.

### Model the document for the read (Bad → Good)

```javascript
// BAD: comments embedded in the post — array grows without bound toward 16 MB,
// every post read drags the entire comment history, money is a float.
db.posts.insertOne({
  _id: ObjectId(),
  title: "Indexing 101",
  authorId: ObjectId(),
  price: 9.99,                       // double — drifts in arithmetic
  comments: [ /* ...unbounded... */ ] // chases the 16 MB cap
})

// GOOD: post stays small; comments referenced; money is Decimal128;
// the few fields the feed needs are duplicated (extended reference).
db.posts.insertOne({
  _id: ObjectId(),
  title: "Indexing 101",
  author: { _id: ObjectId(), name: "Ada" }, // extended ref: name shown without a $lookup
  price: NumberDecimal("9.99"),
  commentCount: 0,                            // computed rollup, bumped on write
  createdAt: new Date()
})
db.comments.insertOne({ _id: ObjectId(), postId: ObjectId(), body: "…", createdAt: new Date() })
```

### Compound index in ESR order + the query that uses it

```javascript
// Feed query: filter by author (equality), sort by date (sort), bound by a date (range).
// ESR => author first, then the sort/range key.
db.posts.createIndex({ "author._id": 1, createdAt: -1 })

db.posts.find({ "author._id": authorId, createdAt: { $gte: since } })
        .sort({ createdAt: -1 })
        .limit(20)
// Confirm the plan: IXSCAN on the index above, no in-memory SORT stage.
```

### Partial + TTL indexes

```javascript
// Partial: index only the rows you actually query (active orders), not the archive.
db.orders.createIndex(
  { customerId: 1, createdAt: -1 },
  { partialFilterExpression: { status: "active" } }
)

// TTL: expire sessions 30 minutes after lastSeen. Field MUST be a Date.
db.sessions.createIndex({ lastSeen: 1 }, { expireAfterSeconds: 1800 })
```

### Aggregation: `$match` first, `$lookup`, `$group` with `allowDiskUse`

```javascript
db.orders.aggregate([
  // $match FIRST so it uses the compound index and shrinks the working set early.
  { $match: { status: "paid", createdAt: { $gte: since } } },
  { $sort:  { createdAt: -1 } },                  // index-eligible here, before any $group/$project
  { $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customer",
      pipeline: [ { $project: { name: 1 } } ]     // project inside $lookup: pull only what you need
  }},
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } }
], { allowDiskUse: true })  // $group/$sort spill past 100 MB/stage; this lets large groups complete,
                            // it is NOT a substitute for a missing $match index — see anti-patterns.
```

### Read `explain("executionStats")` — the four numbers

```javascript
db.posts.find({ "author._id": authorId }).sort({ createdAt: -1 })
        .explain("executionStats")
```

Read these before declaring a fix:

1. **`winningPlan.stage`** — must be `IXSCAN` (or `FETCH`→`IXSCAN`), not `COLLSCAN`.
2. **`totalKeysExamined` vs `nReturned`** — close means the index is selective; a huge ratio means
   the index scans far more than it returns (wrong key order, low selectivity).
3. **A `SORT` stage** — an in-memory sort the index should have satisfied; reorder by ESR to remove it.
4. **`rejectedPlans`** — what the planner considered and dropped; a near-miss hints at a better index.

### Multi-document transaction with full retry

```javascript
// Use withTransaction — it retries the body on TransientTransactionError and retries the
// commit on UnknownTransactionCommitResult for you. Requires a replica set / sharded cluster.
const session = db.getMongo().startSession();
try {
  session.withTransaction(() => {
    const orders  = session.getDatabase("shop").orders;
    const ledger  = session.getDatabase("shop").ledger;
    orders.updateOne({ _id: orderId, status: "pending" }, { $set: { status: "paid" } }, { session });
    ledger.insertOne({ orderId, amount: NumberDecimal("9.99"), at: new Date() }, { session });
  }, { readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } });
} finally {
  session.endSession();
}
// If both writes target ONE document, drop the transaction — that write is already atomic.
```

### `bulkWrite` upsert

```javascript
db.inventory.bulkWrite([
  { updateOne: {
      filter: { sku: "ABC-1" },
      update: { $inc: { qty: 5 }, $setOnInsert: { createdAt: new Date() } },
      upsert: true
  }}
], { ordered: false })  // ordered:false keeps going past one failed op and parallelizes
```

### Change stream (resumable tail)

```javascript
// Watch only the events you care about; persist resumeToken to restart without gaps.
const cs = db.orders.watch([{ $match: { operationType: { $in: ["insert", "update"] } } }]);
while (cs.hasNext()) { const change = cs.next(); /* process; save change._id as resume token */ }
```

More variants (`$facet`, window functions, `$merge`/`$out`, vector search) live in the references.

## Anti-patterns / rationalizations → STOP

| Rationalization | Reality → STOP |
| --- | --- |
| "Embed all the comments, it's one read" | Array grows unbounded toward 16 MB and bloats every post read. Reference or bucket. |
| "`$lookup` is just a JOIN, use it everywhere" | Mongo is not relational; per-document `$lookup` is expensive. Prefer modeling (extended reference) so the read needs no join. |
| "Wrap this single-document update in a transaction to be safe" | A single-doc write is already atomic. The transaction adds latency and a replica-set requirement for zero gain. |
| "`COLLSCAN` is fine, it's fast on my 100 docs" | It is O(n); at 4M docs it is a full table read. Add the index now and prove `IXSCAN`. |
| "Set `allowDiskUse:true` and the slow pipeline is fixed" | That masks a missing `$match` index by spilling to disk. Fix stage order / add the index first. |
| "Store the price as a number, round on display" | JS doubles drift across `$sum`/`$inc`. Use `NumberDecimal` (`Decimal128`). |
| "Group the whole collection, no `$match`" | A blocking `$group` over everything blows the 100 MB/stage limit. `$match` first to shrink it. |
| "One collection for users, orders, logs — fewer to manage" | Mixed shapes kill index selectivity and balloon working set. Split by access pattern. |
| "`$where` lets me run a quick JS predicate" | Runs JS per document, no index, a server-side injection surface. Use query operators / `$expr`. |
| "Index every field just in case" | Each index is a write tax and RAM cost; unused indexes return nothing. Index for real query shapes only. |

## Quick reference

### Read/write concern matrix

| Need | Write concern | Read concern | Note |
| --- | --- | --- | --- |
| Money / state transition | `w:"majority"` | `"majority"` | survives a primary failover |
| Read your own durable write | `w:"majority"` | `"majority"` (+ causal session) | no rollback window |
| Transaction default | `w:"majority"` | `"snapshot"` | consistent point-in-time |
| Logs / fire-and-forget | `w:1` | `"local"` | fast, may be rolled back |

### Atlas tier chooser

| Tier | Use it for | Limits |
| --- | --- | --- |
| **M0** | learning, tiny prototypes | free forever, up to 5 GB, shared, no SLA |
| **Flex** (GA Feb 2025) | small prod / variable load | $8 base capped at $30/mo, 100 ops/sec (burst 500), 5 GB; supports Atlas Search, Vector Search, Change Streams, Triggers |
| **M10+** (dedicated) | production, isolation, scale-up | from ~$0.08/hr (~$57/mo); dedicated resources, full features |

M0 does **not** run Vector Search well for real workloads — move to Flex or dedicated. Legacy
Serverless / M2 / M5 were auto-migrated to Flex.

### Aggregation memory

Each blocking stage (`$group`, `$sort` without an index, `$bucket`) is capped at **100 MB**. Past it
the stage errors unless `allowDiskUse:true` lets it spill. Spilling is a correctness fallback for
genuinely large groups, not a performance fix for a missing index.

## Verify

Run `scripts/verify.sh` from your project root. It is read-only, never connects to a database, and
never writes. It scans discovered `.js`/`.mongodb.js` files and flags foot-guns: a committed
plaintext `mongodb://user:pass@` credential (the **only** hard failure), `createIndex` calls with no
options, redundant compound-index prefixes, `$where` predicates, unbounded `$lookup`,
`allowDiskUse:true` that may be masking a missing index, and money stored as a JS number in seed
scripts. If `node` is present it runs `node --check` for a syntax pass; otherwise that step is
`[skip]`. Everything except a committed credential is advisory `[warn]`/`[skip]`. It runs on stock
macOS bash 3.2 and exits 0 on a clean or empty target.

## Project grounding (02-DOCS + CLAUDE.md)

When this skill runs in a project with a `02-DOCS/` layer (the [`harness`](../harness/SKILL.md)
Karpathy wiki), record this project's MongoDB decisions there and index them from the root
`CLAUDE.md`, so the next agent inherits the conventions instead of re-deriving them.

1. **Find the article** `02-DOCS/wiki/stack/mongodb.md`, linked from a `## Knowledge map` section in
   the root `CLAUDE.md`.
2. **If missing or stale**, create/update it with the project's real choices — collection layout and
   embed/reference decisions, the index set and its ESR rationale, read/write concern policy, the
   Atlas tier, and any encryption/RBAC setup — then add/refresh the `CLAUDE.md` link (create the
   `## Knowledge map` section, and `CLAUDE.md` itself, if absent).
3. **Read it first on every use** and stay consistent; when a convention changes, update the article
   (bump its `Updated` date) in the same change.

No `02-DOCS/` layer? Skip silently (optionally suggest `harness`). Technical conventions are
*recorded, not gated* — never block the task on this.

## See Also

- [references/data-modeling.md](references/data-modeling.md) — embed/reference tree, the six
  patterns with worked documents, 16 MB math, polymorphic & schema versioning.
- [references/aggregation.md](references/aggregation.md) — per-stage index eligibility, `$lookup`
  variants, `$facet`, window functions, `$merge`/`$out`, hybrid `$scoreFusion`, reading pipeline
  `explain`.
- [references/transactions-and-ops.md](references/transactions-and-ops.md) — retry wrappers, concern
  semantics, replica-set requirement, Atlas tiers, Search/Vector Search, Queryable Encryption, RBAC,
  pooling, change streams.
- Sibling skills: [`harness`](../harness/SKILL.md) (scaffolds the `01-TOOLS/MONGODB` operational
  tool) and [`secure-coding`](../secure-coding/SKILL.md) (auth, encryption, least-privilege).
- For relational work — SQL, foreign keys, `EXPLAIN ANALYZE`, MVCC — use
  [`postgresdb`](../postgresdb/SKILL.md), not this skill. Different engine and planner.
- Out of scope here — external tools with their own docs: ODM/driver API surface (Mongoose hooks,
  the Node driver's `bulkWrite`/`updateMany` return shapes) and cross-engine vector-store selection.
  This skill owns the server query and the index the server picks.
