# Local-first reconciliation plan

> Status: **plan, not implementation**. Reviewed before any code is touched.
> Owner: Claude. Approver: Bob.

## The rule

The local store is the source of truth for everything the user sees and does.
The server is a separate, asynchronously-reconciled mirror.
**No user-action code path waits on a network call.**

Every user-visible read returns from local SQLite + local files in microseconds.
Every user-visible write commits to the local store synchronously and returns
the local id immediately. Server-side mirroring is the sync layer's job, not
the UI's.

This document defines the invariants, the new API split, the event protocol,
the migration order, and the visible-state model.

## Invariants the implementation must enforce

1. **Reads never await server.** `getMessage`, `getMessages`, `getUnifiedInbox`,
   `searchMessages` (local scope), `getCalendarEvents`, `getTasks`, autocomplete —
   all read from local DB and return synchronously. If the IPC layer can't be
   sync, it must return a Promise that resolves on the same tick.

2. **Writes ACK from a local commit.**
   - `saveDraft` writes the local Drafts row + `.eml` and returns the local UUID
     in the same tick. The IMAP push is queued, never awaited.
   - `send` writes the local Outbox row + `.ltr` and returns immediately. SMTP
     and Sent-folder append are background work.
   - `move`, `flag`, `delete` commit the local row mutation, return, and queue
     the server-side mirror.

3. **The sync queue is the only IMAP-touching code path.** A single background
   reconciler drains queued local actions to the server, polls for server
   changes, and emits events to the UI. Nothing else opens an IMAP client.

4. **Local UUIDs are stable; server UIDs are metadata.** Every message gets a
   local UUID at first sight (sync, draft save, server-search hit). Sync
   maintains a UUID ↔ (account, folder, server-UID) map. Server-side moves
   change the UID; the UUID stays. Local moves keep the UUID and queue a
   server-side move. Tombstones (UUID + deleted flag) prevent re-fetch.

5. **Slow / unreachable server is not a UI failure mode.** Connection caps,
   IDLE drops, OAuth refresh hangs, 60-second IMAP timeouts — none reach the
   click → render path. They surface in a non-blocking sync-status indicator.

## Service-side API split

Today's `MailxService` mixes local reads with server-touching code. Split into
three layers:

### Layer 1: `LocalStore` (sync, never touches network)

Pure local reads + local commits. All synchronous (or trivially async over
sql.js / fs). Throws if asked to do anything network-y.

```typescript
class LocalStore {
    // Reads
    getMessage(uuid: string): LocalMessage | null;
    getMessageByUid(accountId: string, uid: number): LocalMessage | null;
    getMessages(query: MessageQuery): LocalMessage[];
    getUnifiedInbox(page, pageSize): { items: LocalMessage[]; total: number };
    searchMessages(query: string, ...): LocalMessage[];   // FTS5 only
    getCalendarEvents(fromMs, toMs): CalEvent[];
    getTasks(includeCompleted): Task[];
    searchContacts(query): Contact[];
    getMessageBody(uuid: string): { html?: string; text?: string; attachments: ... } | null;

    // Writes — return local UUID, queue server mirror
    saveDraft(draft: ComposeDraft): { uuid: string };       // queues
    sendMessage(msg: ComposeMessage): { uuid: string };     // queues
    moveMessages(uuids: string[], toFolderId: number): { ok: true };
    flagMessages(uuids: string[], flags: string[]): { ok: true };
    deleteMessages(uuids: string[]): { ok: true };
    createCalendarEvent(ev): { uuid: string };
    updateCalendarEvent(uuid, patch): { ok: true };
    deleteCalendarEvent(uuid): { ok: true };
    createTask(t): { uuid: string };
    updateTask(uuid, patch): { ok: true };
    deleteTask(uuid): { ok: true };
}
```

Every UI IPC call routes through this layer first. The webview never sees
beyond this.

### Layer 2: `SyncQueue` (async, mediates server)

The only code path that opens IMAP clients, calls Gmail API, posts SMTP, etc.
Single-threaded per account. Queue persisted across restarts.

```typescript
class SyncQueue {
    enqueueMove(uuid, fromFolderId, toFolderId): void;
    enqueueFlag(uuid, addFlags, removeFlags): void;
    enqueueDelete(uuid): void;
    enqueueDraftPush(uuid): void;
    enqueueSend(uuid): void;
    enqueueBodyRefresh(uuid): void;          // fire-and-forget; reconciler may dedupe

    // Periodic
    runPoll(): Promise<void>;                // checks server for new mail, etc.

    // Event emit
    on(event: "mirrored" | "conflict" | "error", handler): void;
}
```

Items in the queue are persistent (SQLite `sync_queue` table with `kind`,
`payload`, `attempts`, `created_at`, `last_attempt`, `next_attempt`, `status`).

### Layer 3: `Reconciler` (background loop)

Drains the queue, polls for server changes, emits events.

```typescript
class Reconciler {
    start(): void;       // long-running tick
    stop(): void;
    syncNow(accountId?): void;       // user-initiated "sync now"
}
```

Reconciler events:
- `messageUpserted(uuid, source: "local"|"server")` — UI updates that row
- `messageRemoved(uuid)` — UI removes the row
- `bodyAvailable(uuid)` — UI re-fetches body (will hit local cache instantly)
- `syncStateChanged(accountId, state)` — status indicator updates
- `conflict(uuid, kind, detail)` — user gets a banner ("local move failed
  on server because folder is gone")

## Event protocol (local store → UI)

Events flow one-way: store changes → UI redraws affected rows.

| Event | When | UI behavior |
|---|---|---|
| `messageUpserted` | new mail synced; flag changed; body cached; envelope refreshed | re-render that row in place; if it's the focused row, re-render preview |
| `messageRemoved` | server-side delete reconciled; local delete with no undo window | remove row; advance focus to next |
| `bodyAvailable` | body fetch completed (from prefetch or on-demand reconcile) | re-render preview if this is the focused row |
| `folderCountsChanged` | unread counts changed | update folder-tree pill |
| `syncStateChanged` | reconciler started/stopped, account auth issue, sync error | update status bar indicator |
| `conflict` | local action failed on server | show banner with retry/discard |

Events carry a "sequence number" so the UI can drop stale events when the
user has navigated away from the affected row.

## Visible state model

Each row carries three independent indicators:

| Indicator | Source | Meaning |
|---|---|---|
| Body cached dot (existing teal/blue) | `body_path != null && file exists` | Tap is instant |
| Local-only outline (new) | `dirty=true && providerId=null` | Not yet pushed to server |
| Reconciliation pending (new) | `dirty=true && providerId!=null` | Push queued; will sync soon |

Only "body cached" affects how a click renders. The other two are advisory —
the user knows their action is committed locally and the server side will
catch up.

A status-bar pill shows `Sync OK` / `Syncing N items` / `Sync errors (click to view)`.
That's the only UI surface that reflects server state.

## Migration order

Land in this order so each step is independently shippable:

1. **Add `LocalStore` interface and route ALL UI IPC through it.** Service
   methods become thin wrappers that pull from local DB only. No behavior
   change yet — server fetches just stop happening on every UI call. Bodies
   that aren't cached return null with a "not cached" hint.

2. **UI handles the "not cached" case explicitly.** Show a placeholder-with-
   spinner that says "downloading body…" with no time pressure (the click is
   already responsive — body just takes time to arrive). Listen for
   `bodyAvailable` event and re-render. Remove the 15s retry timer band-aid.
   Remove `body-broken` red-dot band-aid.

3. **Move all current `await imap*` calls in the service into `SyncQueue`
   enqueue calls.** Drafts: write local + enqueue push. Send: write local +
   enqueue SMTP. Move/flag/delete: commit local + enqueue mirror. Body fetch
   on click: read local file or enqueue refresh (with `bodyAvailable` emit
   when done).

4. **Build the `Reconciler` loop** that drains the queue, runs periodic polls,
   and emits events. Replace the current `syncAll` with a reconciler that
   processes both queue items and "sync from server" pulls in priority order.

5. **Add the visible-state indicators** (local-only outline, pending dot,
   sync-status pill). Wire to the new event stream.

6. **Migrate the conflict path.** When a queued item fails on the server
   (folder deleted, message gone, auth lapsed), surface a single banner
   listing the affected items with retry/discard.

7. **Delete the band-aids**: 60s body-fetch timeout, 15s client retry timer,
   `body-broken` class, "Fetching message body…" placeholder, all stale-gen
   `if (gen !== showMessageGeneration) return` early-returns.

Each step ships standalone. After step 3 the UI is already responsive
even if the reconciler isn't built — it just stops doing sync until step 4.

## Concrete call-site changes

### Client side (TypeScript)

Files to touch and what changes:

- `client/components/message-viewer.ts` — `showMessage` becomes synchronous.
  Reads from local cache, never awaits a server fetch. Body display routed
  through new `bodyAvailable` event listener.
- `client/components/message-list.ts` — same for `loadMessages`,
  `loadUnifiedInbox`, `loadSearchResults` (local scope only).
- `client/compose/compose.ts` — `saveDraft` and `send` get the local UUID
  back immediately, no spinner.
- `client/components/calendar-sidebar.ts` — `getCalendarEvents` /
  `createCalendarEvent` etc. pure local; events update on `messageUpserted`.
- `client/lib/api-client.ts` — every method documented as "local read" or
  "queues server mirror". No method returns a server result directly anymore.
- `client/app.ts` — wire the new event types into the existing service-
  channel listener.

### Service side (TypeScript)

- `mailx-service/index.ts` — split. The `MailxService` class stays as the
  IPC entry point but delegates: reads → `LocalStore`, writes → `LocalStore`
  + `SyncQueue.enqueue*`, no more direct `imapManager.*` calls from UI
  handlers.
- New file `mailx-service/local-store.ts` — pure local reads/writes.
- New file `mailx-service/sync-queue.ts` — persistent queue + enqueue API.
- New file `mailx-service/reconciler.ts` — background loop, draining +
  polling, emits events.
- `mailx-imap/index.ts` — becomes a worker library called only by the
  reconciler. UI never sees it.

### Database

- New table `sync_queue (id, kind, payload, attempts, created_at,
  last_attempt, next_attempt, status, account_id)`.
- New columns on `messages`: `dirty BOOLEAN` (local-only or pending),
  `last_local_change_ms` (so reconciler picks oldest first).
- Existing `body_path` semantics stay.

## What the user sees afterward

- Every click renders in <50ms regardless of server state.
- Drafts save and Reply opens are instant.
- New mail appears in the list as soon as the reconciler pulls it (no
  user wait).
- A persistent status pill in the bottom-right shows whether sync is
  current, lagging, or failing.
- Body fetches that haven't completed show a spinner inside the preview
  pane that doesn't block interaction with the list.
- Multi-account: bobma's slowness affects only bobma's status pill, never
  the UI as a whole.

## Effort

Realistic for solo work, your pace:

- Step 1 (route UI through LocalStore): 1-2 days
- Step 2 (UI body-not-cached handling): 1 day
- Step 3 (move IMAP calls into SyncQueue): 2-3 days
- Step 4 (build Reconciler): 2-3 days
- Step 5 (visible-state indicators): 1 day
- Step 6 (conflict path): 1 day
- Step 7 (delete band-aids): 1 day

Total: ~9-12 working days, shipped incrementally. Each step is its own
publish; no big-bang.

## Decisions (2026-05-06)

1. **sync_queue concurrency**: priority lanes (interactive > sync > prefetch
   > backfill) **plus** multiple handlers per lane. The interactive lane is
   reserved for on-demand body fetches the user just clicked; sync runs in
   parallel on a different lane. Per-account serialization still applies
   inside each lane to keep us under Dovecot/Gmail connection caps.

2. **Multi-device conflict**: last-writer-wins. The case (drag-at-the-same-time
   on two devices) is rare enough that no banner / merge UI is needed. The
   later push wins; the earlier one is silently superseded.

3. **Tombstones**: live forever, no retention policy. Cost is ~200 bytes per
   row in the `messages` table — negligible even at 10k deletes/year × 10
   years. They're shared across devices implicitly: every device syncs the
   same server state, so a delete on phone propagates to desktop via the
   server reconcile, no peer-to-peer sync needed. Local-only tombstones
   (queued delete not yet pushed) stay per-device until the queue drains.

4. **Calendar / tasks / contacts**: same model. No special cases. Today's
   two-way cache code is already on this pattern; the refactor just makes
   message paths match.

5. **Outbox**: keep the existing flow as-is. The Drafts → Outbox → Sent
   directory-based queue works and isn't blocking the UI today; subsuming
   it would be churn for no user benefit.

Step 1 starts immediately.
