# Session Capture Foundation Design

Date: 2026-04-28
Status: design draft, source-code reuse pass applied
Owner: CTM

## Summary

Build a shared `SessionCapture` foundation by promoting the existing
`SessionStream`/session-history/status/approval machinery into a clearer
provider-neutral contract. The codebase already captures most of the requested
surface: live transcript events, persisted conversation messages, active-session
status, hover summaries, prompt queues, approval decisions, and restart
scrollback. The foundation should reuse those pieces first.

This should not replace the existing terminal scrollback recorder or the
existing provider-specific JSONL stream tailers. The foundation should sit above
them as a normalization and projection layer:

1. Capture from the richest source available for each provider.
2. Normalize provider events into a small CTM event vocabulary.
3. Maintain cheap in-memory live projections for UI and automation.
4. Persist durable message/search/history data through existing tables.
5. Let downstream features subscribe to the same substrate instead of each
   feature scraping the terminal or re-parsing provider files independently.

The key architectural decision is to make `SessionStream` the first-class live
capture bus, make raw PTY bytes a fallback signal, and avoid adding new tables
where existing tables already own the data.

## Why This Matters

Several features want the same facts:

- What did the user ask recently?
- What has the coding agent emitted recently?
- Is the session running, waiting for input, waiting for approval, idle, exited,
  or unknown?
- Is there an approval request, and what exact command or action needs a
  decision?
- What is the recent work summary for a tooltip or active-session preview?
- Should a monitor agent intervene because the session is stuck, looping,
  blocked, failing, or drifting?

Today these questions are answered by separate mechanisms:

- PTY scrollback capture restores terminal output.
- Provider-specific stream readers tail structured files where possible.
- Hooks and telemetry provide partial status signals.
- The approver still relies heavily on screen parsing.
- UI status and summaries are computed through existing stream APIs.

Those pieces are valuable, but they are not yet exposed as one reusable
foundation. `SessionCapture` should be a thin contract over existing code first,
not a duplicate recorder.

## Current Codebase Findings

### Existing Session And Scrollback Capture

The codebase already captures raw terminal output for active sessions.

Relevant files:

- `claude-task-manager/server.js`
- `claude-task-manager/workers/scrollback-worker.js`
- `claude-task-manager/workers/headless-term-worker.js`
- `claude-task-manager/lib/session-history.js`
- `claude-task-manager/db.js`

Observed behavior:

- PTY output is observed in the server and fed unthrottled to the headless xterm
  worker for current terminal state snapshots.
- PTY output is also batched into `scrollback_log` for restart survival.
- `scrollback_log` is explicitly cleared on normal session exit, so it is not a
  long-term semantic capture store.
- This is useful for restoring the terminal view and as a fallback, but raw
  terminal bytes are expensive and ambiguous for semantic features.

Implication:

- Keep scrollback capture.
- Do not use it as the primary semantic conversation source when structured
  provider logs exist.
- Use it for:
  - visual restore,
  - coarse activity heartbeat,
  - fallback for providers without structured transcripts,
  - debugging capture gaps.

### Existing Structured Stream Layer

The codebase already has a structured stream layer.

Relevant files:

- `claude-task-manager/lib/session-stream.js`
- `claude-task-manager/lib/session-state-bus.js`
- `claude-task-manager/lib/telemetry-receiver.js`
- `claude-task-manager/public/js/stream-view.js`
- `claude-task-manager/public/index.html`

Observed behavior:

- Provider session files are tailed and converted into events/status.
- The server exposes stream status and session stream APIs.
- The frontend subscribes to stream events over websocket.
- The UI already uses stream summaries/status for active sessions in some
  places.
- `SessionStream` already has:
  - `JsonlTailer` with byte offsets and partial-line handling,
  - a per-agent ring buffer,
  - CTM session ID to agent session ID mapping,
  - Claude and Codex JSONL parsing,
  - user prompt cache,
  - debounced summary generation,
  - `getRecentEvents`, `getSummary`, and `getAllStatuses`.
- `stream-view.js` already consumes `/api/sessions/:id/summary`,
  `/api/stream/status`, `stream-init`, `stream-event`, and `stream-status`.

Implication:

- `SessionCapture` should evolve out of this layer rather than start as an
  unrelated subsystem.
- The new foundation should preserve existing APIs while tightening vocabulary,
  storage, projections, and downstream contracts.

### Existing Durable Session Tables

Relevant file:

- `claude-task-manager/db.js`

Existing durable data includes:

- `ctm_sessions`
- `agent_sessions`
- `session_conversations`
- `session_messages`
- `session_messages_fts`
- `session_analyses`
- `session_analyses_fts`
- `scrollback_log`
- `startup_tasks`
- `approval_decisions`
- `approval_rules`
- `permission_rules` / `perm_rules`
- `prompt_queues`

Implication:

- There is already a durable conversation cache, message search index, session
  identity model, active-session restore model, prompt queue store, and approval
  audit log.
- Do not add `session_live_state` or `session_turns` in the first implementation.
- Do not duplicate `session_messages` for user/assistant conversation text.
- The only plausible new durable table is an append-only capture/event table for
  facts that existing tables do not own: status transitions, approval-request
  lifecycle, tool calls/results, capture health, and monitor-agent annotations.

### Existing Approval Path

Relevant files:

- `claude-task-manager/approval-agent.js`
- `claude-task-manager/lib/session-jobs.js`
- provider-specific approval parsers
- `claude-task-manager/server.js`

Observed behavior:

- The approver has substantial logic for interpreting terminal screens and
  provider-specific approval surfaces.
- `approval_decisions` already audits approved/escalated decisions.
- `approval_rules` already stores learned auto-approval rules and command
  signatures.
- This works as a compatibility strategy, but it couples approval automation to
  screen rendering.

Implication:

- A normalized event like `approval.requested` should become the preferred input
  to future monitor/automation, but the first version should reuse the existing
  provider parsers and `approval_decisions` audit path.
- Screen parsing should remain as fallback.

### Source-Code Reuse Matrix

| Need | Reuse first | Why |
| --- | --- | --- |
| CTM session identity | `ctm_sessions` | Existing root row for tab/session title, cwd, provider, starred state. |
| Provider transcript identity | `agent_sessions` | Already maps provider session IDs to CTM IDs and stores provider, `jsonl_path`, model, branch, counts, slug. |
| Active-session restore | `startup_tasks` | Already tracks live CTM tasks, command, cwd, model, provider type, chat/session IDs, worktree, branch. |
| Live transcript stream | `SessionStream` | Already tails JSONL, normalizes Claude/Codex user/assistant/tool-result events, holds a ring, emits WS events/status, and builds summaries. |
| File watching | `JsonlWatcher` + `session-jobs` reconciliation | Already combines `fs.watch`, periodic rescans, symlink hardening, compact `.bak` handling, and 10-minute reconciliation. |
| Durable rendered conversation | `session_conversations` | Existing JSON message cache used by Review and Conversation views. |
| Durable message search | `session_messages` + `session_messages_fts` | Existing per-message table and FTS index for deep search. Extend only if capture needs metadata columns. |
| Historical AI analysis | `session_analyses` + FTS | Existing title/summary/topics/category table for completed/offline session analysis. |
| Live terminal restore | `headless-term-worker` and `scrollback_log` | Existing xterm snapshot source and restart scrollback persistence. Do not use as semantic history. |
| Status signals | `telemetry-receiver`, `status-hooks`, `SessionStream.getAllStatuses` | Existing hook/OTEL/stream status paths already feed active-session UI. |
| Approval automation | `approval-agent`, provider parsers, `approval_decisions`, `approval_rules` | Existing parser/rule/decision pipeline should remain the execution path. |
| Prompt queue input | `queue-engine`, `prompt_queues` | Existing queued prompts and auto-advance state can emit capture-side prompt-submitted events. |
| Active-session UI | `stream-view.js`, `getSessionStatus`, `SessionActivityUtils` | Existing tooltip/status/grouping surface already consumes stream and authoritative status signals. |

Reuse rule: add new schema only where the fact is not already represented above,
or where overloading an existing table would break its current contract.

## Online Research

### Claude Code Transcripts

Source: https://code.claude.com/docs/en/claude-directory

Claude Code stores conversation history under the user-level Claude directory,
including project-scoped transcript JSONL files. This means CTM can usually
read a structured conversation stream without scraping terminal output.

Design implication:

- Prefer Claude transcript JSONL as the source for user prompts, assistant
  responses, tool calls, tool results, and session IDs.
- Tail incrementally by byte offset and inode.
- Treat file rotation, compaction, and permission failures as recoverable.

### Claude Code Hooks

Source: https://code.claude.com/docs/en/hooks

Claude Code supports lifecycle hooks around prompts, tool use, stop events, and
related agent activity.

Design implication:

- Hooks are strong status and event signals.
- They should not be the only source of truth because hooks can be disabled,
  misconfigured, or fail, but they are useful for low-latency status changes and
  approval/tool events.

### Claude Agent SDK Sessions

Source: https://code.claude.com/docs/en/agent-sdk/sessions

Claude's SDK session model supports continuing and resuming sessions by ID.

Design implication:

- CTM should persist provider session IDs and map them to CTM session IDs.
- Capture should keep the provider ID in every normalized event for replay and
  cross-provider debugging.

### Gemini CLI Save/Resume And Checkpointing

Sources:

- https://google-gemini.github.io/gemini-cli/docs/cli/commands.html
- https://google-gemini.github.io/gemini-cli/docs/checkpointing.html

Gemini CLI has explicit chat save/resume behavior and checkpointing features.

Design implication:

- Provider adapters must not assume every coding agent exposes a Claude-like
  JSONL transcript.
- The adapter boundary should allow:
  - durable transcript tailing,
  - exported/saved chat files,
  - checkpoint metadata,
  - PTY fallback.

### GitHub Copilot CLI Chronicle

Source: https://docs.github.com/en/copilot/concepts/agents/copilot-cli/chronicle

GitHub Copilot CLI documents local session data and a session history concept.

Design implication:

- "Agent session chronicle" is a solved product pattern.
- CTM should model a provider-neutral chronicle for coding-agent sessions, not a
  provider-specific log parser API.

### OpenTelemetry Logs And GenAI Events

Sources:

- https://opentelemetry.io/docs/specs/otel/logs/data-model/
- https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/

OpenTelemetry log records and GenAI semantic conventions provide useful prior
art for event attributes, timestamps, trace/session correlation, and GenAI
message events.

Design implication:

- Use OTel-inspired event fields:
  - timestamp,
  - severity,
  - body,
  - attributes,
  - trace/span/session correlation where available.
- Keep CTM's internal schema small, but align naming where it does not make the
  system awkward.

### Terminal Recording Prior Art

Sources:

- https://docs.asciinema.org/how-it-works/
- https://docs.asciinema.org/manual/asciicast/v1/
- https://github.com/microsoft/node-pty
- https://www.npmjs.com/package/@xterm/addon-serialize/v/0.12.0

asciinema records terminal sessions as timed output events. `node-pty` is the
common primitive for spawning and observing terminal processes. xterm's
serialize addon can serialize terminal buffer state.

Design implication:

- Terminal capture is a solved primitive for visual replay.
- Semantic capture should not be built from ANSI parsing unless no structured
  source is available.
- If CTM wants later visual replay, raw PTY chunks plus periodic xterm
  snapshots are appropriate.

### File Watching Caveats

Source: https://nodejs.org/api/fs.html

Node's `fs.watch` has platform-specific caveats and can miss or coalesce events.

Design implication:

- Transcript tailers should combine watch notifications with periodic polling.
- Store byte offsets.
- Detect truncation and inode changes.
- Make replay idempotent with provider event IDs or content hashes.

## Goals

1. Provide near-realtime normalized events for active coding sessions.
2. Capture both user prompts and coding agent output.
3. Support many concurrent sessions with low CPU and memory overhead.
4. Make session status derivable from shared signals.
5. Feed approver automation without screen scraping when possible.
6. Generate rolling summaries and active-session tooltips.
7. Enable monitor agents and future automation.
8. Preserve current terminal restore behavior.
9. Allow provider-specific adapters without leaking provider details into every
   downstream feature.

## Non-Goals

1. Full visual terminal replay in the first implementation.
2. Replacing every existing stream/status API at once.
3. Perfect semantic parsing from raw PTY output.
4. Capturing secrets or hidden terminal input.
5. Running a summarizer on every output chunk.
6. Building a distributed event bus unless local-process scaling becomes a real
   bottleneck.

## Proposed Architecture

### High-Level Flow

```text
Existing provider files/hooks/PTY/input events
        |
        v
Existing adapters:
JsonlWatcher + SessionStream + telemetry-receiver + status-hooks + approver
        |
        v
SessionCapture contract over SessionStream
        |
        +--> existing hot ring buffers
        +--> existing session_conversations/session_messages projections
        +--> optional capture_events for non-message lifecycle facts
        +--> existing websocket/API stream
        +--> existing active-session status/tooltip UI
        +--> existing approver + future monitor agents
```

### Source Adapters

Do not start by creating parallel adapters. Start by wrapping and extending the
adapters that already exist.

Initial reuse mapping:

- `JsonlWatcher`
  - Existing source of Claude JSONL `file-new` / `file-change` events.
  - Already hardened against symlink/path traversal issues.

- `SessionStream.JsonlTailer`
  - Existing byte-offset incremental reader.
  - Reuse it for live transcript tailing rather than adding another tailer.

- `SessionStream._processEntry`
  - Existing Claude user/assistant/tool-result normalizer.
  - Extend it with normalized `kind`/`source` metadata instead of reparsing JSONL
    somewhere else.

- `SessionStream._processCodexEntry`
  - Existing Codex user/assistant normalizer using `session-history.js`.
  - Extend here for Codex capture rather than adding a second Codex parser.

- `telemetry-receiver`
  - Existing hook/OTEL fan-in for Claude, Codex, and Gemini status.
  - Add capture-state notifications here if needed, but preserve the existing
    `session.status` websocket contract.

- `approval-agent` + provider parsers
  - Existing screen-derived approval context parser and decision engine.
  - Emit capture lifecycle events from this path; do not duplicate approval
    parsing.

- `queue-engine` and `handleInput`
  - Existing paths that write user/queued prompts to PTY.
  - These are useful low-latency prompt-submission signals before provider JSONL
    catches up, but transcript events should still be the high-confidence source.

Future adapters:

- Gemini transcript/checkpoint adapter only if Gemini exposes a durable
  conversation source CTM does not already ingest.
- A provider-neutral adapter interface only after at least two providers need
  different code paths that cannot live inside `SessionStream` cleanly.

### Normalized Event Shape

```ts
type SessionCaptureEvent = {
  id: string;
  sessionId: string;          // CTM session id
  provider: 'claude' | 'codex' | 'gemini' | 'unknown';
  providerSessionId?: string;
  source: 'transcript' | 'hook' | 'pty' | 'ctm-input' | 'telemetry';
  kind:
    | 'session.started'
    | 'session.exited'
    | 'turn.started'
    | 'turn.completed'
    | 'user.prompt'
    | 'assistant.delta'
    | 'assistant.message'
    | 'tool.call'
    | 'tool.result'
    | 'approval.requested'
    | 'approval.resolved'
    | 'status.changed'
    | 'error'
    | 'heartbeat';
  createdAt: string;          // ISO timestamp
  observedAt: string;         // ISO timestamp when CTM observed it
  sequence: number;           // per CTM session monotonic sequence
  turnId?: string;
  parentId?: string;
  text?: string;
  data?: Record<string, unknown>;
  confidence: 'high' | 'medium' | 'low';
};
```

Important rules:

- `createdAt` comes from the provider if available.
- `observedAt` always comes from CTM.
- `sequence` is assigned by CTM after normalization.
- `confidence` lets downstream users prefer transcript events over PTY fallback.
- Large raw payloads should not be copied into every event. Store compact text
  and structured metadata; keep raw payload references when needed.

### Event Identity And Idempotency

Events need stable IDs because file watchers can replay lines.

Preferred ID order:

1. Provider event/message ID if available.
2. Provider session ID plus transcript byte offset.
3. Hash of provider session ID, source, kind, timestamp, and compact payload.

The event bus should dedupe per session by event ID.

### Hot In-Memory State

Maintain a compact per-session live object:

```ts
type SessionLiveState = {
  sessionId: string;
  status: 'running' | 'waiting' | 'waiting_approval' | 'idle' | 'exited' | 'unknown';
  provider: string;
  providerSessionId?: string;
  lastEventAt?: string;
  lastUserPromptAt?: string;
  lastAssistantOutputAt?: string;
  activeTurnId?: string;
  pendingApprovalId?: string;
  recentEvents: RingBuffer<SessionCaptureEvent>;
  recentText: RingBuffer<CompactTextChunk>;
  recentTurns: RingBuffer<SessionTurn>;
  summary?: SessionSummary;
};
```

Default ring sizes:

- recent events: 200 per session
- recent text chunks: 64 KB per session
- recent turns: 10 per session

For 100 active sessions, this stays small enough for one Node process if events
are compact and raw PTY chunks are not duplicated.

### Durable Storage

Reuse existing tables first.

Keep using:

- `ctm_sessions` for CTM tab/session identity.
- `agent_sessions` for provider session identity, provider type, transcript
  path, model, branch, file size, modified time, and user-message count.
- `startup_tasks` for active process restore and live task metadata.
- `session_conversations` for rendered durable conversation JSON.
- `session_messages` and `session_messages_fts` for durable per-message search.
- `session_analyses` and `session_analyses_fts` for completed-session summaries,
  title/category/topic analysis, and search enrichment.
- `approval_decisions` for approval audit records.
- `approval_rules` and `perm_rules` / `permission_rules` for approval policy.
- `prompt_queues` for queued prompt state.
- `scrollback_log` only for restart scrollback, not semantic history.

Do not add in phase 1:

- `session_live_state`: `SessionStream`, `sessions`, frontend `_streamStatus`,
  and authoritative `session.status` already form the live state projection.
- `session_turns`: turns can be computed from `session_messages` and the
  existing `SessionStream.userPromptCache` for the first tooltip/monitoring
  use cases.
- A second message table: `session_messages` already exists for per-message
  persistence/search.

Possible later addition:

```sql
CREATE TABLE IF NOT EXISTS session_capture_events (
  id TEXT PRIMARY KEY,
  ctm_session_id TEXT NOT NULL,
  agent_session_id TEXT,
  provider TEXT,
  source TEXT NOT NULL,
  kind TEXT NOT NULL,
  sequence INTEGER NOT NULL,
  parent_id TEXT,
  provider_event_id TEXT,
  created_at TEXT,
  observed_at TEXT NOT NULL,
  text TEXT,
  data_json TEXT,
  confidence TEXT NOT NULL,
  inserted_at TEXT DEFAULT (datetime('now')),
  UNIQUE(ctm_session_id, sequence)
);

CREATE INDEX IF NOT EXISTS idx_capture_events_session_kind_time
  ON session_capture_events(ctm_session_id, kind, observed_at);
```

Only add this table when there is a real consumer for durable non-message
events. Good candidates:

- `approval.requested`
- `approval.resolved`
- `status.changed`
- `tool.call`
- `tool.result`
- `session.started`
- `session.exited`
- monitor-agent annotations

For user and assistant text, continue writing through `session_conversations`
and `session_messages`; duplicating full message text into
`session_capture_events` would increase storage and create two sources of truth.

### Status Projection

Session status is already a derived projection with multiple inputs:

- `telemetry-receiver` emits authoritative `session.status` from hooks/OTEL.
- `SessionStream` emits `stream-status` from JSONL and filtered PTY activity.
- `status-hooks` owns an idle/busy/waiting-input state bus for user hooks.
- The frontend `getSessionStatus` merges authoritative status, stream status,
  and local PTY fallback.

The capture foundation should consolidate vocabulary and expose the evidence
behind the status, not replace all of those paths in one patch.

Suggested status vocabulary:

- `running`
  - Recent assistant/tool output, active hook event, or active turn.

- `waiting`
  - Provider is waiting for user prompt and no approval is pending.

- `waiting_approval`
  - A high-confidence approval request is pending.

- `idle`
  - No recent output for a configurable window, process still alive, no known
    prompt/approval wait state.

- `exited`
  - PTY/process ended.

- `unknown`
  - Session exists but capture has insufficient evidence.

Priority order:

1. exited
2. waiting_approval
3. running
4. waiting
5. idle
6. unknown

Recommended idle thresholds:

- 30 seconds with no output while active turn is open: still `running`.
- 2 minutes with no output and no open turn: `idle`.
- Provider-specific prompt-ready signal: `waiting`.

Avoid deriving `waiting` solely from "no output"; that confuses long-running
commands with user-wait states.

Mapping to existing vocabulary:

- `SessionStream.running` maps to capture `running`.
- `SessionStream.waiting` maps to capture `waiting`.
- `SessionStream.idle` maps to capture `idle`.
- `SessionStateBus.busy` maps to capture `running`.
- `SessionStateBus.waiting_input` maps to capture `waiting` or
  `waiting_approval` when the source reason is an approval prompt.
- `session.status working=true` maps to capture `running`.
- `session.status working=false` maps to `waiting` or `idle` depending on prompt
  evidence.

### Turn Projection

Turns are the user-facing unit for summaries and tooltips.

A turn starts on:

- `user.prompt`

A turn may include:

- assistant deltas/messages,
- tool calls/results,
- approvals,
- errors,
- status changes.

A turn completes on:

- provider stop event,
- hook stop event,
- next user prompt,
- session exit,
- timeout plus prompt-ready signal.

Store:

- prompt text,
- compact assistant text,
- tool call summary,
- approval summary,
- status,
- started/completed timestamps.

### API And Websocket Surface

Keep existing stream APIs as the public compatibility surface. Add capture names
only when a new consumer needs them.

Existing endpoints to reuse:

- `GET /api/stream/status`
- `GET /api/sessions/:id/stream`
- `GET /api/sessions/:id/summary`
- `GET /api/session/messages`

Existing websocket events to reuse:

- `subscribe-stream`
- `stream-init`
- `stream-event`
- `stream-status`
- `session.status`
- `waiting-for-input`
- `approval-decision`

Possible later additions:

- `GET /api/sessions/:id/capture/events?after=<sequence>&limit=<n>` only after
  `session_capture_events` exists.
- `capture-event` only for non-message lifecycle events that do not fit the
  existing stream-event contract.
- Do not force the frontend migration into the first backend patch.

## Downstream Consumers

### Approver

Current pain:

- Approval automation has to inspect output directly.
- Terminal rendering is not a stable API.

New path:

1. Provider transcript or hook emits `approval.requested`.
2. Capture projection stores pending approval with command/action metadata.
3. Approver consumes pending approval event.
4. Approver sends decision through the existing provider-specific input path.
5. Capture emits `approval.resolved`.

Fallback:

- If no structured approval event arrives but the screen parser finds an
  approval prompt, emit a low-confidence `approval.requested` event sourced from
  `pty`.

This lets approval automation move to structured events without losing
compatibility.

### Active Sessions UI

Use existing active-session state for:

- status pill,
- activity timestamp,
- pending approval badge,
- recent prompt preview,
- tooltip summary,
- "what is this session doing?" detail.

The screenshot in the original request shows active sessions as the natural
surface for this. The UI should not need to read raw terminal output to populate
those affordances.

Current implementation already covers much of this:

- `stream-view.js` fetches `/api/sessions/:id/summary` on hover.
- `getSessionStatus` uses authoritative status and stream status before PTY
  fallback.
- Active sessions are grouped by Running, Waiting, Idle, Exited.

Recommended change:

- Improve the existing summary/status payload and tooltip rather than adding a
  new `session_live_state` table.

### Rolling Summaries

Generate summaries over the last N turns, default N = 5.

Inputs:

- prompt text,
- assistant summary text,
- tool calls,
- approvals,
- errors,
- status.

Trigger policy:

- On completed turn.
- On session exit.
- On demand if summary missing.
- Rate limit per session, for example no more than once every 30 seconds unless
  a turn completes.

Storage:

- Keep live tooltip summary in `SessionStream.cachedSummary`.
- For completed/offline analysis, reuse `session_analyses.summary`.
- If live summaries need restart persistence later, first consider adding
  `live_summary`, `live_summary_updated_at`, and `live_summary_source` to
  `session_conversations` or `agent_sessions` before creating a new table.

Recommended tooltip shape:

```text
Current: waiting for approval to run npm test
Recent: implemented stream parser changes, fixed status projection, ran server tests
Last prompt: "Add capture foundation design"
```

Keep this short. The active sessions UI needs a preview, not a full transcript.

### Monitor Agent

The monitor agent should subscribe to capture state/events instead of watching
the terminal.

Potential monitor behaviors:

- Detect stuck sessions:
  - active turn open,
  - no output for threshold,
  - no long-running command marker,
  - process alive.

- Detect repeated failures:
  - repeated test failures,
  - repeated same command,
  - repeated edit/revert loops.

- Detect approval backlog:
  - pending approval too long,
  - approval requested for risky command,
  - multiple sessions blocked.

- Detect user attention needs:
  - provider asks a question,
  - merge conflict appears,
  - auth/login failure,
  - rate limit,
  - network failure.

- Detect completion:
  - task summary emitted,
  - tests pass,
  - no active turn,
  - provider waiting.

Actions should start conservative:

- annotate session,
- update tooltip/status,
- notify user,
- request approval,
- queue a suggested next prompt.

Autonomous intervention should require a separate policy layer.

## Additional Product Opportunities

The same capture foundation can support:

- Cross-session command center:
  - "show sessions waiting on me"
  - "show sessions running tests"
  - "show sessions with failures"

- Global work feed:
  - timeline of prompts, tool calls, approvals, completions.

- Session search:
  - search recent prompts, assistant output, commands, errors.

- Smart resume:
  - restore last five turns plus summary when reopening a session.

- Automated handoff:
  - create concise handoff note from capture history.

- Cost/performance attribution:
  - estimate expensive sessions by duration, tool volume, token metadata where
    providers expose it.

- Quality analytics:
  - number of turns to completion,
  - test-fix loops,
  - approval latency,
  - idle time.

- Failure taxonomy:
  - auth,
  - missing dependency,
  - test failure,
  - type error,
  - merge conflict,
  - rate limit,
  - provider crash.

- Alerting:
  - notify when a long job completes,
  - notify when a session waits for input,
  - notify when a session reaches a risky approval.

- Dataset creation:
  - collect sanitized prompt/output/tool sequences for internal evaluation.

- Replay and audit:
  - reconstruct what happened without needing terminal scrollback.

- Better auto-titles:
  - derive active-session names from last user intent and current activity.

- Multi-agent coordination:
  - let one supervisor understand what multiple coding agents are doing.

## Efficiency Design

### Principle 1: Prefer Structured Deltas

Provider transcript lines are usually much smaller and more meaningful than PTY
screen updates. Tail transcript files incrementally and store offsets.

### Principle 2: Avoid Re-Parsing Per Consumer

Adapters parse once, then publish normalized events. Consumers subscribe to the
bus or query projections.

### Principle 3: Keep Raw Payloads Out Of Hot Paths

Do not store full raw provider messages in every in-memory projection. Keep:

- compact text,
- structured metadata,
- pointer to raw source if needed,
- rolling rings.

### Principle 4: Batch Durable Writes

Use small batch inserts for high-volume streams:

- flush every 100 to 250 ms,
- or every N events,
- whichever comes first.

Status updates can be coalesced because only the latest live projection matters.

### Principle 5: Backpressure And Drop Policy

For each session:

- never drop durable high-value events:
  - user prompts,
  - assistant final messages,
  - approvals,
  - errors,
  - status transitions,
  - tool calls/results.

- allow coalescing of:
  - assistant deltas,
  - heartbeat,
  - raw PTY snippets.

If a consumer is slow, it should resume from the existing stream snapshot plus
`session_conversations`/`session_messages`. If the optional
`session_capture_events` table is added later, non-message event consumers can
resume from that table by sequence.

### Principle 6: Lazy Summarization

Summarization is the expensive part. It should run:

- after turn completion,
- with rate limits,
- on compact turn text,
- only for sessions visible in active UI or recently active unless explicitly
  requested.

### Principle 7: Watch Plus Poll

Use file watch notifications for latency, but backstop with polling because
watchers can miss events. Store offsets and detect truncation/rotation.

Recommended default:

- watch transcript directory where possible.
- poll active transcript files every 500 to 1000 ms.
- use exponential slowdown for idle/exited sessions.

## Privacy And Safety

Capture will contain user prompts, file paths, commands, errors, and possibly
secrets accidentally pasted into a terminal.

Rules:

- Do not capture hidden password input.
- Redact known secret patterns before summaries or monitor-agent prompts.
- Keep raw event retention configurable.
- Separate local-only raw capture from any exported telemetry.
- Mark low-confidence PTY-derived text so automations avoid over-trusting it.
- Allow per-session capture disablement if needed.

Redaction should apply before:

- summaries,
- monitor-agent prompts,
- notifications,
- external exports.

Durable local event storage can retain original text by default if CTM already
stores local transcripts, but exports should use redacted text.

## Migration Plan

### Phase 0: Inventory And Contracts

- Document existing stream event shapes.
- Document current session status values and frontend consumers.
- Confirm provider transcript paths for Claude, Codex, and Gemini in this
  installation.
- Treat `SessionStream` as the live bus unless a concrete incompatibility is
  found.
- Treat `session_conversations` and `session_messages` as the durable message
  projection.
- Do not add schema until a consumer needs data that the existing tables cannot
  represent.

Deliverable:

- design doc and implementation checklist.

### Phase 1: Capture Core

- Add a small `session-capture` module or wrapper that delegates to
  `SessionStream` rather than competing with it.
- Define a normalized event vocabulary as an adapter over existing
  `stream-event` payloads.
- Add `kind`, `source`, `confidence`, and provider metadata to emitted stream
  events where possible without breaking the frontend.
- Reuse existing ring buffers and `getRecentEvents`.
- Reuse existing `/api/stream/status`, `/api/sessions/:id/stream`, and
  websocket `stream-*` contracts.

No UI rewrite yet.

### Phase 2: Status Projection

- Write a reducer that merges existing evidence:
  - `SessionStream` status,
  - `telemetry-receiver` `session.status`,
  - `status-hooks` state,
  - approval detection,
  - prompt-ready fallback.
- Keep existing frontend status behavior, but expose debug/evidence fields for
  monitor agents.
- Add tests for status transitions:
  - running,
  - waiting,
  - waiting approval,
  - idle,
  - exited.

### Phase 3: Approver Integration

- Emit normalized approval lifecycle events from the existing approver path.
- Keep `approval-agent` as the execution and parsing owner.
- Keep writing `approval_decisions`.
- Consider `decision='pending'` only if the UI/monitor needs a durable pending
  record; otherwise keep pending approval in hot capture state.
- Add tests for approval request/resolution lifecycle.

### Phase 4: Summaries And Tooltips

- Extend existing `SessionStream.getSummary` from the current prompt-cache model
  toward a last-five-prompt/turn model.
- Reuse the existing `/api/sessions/:id/summary` endpoint and stream tooltip.
- Reuse `session_analyses.summary` for completed/offline sessions.
- Add rate limits and redaction.

### Phase 5: Monitor Agent Substrate

- Add capture subscription for monitor agent.
- Start with read-only classification and notifications.
- Add policy-gated actions later.

### Phase 6: Provider Expansion And Hardening

- Harden Codex and Gemini adapters.
- Add replay from durable events on restart.
- Add capture health dashboard:
  - active adapters,
  - lag,
  - dropped/coalesced events,
  - last event time,
  - errors.

Only in this phase should we revisit a durable `session_capture_events` table,
and only if monitor/replay/approval consumers need non-message event history
that cannot be reconstructed from transcripts plus existing tables.

## Testing Strategy

### Unit Tests

- Event normalization per provider.
- Event ID/idempotency logic.
- Status reducer.
- Turn reducer.
- Summary input builder.
- Redaction.

### Integration Tests

- Tail a fixture transcript and verify emitted events.
- Simulate file truncation/rotation.
- Simulate duplicate watcher notifications.
- Simulate PTY fallback when transcript is absent.
- Verify websocket subscribers receive ordered events.
- Verify slow consumer can catch up by sequence.

### Regression Tests

- Existing session stream APIs still work.
- Existing terminal scrollback restore still works.
- Existing approver screen parsing still works as fallback.

### Performance Tests

Simulate:

- 10 active sessions,
- 50 active sessions,
- 100 active sessions.

Measure:

- CPU while tailing idle files,
- CPU while streaming output,
- memory per session,
- database write rate,
- websocket fanout overhead,
- summary invocation count.

Target:

- Near-zero CPU for idle sessions.
- No per-consumer transcript parsing.
- Bounded memory per session.
- Durable writes batched under load.

## Open Decisions

1. Should durable non-message events get a new table, or can we stay entirely on
   existing tables for now?

   Recommendation: stay on existing tables for phase 1. Add
   `session_capture_events` later only for non-message events such as approval
   request lifecycle, status transitions, monitor annotations, and tool events.
   Keep `session_messages` as the user/assistant text projection.

2. Should summaries be generated locally by the coding-agent provider, by CTM's
   chosen model, or by a lightweight heuristic first?

   Recommendation: reuse `SessionStream`'s current cloud/local/fallback summary
   tiers, but change the input from "last prompt cache only" toward the last five
   prompt/assistant turns where available. Keep completed-session summaries in
   `session_analyses`.

3. How much raw assistant delta text should be stored?

   Recommendation: keep final/rendered messages in `session_conversations` and
   `session_messages`; keep deltas in the existing hot ring only. Do not persist
   token-level deltas unless visual replay becomes a goal.

4. How should monitor agents be allowed to act?

   Recommendation: start read-only. Add action policies separately.

5. Should capture run for exited sessions?

   Recommendation: no active watchers after exit. Keep durable replay and final
   summary generation through `session_conversations`, `session_messages`, and
   `session_analyses`.

## Main Risks

### Provider Format Drift

Provider transcript schemas can change.

Mitigation:

- Keep adapters small and fixture-tested.
- Preserve unknown payload fields in `data`.
- Use confidence levels.

### Watcher Misses

File watching can miss changes.

Mitigation:

- Watch plus poll.
- Store offsets.
- Detect truncation/rotation.
- Make parsing idempotent.

### Over-Capture

Capturing everything can create privacy and performance problems.

Mitigation:

- Compact events.
- Redact summaries/exports.
- Bound hot buffers.
- Configurable retention.

### Status Misclassification

Idle/running/waiting can be ambiguous.

Mitigation:

- Prefer explicit provider/hook signals.
- Use priority reducer.
- Expose confidence and "last evidence" in debug state.

### Duplicate Systems

Adding a new foundation could duplicate existing `session-stream.js`.

Mitigation:

- Build by evolving/wrapping the existing stream layer.
- Keep old APIs as compatibility wrappers.
- Move consumers gradually.

## Recommended First Implementation

The first patch should be deliberately small:

1. Add a `session-capture` adapter module that wraps `SessionStream` and
   existing status/approval signals.
2. Add normalized event vocabulary fields to `SessionStream` events in a
   backwards-compatible way.
3. Add a status evidence reducer that consumes existing `stream-status`,
   `session.status`, `waiting-for-input`, and approval signals.
4. Reuse existing `/api/stream/status`, `/api/sessions/:id/stream`, and
   `/api/sessions/:id/summary`; add no tables in the first patch.
5. Add tests around the reducer and around event compatibility with
   `stream-view.js`.

Then:

1. Extend `SessionStream.getSummary` to use the last five meaningful
   prompt/assistant turns.
2. Emit approval lifecycle events from the existing approver path.
3. Teach monitor-agent code to consume the capture adapter.
4. Add `session_capture_events` only if a durable monitor/replay use case needs
   non-message event history.

This lets the foundation prove value quickly without destabilizing terminal
restore or provider-specific session handling.

## Bottom Line

The codebase already has more than raw ingredients: it has a live structured
stream (`SessionStream`), durable conversation/message tables, FTS search,
hook/OTEL state, active-session status UI, hover summaries, prompt queues,
headless terminal snapshots, restart scrollback, and approval automation. The
missing layer is a provider-neutral capture contract that reuses those systems
and fills only the gaps.

Build `SessionCapture` as that contract. Treat `SessionStream` as the live bus,
structured provider transcripts and hooks as primary evidence, PTY output as
fallback, existing message tables as durable conversation storage, and any new
event table as a later, narrow store for non-message lifecycle facts.
