# Prompt Editing Tree Design

> Status: staged implementation. Wall-E chat already has branch snapshots;
> CTM Wall-E coding sessions now use the same active-path snapshot contract for
> delete-from-prompt and edit/resend forks. The full immutable DAG schema below
> remains the longer-term target.
> Date: 2026-05-03
> Scope: Wall-E chat, CTM Wall-E sessions, CTM transcript/review/session-memory caches

## Problem

CTM and Wall-E should support ChatGPT-style prompt editing: when a user edits an
earlier prompt and resends, the conversation should fork from that point instead
of mutating or truncating history. The result is a tree, not a single flat list.

Today the codebase has partial ingredients for this, but the active product
paths still treat conversations as chronological streams:

- Wall-E chat persists `chat_messages` as flat rows ordered by `created_at`.
- Wall-E chat has a `chat_branches` sidecar table, but it is not the source of
  truth for model-context assembly.
- Wall-E JSONL records already include `uuid` and `parentUuid`, so they can
  represent ancestry.
- CTM imports JSONL into `session_conversations.messages` and
  `session_messages`, but those caches are flat active-history/search records.
- CTM conversation UI uses `parentUuid` for dedup/replacement, not branch
  selection.

The core risk is silent wrong context. If the UI shows an edited branch but
`chat()` or session-memory search still reads chronological rows, the model,
review panel, resume path, and Wall-E memory tools will use the wrong branch.

## Goals

- Preserve original prompts and assistant responses after edit/resend.
- Make the selected branch explicit and durable.
- Build model input from the selected branch only.
- Keep flat chronological APIs available as compatibility views.
- Support Wall-E-native chat fully.
- Support CTM terminal/provider sessions safely by forking instead of pretending
  provider CLIs can rewrite native history.
- Avoid destructive JSONL truncation as the default editing primitive.

## Implemented CTM Wall-E Coding Session Contract

The first implementation intentionally reuses Wall-E's existing
`chat_branches` sidecar instead of introducing a second CTM table.

- The browser keeps `messages` as the active visible path.
- Editing a user prompt saves the old suffix as a branch version, replaces the
  visible suffix with the edited prompt, and resends from that point.
- Deleting a user prompt truncates the visible path before that prompt and
  saves the truncated active path.
- Branch selection swaps the visible suffix without mutating message content.
- Every Wall-E coding send includes `contextMessages`, a compact active-path
  history, so `wall-e/chat.js` does not build model context from abandoned
  chronological rows.
- Review/session history reads prefer the durable branch snapshot when it
  exists, then fall back to legacy CTM JSONL and Wall-E chat history.

This gives the user-visible and model-visible behavior now while keeping the
raw JSONL append-only. The later DAG migration can move the same semantics into
first-class message-parent tables without changing the product behavior.

## Non-Goals

- Do not implement this design in this document.
- Do not rewrite `ctm_sessions` or `agent_sessions` identity tables.
- Do not require provider-native CLIs to support arbitrary branch state.
- Do not make `session_conversations.messages` the tree source of truth.
- Do not remove existing flat history/search APIs in the first migration.

## Current State

### Wall-E Chat DB

Current schema:

- `wall-e/brain.js` defines `chat_messages` with `id`, `role`, `content`,
  `channel`, `session_id`, model metadata, latency, token fields, and
  `created_at`.
- `insertChatMessage()` inserts flat rows.
- `listChatMessages()` returns chronological rows, optionally limited to recent
  history.
- `chat_branches` exists with JSON fields `branches`, `active`, and `history`,
  but current model context does not use it as the authoritative branch graph.

Current model context path:

- `wall-e/chat.js` persists the new user turn.
- It calls `brain.listChatMessages({ session_id, limit })`.
- It maps those rows to provider messages.
- The provider receives the recent chronological stream.

For prompt editing, this is insufficient because "recent chronological stream"
can contain siblings from abandoned branches.

### Wall-E JSONL

There are two Wall-E transcript writers:

- `wall-e/session-files.js` for Wall-E chat transcripts under Wall-E-owned data.
- `claude-task-manager/lib/walle-transcript.js` for CTM Wall-E session JSONL.

Both produce records with:

- `uuid`
- `parentUuid`
- `sessionId`
- `chatSessionId`
- `type`
- `message`

This is close to an append-only DAG event format. The missing pieces are branch
metadata and active-head selection.

### CTM Session Cache

CTM cache tables:

- `session_conversations`: imported transcript blob, currently `messages TEXT`.
- `session_messages`: per-message rows for search and context transfer.
- `session_messages_fts`: FTS index for content search.

These tables should remain read-optimized caches. They should learn tree fields,
but they should not own the canonical tree for Wall-E-native chat.

### CTM Conversation UI

The structured conversation view:

- Fetches `/api/session/messages`.
- Converts messages into user/assistant events.
- Uses `parentUuid` to dedupe streaming assistant updates.
- Groups turns by prompts for display.

This gives us a UI foundation, but not branch semantics. The UI needs to know
which messages are siblings, which branch is active, and which leaf is selected.

## Prior Art

OpenAI's public API docs describe two relevant state models:

- Conversation objects persist items with durable identifiers. Items can include
  messages, tool calls, tool outputs, and other data.
- `previous_response_id` chains responses and creates threaded continuation
  semantics, but it cannot be used together with `conversation`.

These are useful primitives, but they are not enough by themselves for our UI.
We need app-level branch heads because prompt editing creates multiple possible
descendant paths from the same ancestor.

OpenAI Help documents that ChatGPT data exports include chat history and that
account-transfer workflows can use `conversations.json`, but those public help
pages do not define a stable internal edit-tree contract. We already have a
local ChatGPT adapter that recognizes the export `mapping` shape with parent and
children fields, but it currently flattens the tree for memory import.

References:

- https://developers.openai.com/api/docs/guides/conversation-state
- https://developers.openai.com/api/reference/resources/responses/methods/create
- https://help.openai.com/en/articles/7260999-how-do-i-export-my-chatgpt-history-and-data
- https://help.openai.com/en/articles/9106926-transferring-conversations-from-1-chatgpt-account-to-another-chatgpt-account

## Core Design

Use an immutable message DAG.

Each message is a node:

- `id`: stable message ID.
- `parent_id`: previous message on this branch.
- `role`: `user`, `assistant`, `tool`, or `system`.
- `content`: message content.
- `branch_id`: logical branch namespace.
- `edited_from_id`: original message ID when this node was created by editing.
- `created_at`: insertion time.
- `metadata`: model, tokens, latency, tool calls, attachment descriptors, etc.

Each session has one or more branch heads:

- `session_id`
- `branch_id`
- `active_leaf_id`
- `forked_from_id`
- `title`
- timestamps

Rendering an active conversation means:

1. Start from `active_leaf_id`.
2. Walk `parent_id` until the root.
3. Reverse the list.
4. Render that active path.

Editing a prompt means:

1. User selects a prior user message node.
2. UI opens editable content for that node.
3. On resend, create a new user node with:
   - `parent_id` equal to the original message's parent.
   - `edited_from_id` equal to the original message ID.
   - a new `branch_id` or selected existing branch.
4. Generate a new assistant response as a child of the edited user node.
5. Update the branch head to the new assistant leaf.
6. Keep all old nodes intact.

This mirrors how the product should behave: editing is a fork, not mutation.

## Data Model

### Wall-E Chat Tables

Add tree columns to `chat_messages`:

```sql
ALTER TABLE chat_messages ADD COLUMN parent_id TEXT;
ALTER TABLE chat_messages ADD COLUMN branch_id TEXT DEFAULT 'main';
ALTER TABLE chat_messages ADD COLUMN edited_from_id TEXT;
ALTER TABLE chat_messages ADD COLUMN metadata TEXT;
```

Add branch-head table:

```sql
CREATE TABLE chat_branch_heads (
  session_id TEXT NOT NULL,
  branch_id TEXT NOT NULL,
  active_leaf_id TEXT NOT NULL,
  forked_from_id TEXT,
  title TEXT,
  created_at TEXT DEFAULT (datetime('now')),
  updated_at TEXT DEFAULT (datetime('now')),
  PRIMARY KEY (session_id, branch_id)
);

CREATE INDEX idx_chat_messages_parent ON chat_messages(parent_id);
CREATE INDEX idx_chat_messages_branch ON chat_messages(session_id, branch_id);
CREATE INDEX idx_chat_messages_edited_from ON chat_messages(edited_from_id);
```

`chat_branches` can remain temporarily as a compatibility payload, but new code
should prefer `chat_branch_heads` plus message parent links.

### CTM Cache Tables

Add nullable tree fields to `session_messages`:

```sql
ALTER TABLE session_messages ADD COLUMN message_uuid TEXT;
ALTER TABLE session_messages ADD COLUMN parent_uuid TEXT;
ALTER TABLE session_messages ADD COLUMN branch_id TEXT DEFAULT 'main';
ALTER TABLE session_messages ADD COLUMN edited_from_uuid TEXT;
ALTER TABLE session_messages ADD COLUMN source_event_uuid TEXT;
ALTER TABLE session_messages ADD COLUMN is_active_path INTEGER DEFAULT 1;
ALTER TABLE session_messages ADD COLUMN raw_event_json TEXT;
```

Add cache branch-head table:

```sql
CREATE TABLE session_branches (
  conversation_id TEXT NOT NULL,
  branch_id TEXT NOT NULL,
  active_leaf_uuid TEXT NOT NULL,
  forked_from_uuid TEXT,
  created_at TEXT DEFAULT (datetime('now')),
  updated_at TEXT DEFAULT (datetime('now')),
  PRIMARY KEY (conversation_id, branch_id)
);

CREATE INDEX idx_session_messages_uuid ON session_messages(message_uuid);
CREATE INDEX idx_session_messages_parent ON session_messages(parent_uuid);
CREATE INDEX idx_session_messages_branch ON session_messages(ctm_session_id, branch_id);
CREATE INDEX idx_session_messages_active_path ON session_messages(ctm_session_id, is_active_path);
```

`session_conversations.messages` remains a compatibility cache. It should
represent the selected active path unless an API explicitly asks for all nodes.

## JSONL Event Shape

Existing fields remain valid:

```json
{
  "type": "user",
  "provider": "walle",
  "version": "walle-jsonl-v1",
  "sessionId": "ctm-session-id",
  "chatSessionId": "wall-e-chat-session-id",
  "uuid": "message-node-id",
  "parentUuid": "previous-node-id",
  "timestamp": "2026-05-03T00:00:00.000Z",
  "message": {
    "role": "user",
    "content": [{ "type": "text", "text": "..." }]
  }
}
```

Add optional fields:

```json
{
  "branchId": "branch-abc",
  "editedFromUuid": "original-user-message-id",
  "activeLeaf": true
}
```

Rules:

- Omitted `branchId` means `main`.
- Omitted `editedFromUuid` means original node, not edit-derived.
- `parentUuid` is structural ancestry, not a streaming dedup hint only.
- Tool/result events should point to the assistant turn they belong to.
- Compaction and synthetic system events must have explicit metadata so they do
  not appear as editable user prompts.

## APIs

### Wall-E Chat

New read APIs:

- `GET /api/wall-e/chat/tree?session_id=...`
  - Returns nodes, branch heads, active branch, and active leaf.
- `GET /api/wall-e/chat/path?session_id=...&branch_id=...`
  - Returns only the active path for rendering/model replay.

New write APIs:

- `POST /api/wall-e/chat/edit`
  - Input: `session_id`, `message_id`, `content`, optional `branch_id`.
  - Creates a forked user node.
  - Starts generation from that branch if requested.
- `POST /api/wall-e/chat/branch/select`
  - Input: `session_id`, `branch_id`.
  - Updates the active branch for UI and future sends.
- `POST /api/wall-e/chat/branch/delete`
  - Soft-delete or hide branch head only; do not delete message nodes in v1.

Existing APIs:

- `/api/wall-e/chat/history` should continue returning active-path messages.
- `/api/wall-e/chat/branches` can remain as legacy compatibility, but should
  eventually be backed by first-class branch tables.

### CTM Session Messages

Extend `/api/session/messages`:

- Default response remains active-path, paginated.
- Add `branch_id=...` for a specific branch.
- Add `all_branches=1` for tree-aware clients.
- Include `message_uuid`, `parent_uuid`, `branch_id`, and `edited_from_uuid` in
  each message when available.

Add branch metadata endpoint:

- `GET /api/session/branches?id=...`
  - Returns branch heads and active branch for a CTM session or provider
    conversation source.

## Model Context Assembly

Wall-E chat must stop using chronological recent rows as model context.

Replace:

```js
brain.listChatMessages({ session_id, limit })
```

with:

```js
brain.listChatPath({ session_id, branch_id, leaf_id, limit })
```

The provider should receive only the active path:

- root to selected leaf
- plus the current user node if already inserted
- with attachments/tool results reconstructed from node metadata

Compaction should be branch-scoped:

- Store summaries by `branch_id` and `leaf_id`.
- A summary for one branch must not be reused as context for a sibling branch
  unless it summarizes only common ancestors before the fork point.

## UI Behavior

### Message Actions

Each user prompt in Wall-E chat and CTM structured conversation view should
support:

- Edit
- Copy
- Fork from here
- View versions if siblings exist

Edit flow:

1. User clicks Edit on a user prompt.
2. Prompt body becomes an editor.
3. User submits.
4. UI shows a pending branch.
5. Assistant response streams into the new branch.
6. Branch selector appears on the edited prompt if siblings exist.

### Branch Selector

At any message with multiple children, show branch navigation:

- `1 / N` branch count.
- Previous/next sibling controls.
- Optional branch title/model metadata.

Selecting a sibling changes the active leaf to the descendant path under that
sibling. It must not mutate node content.

### CTM Terminal Provider Sessions

For Claude/Codex/Gemini terminal sessions, prompt editing should be exposed as
"Fork from this prompt" rather than "edit in place".

Reason:

- Provider-native CLIs own their session state.
- Some providers support resume IDs, some have JSONL files, and Codex rollout
  paths can differ from true thread IDs.
- Rewriting old provider JSONL files would break auditability and can desync
  provider state.

Safe behavior:

1. Select a prompt in CTM conversation view.
2. Create a new CTM session/tab.
3. Seed the new session with reconstructed active-path context up to the parent
   of the edited prompt plus the edited prompt.
4. Run the provider from that fork.
5. Link the new session branch back to the source message.

Wall-E-native sessions can support true same-session branch selection because
we own both state and model context assembly.

## Search And Memory

Default search should index active-path messages only for user-facing recall.

Tree-aware search should support:

- active path only
- all branches
- branch ID filter
- edited-from lineage filter

Wall-E memory ingestion should avoid filing abandoned branch content as if it
were canonical user intent. Recommended behavior:

- Ingest active-path messages as normal.
- Ingest non-active branches with metadata:
  - `branch_id`
  - `is_active_path: false`
  - `edited_from_id`
- Retrieval tools should prefer active path unless user asks for versions,
  alternatives, abandoned branches, or prompt-edit history.

## Migration Strategy

### Phase 0: Documentation And Contracts

- Land this document.
- Add code comments/contracts later where branch-aware APIs will live.
- Do not change behavior yet.

### Phase 1: Add Schema, No Behavior Change

- Add nullable tree columns.
- Backfill existing flat rows as a linear `main` branch.
- Create one branch head per session using the latest message as leaf.
- Keep existing APIs returning the same flat data.

Backfill algorithm:

1. For each `session_id`, order messages by `created_at`, then `id`.
2. Set each row's `parent_id` to the previous row ID.
3. Set `branch_id = 'main'`.
4. Insert `chat_branch_heads(session_id, 'main', last_message_id)`.

### Phase 2: Read From Active Path

- Add `listChatPath()`.
- Switch Wall-E chat model context to active path.
- Keep `/history` response compatible by returning active path.
- Add tests proving sibling branch messages are excluded.

### Phase 3: JSONL And CTM Cache Preservation

- Add branch metadata to Wall-E JSONL writes.
- Update JSONL parsers to preserve `uuid`, `parentUuid`, `branchId`, and
  `editedFromUuid`.
- Add tree columns to `session_messages` extraction.
- Keep `session_conversations.messages` as active-path cache.

### Phase 4: UI Edit And Branch Select

- Add edit controls for Wall-E-native chat.
- Add branch selector UI.
- Add "Fork from this prompt" for terminal provider sessions.
- Add review/session-memory awareness of branch metadata.

### Phase 5: Advanced Cleanup

- Retire `chat_branches` JSON blob or make it a derived cache.
- Add branch-aware memory ranking.
- Add branch-aware export/import.
- Add UI for branch rename/delete/hide.

## Test Plan

Unit tests:

- `listChatPath()` returns root-to-leaf order.
- Edited prompt creates sibling branch, not mutation.
- Model context excludes abandoned sibling content.
- Flat migration creates linear `main` branch.
- JSONL parser preserves `uuid`, `parentUuid`, `branchId`,
  `editedFromUuid`.
- `session_messages` backfill preserves active-path and all-branch modes.

API tests:

- `/api/wall-e/chat/history` remains backward compatible.
- `/api/wall-e/chat/tree` returns nodes and branch heads.
- `/api/session/messages` defaults to active path.
- `/api/session/messages?all_branches=1` returns sibling metadata.

Browser tests:

- Editing a Wall-E prompt forks the conversation.
- Branch selector switches visible assistant response.
- Sending after branch switch continues from selected branch.
- Terminal provider sessions show "Fork from this prompt", not destructive edit.
- Review panel and latest-answer hydrate from the selected active branch.

Regression tests:

- Old sessions without branch fields still render.
- Attachments survive edit/fork metadata.
- Tool-call turns stay attached to the correct assistant node.
- Compaction summary from one branch does not contaminate sibling branch.

## Risks

- Chronological fallbacks may leak sibling branch content into model context.
- Search and memory may over-index abandoned drafts as canonical facts.
- Tool-result messages can be mis-parented if they remain synthetic user
  messages without explicit parent metadata.
- Provider CLI sessions cannot reliably support same-session edit semantics.
- Existing `chat_branches` JSON state may conflict with new first-class branch
  heads if both are written during migration.

## Design Decisions

1. Use immutable append-only nodes.
   - Reason: preserves auditability and avoids destructive transcript edits.

2. Make branch heads first-class rows.
   - Reason: active branch selection must be queryable without parsing a JSON
     blob.

3. Keep flat active-path views.
   - Reason: existing UI, review, search, export, and memory code expects lists.

4. Treat CTM terminal prompt editing as fork, not edit-in-place.
   - Reason: provider-native state is external and not uniformly rewriteable.

5. Keep `ctm_sessions` and `agent_sessions` unchanged.
   - Reason: session identity and message topology are separate concerns.

## Open Questions

- Should branch IDs be human-readable slugs, UUIDs, or deterministic
  `branch-{message_id}` values?
- Should branch selection be per user/browser, or persisted globally per
  session?
- Should abandoned branch content be included in normal Wall-E memory at all, or
  only when explicitly selected?
- Should editing a user prompt auto-regenerate the assistant response, or allow
  saving a draft branch before running?
- How should branch-aware compaction summarize common ancestors versus
  branch-specific tails?
- Should CTM forked provider sessions live under the same CTM parent session or
  become independent tabs linked by metadata?

## Implementation Guardrails

- Do not use JSONL truncation as prompt editing.
- Do not mutate original message content when editing.
- Do not build provider context from chronological rows after branches exist.
- Do not index non-active branches as canonical memory without metadata.
- Do not assume provider session ID equals transcript filename.
- Do not overload `session_id` to mean CTM tab ID, provider ID, branch ID, and
  message ID. Keep names explicit.
