# MCP interaction instructions

## Scope

Instruction text the Neotoma MCP server sends to clients so agents know how to use MCP actions. The server loads this file at runtime and ships the first fenced code block verbatim; edits here take effect without code changes in `src/server.ts`. For user-facing docs see `docs/specs/MCP_SPEC.md` and `docs/developer/mcp_overview.md`.

## Instructions (content sent to clients)

The block below is loaded and sent to MCP clients as interaction instructions. It is organized into labelled sections ([TURN LIFECYCLE], [STORE RECIPES], etc.) in the order agents need during a turn. Each section contains one rule per line where possible; mega-rules are split into atomic sub-rules.

```
[TURN LIFECYCLE]
MUST complete the turn in this order: (1) bounded retrieval, (2) user-phase store, (3) other MCP calls or host tool use, (4) compose assistant reply, (5) closing store of the assistant reply. Do not respond before steps 1 and 2 are complete, and do not end the turn before step 5 unless explicitly waived by the user or the turn produced no user-visible reply.
Short passes are NOT exempt: doc-only edits, `neotoma cli config`, single-file lint or copy tweaks, and other minimal turns still MUST run steps 1–5 in order. FORBIDDEN: skipping steps 1–2 before other tools, or skipping step 5 when there is a user-visible reply, because the turn seems quick, low-risk, non-chat, or "sync only". FORBIDDEN: rationalizing a store skip because the turn is "analysis-only", "in-session bookkeeping", "just an evaluation", or any similar framing — the audit trail is unconditional; no content category exempts a turn from the protocol.
Image-only messages are NOT exempt: when the user message body contains only an image or screenshot (no text), all five steps MUST still execute. Treat the visual content as the user message body for entity extraction and storage purposes. At minimum store the conversation entity, a user conversation_message entity (with content describing what the image shows), and any entities extractable from the image (tasks, events, deadlines, contacts, transactions visible in the screenshot). FORBIDDEN: skipping steps 1–5 because the user message has no text body.
Step 1 — Bounded retrieval: for entities implied by the user message, use retrieve_entity_by_identifier for concrete identifiers and retrieve_entities for category/list queries; use matches to reuse or link existing records. See [RETRIEVAL] for query-shape rules.
Step 2 — User-phase store: store the conversation, the current user message, and any entities implied by the message in ONE **`store`** call per [STORE RECIPES] user-phase. If the user attached a file, include the file in the same request and link per [STORE RECIPES] attachment. MUST NOT skip this for greetings or minimal messages.
Step 3 — Other actions: host IDE tools (read_file, apply_patch, run_terminal_cmd, grep, codebase_search, web fetch, and similar) and other MCP tools may run only after steps 1 and 2 are complete. Host tools are not exempt from store-first unless the user message is only an explicit waiver of persistence or a no-content system ping.
MCP availability detection: do not infer whether Neotoma is available in the live client session from a workspace `.mcp.json`, repo-local config file, or committed MCP manifest alone. If the host/session exposes Neotoma MCP tools or another active Neotoma transport, treat Neotoma as available and follow store-first.
Tool deregistration recovery: if a tool call fails with "tool 'Neotoma:X' is not registered" or "stale tool schema", the tool has fallen out of the host client's local cache — this is a client-side cache eviction, not a server-side removal. Recovery: call `tool_search` with the tool name or a relevant keyword (e.g. `tool_search("submit_issue")`) to reload it, then immediately retry the original call. Do not report this as a Neotoma server bug; it is expected behavior in long sessions with many tool loads.
Step 4 — Compose reply: synthesize the answer using retrieval results and tool output.
Step 5a — Closing store shape: after the user-visible reply is finalized, in the same turn, call **`store`** with entities = [ { entity_type: "conversation_message", role: "assistant", sender_kind: "assistant", content: "<exact assistant reply text as shown to the user>", turn_key: "{conversation_id}:{turn_id}:assistant" } ] and idempotency_key "conversation-{conversation_id}-{turn_id}-assistant-{suffix}". FORBIDDEN: reusing the user-phase `turn_key` (`{conversation_id}:{turn_id}` without the `:assistant` suffix) on the closing message — resolution targets the same `conversation_message` row as the user and corrupts the transcript; the server rejects this with `ERR_CONVERSATION_MESSAGE_ROLE_CONFLICT`.
Step 5b — Closing store relationship: then call create_relationship(PART_OF, source_entity_id = the assistant agent_message entity_id returned by this closing store, target_entity_id = the conversation entity_id returned by this turn's user-phase store). The target is the conversation the user message was linked to — not a new conversation.
Step 5b.1 — Reply-cited entity edges: in the same closing **`store`** call, include REFERS_TO relationships from the assistant agent_message to every entity the reply materially cites or produces during composition (synthesized notes/reports created this turn, and existing entities named in the reply text). Scope to cited-in-reply only; do NOT link every entity returned by retrieval. Skip when the REFERS_TO edge already exists from this same assistant agent_message to that target. Chat bookkeeping (conversation, agent_message) is excluded.
Step 5c — Closing store skip/forbidden: skip the closing store only when there is no user-visible assistant reply or the user explicitly waived persistence. FORBIDDEN: persisting the user message without storing the assistant reply when you did reply; ending the turn without the closing assistant store when you produced a user-visible reply.
Rapid-fire and edit-heavy sessions: the store-first rule applies even during many small, fast instructions (inline edits, DOM-path directives, quick fixes). If performance requires batching, store at minimum every 3–5 turns with all accumulated entities; never go an entire session without storing. Do not batch away closing assistant stores — each user-visible reply must still be stored via the closing action or an explicitly batched equivalent that includes every assistant reply text. Skipping Neotoma for an entire rapid-fire session is the highest-priority failure mode to avoid.
Backfill: if the user asks to store missed earlier turns, run /learn after a persistence gap, or run a compliance pass for prior turns, store every prior user and assistant turn that was missing, partially stored, or inaccurately stored. When the visible transcript is available, store each `conversation_message.content` exactly as shown to the user, including the initial user message and the final assistant reply text; do not substitute a planned draft, summary, or paraphrase. Use the same stable turn_key when correcting an inaccurate stored turn so the new observation repairs that turn instead of creating a disconnected duplicate. If the exact transcript is unavailable, store a concise note listing the unavailable turns and why exact content could not be recovered. Link all backfilled messages to the conversation in the same turn. Do not defer to a later session.

[DATA MODEL]
Neotoma separates **immutable bytes** from **semantic entities**. **Source** is a first-class primitive: rows in the `sources` table hold content-addressed raw bytes (SHA-256 deduplication per user). **Source** is **not** an `entity_type` — you will not see it in entity-type lists. **Interpretation** is an optional versioned extraction run tied to a source. **Observations** attach facts to entities and may carry `source_id` and `interpretation_id`. **Entity snapshots** are reducer output over observations. Canonical MCP tool **`store`** accepts entities-only, file-only (bytes → new source, no entity observations required in that call — use for archival / ingest-before-extract), or combined file + entities + optional `interpretation`. Chat- or tool-sourced structured writes often keep `interpretation_id` null. For file-derived field provenance, use combined `store` with `interpretation` (e.g. `source_ref: "unstructured"`) or `create_interpretation` on an existing `source_id`.

[GUEST ENTITY SUBMISSION]
Use **`submit_entity`** with `entity_type` plus a `fields` object for types that have an active `submission_config` row (operator-seeded only; the repo does not seed default submission rows). Use **`submit_issue`** when the GitHub-first issue orchestration path is required (unchanged behavior). Thread follow-ups: **`add_entity_message`** (`entity_id`, `message`). Read-back: **`get_entity_submission_status`** (`entity_id`, optional `guest_access_token`). List recent rows: **`list_entity_submissions`**. Pull external state: **`sync_entity_submissions`** (defaults to GitHub issue sync when `entity_type` is `issue`). HTTP equivalents: `POST /submit/:entity_type`, `POST /submit/:entity_type/:entity_id/message`, `GET /submit/:entity_type/:entity_id` (optional `guest_access_token` query param).
PII stripping checklist (REQUIRED before every `submit_issue` or `add_issue_message` call):
- Strip real names, email addresses, and phone numbers from the issue body; replace with `<NAME>`, `<EMAIL>`, or `<PHONE>` placeholders.
- Replace private identifiers (session IDs, internal user IDs, account numbers, order IDs) with their Neotoma `entity_id` (e.g. `ent_xxx`) or a generic label (e.g. `<USER_ID>`); never use human-readable private values.
- Never include API keys, tokens, credentials, or secrets in any form.
- Never include verbatim private data excerpts — financial amounts tied to a real person, health or medical details, home addresses, or similarly sensitive fields; replace with a neutral description (e.g. "a transaction amount", "a health condition").
- Replace quoted conversation text that could identify a real person with a neutral paraphrase (e.g. instead of quoting a user's name and message, write "the user reported encountering…").
- After applying the above, re-read the composed issue body once before calling `submit_issue` or `add_issue_message` and confirm no PII remains.
Issue entity linking: IMMEDIATELY after **`submit_issue`** returns, call **`create_relationships`** with REFERS_TO edges from the returned issue entity_id to every Neotoma entity that motivated or is referenced by the issue. Likewise when **`add_issue_message`** references existing entities, link them in the same turn using REFERS_TO. FORBIDDEN: filing an issue or adding a message that references entities without creating the graph edges in the same turn. FORBIDDEN: calling `submit_issue` or `add_issue_message` without first completing the PII stripping checklist above. Scope: link the entities that the issue concerns — do not link bookkeeping entities (conversation, agent_message) that happened to be in context.
Issue entity store on submit_issue: when **`submit_issue`** returns, ensure the issue is present in Neotoma as a fully-populated entity. **`submit_issue`** creates a local issue row automatically; use the returned `entity_id` to confirm and enrich via **`store`** (with `target_id` from the returned entity_id) or **`correct`** with `entity_type: "issue"`, `github_number` (when a GitHub mirror was created), `github_url`, `title`, and `status: "open"`, if those fields were not already set. Use the `github_number + repo` identity rule so subsequent retrieval resolves to the same entity without creating a duplicate. FORBIDDEN: ending a turn in which `submit_issue` was called without verifying that the issue entity carries `github_number`, `github_url`, `title`, and `status` in its Neotoma snapshot.

[CROSS-INSTANCE SYNC — PEERS]
Register peers with **`add_peer`** / HTTP `POST /peers` (returns `shared_secret` when using `shared_secret` auth without an explicit secret; optional **`sync_target_user_id`** is the receiver `user_id` on the peer for outbound `/sync/webhook`). Inspect: **`list_peers`**, **`get_peer_status`** (includes **`remote_health`**: `/health` probe + semver compat vs this server). Remove: **`remove_peer`**. Inbound notifications: operator `POST /sync/webhook` on the *receiving* instance with body signed using `X-Neotoma-Sync-Signature-256` (same HMAC scheme as subscription webhooks: `sign(secret, rawBody)`). Payload must include `sender_peer_id`, `sender_peer_url`, `target_user_id`, `entity_id`, `source_observation_id`, and optional `guest_access_token` for guest reads on the sender. **`subscribe`** accepts optional `sync_peer_id`: when an event carries `source_peer_id` equal to that value, webhook delivery is skipped (loop prevention). Replicated writes use `observation_source: "sync"` and stamp `source_peer_id` on observations. **`sync_peer`** runs bounded outbound webhook fan-out (requires **`NEOTOMA_PUBLIC_BASE_URL`**, **`NEOTOMA_LOCAL_PEER_ID`**, and peer **`sync_target_user_id`**). **`resolve_sync_conflict`**: **`prefer_remote`** re-fetches remote guest snapshot when **`sender_peer_url`** (and optional **`guest_access_token`**) are supplied; **`prefer_local`** retains local state (use **`correct`** for field edits). Substrate subscriptions remain the primary steady-state path; see **`docs/subsystems/peer_sync.md`**.

[SUBSTRATE SUBSCRIPTIONS]
Use **`subscribe`** / HTTP `POST /subscribe` to watch substrate events for specific **`entity_types`**, **`entity_ids`**, and/or **`event_types`** — at least one of those three filters MUST be non-empty (no firehose). Set **`delivery_method`** to `webhook` or `sse`. For `webhook`, **`webhook_url`** is required (HTTPS in production; localhost HTTP allowed in dev); **`webhook_secret`** is optional (server generates one when omitted). Inspect: **`list_subscriptions`**, **`get_subscription_status`**. Stop: **`unsubscribe`**. For `sse`, open **`GET /events/stream?subscription_id=<id>`** (same auth as the API) to receive the event stream. Peering loop prevention on subscriptions is the same optional **`sync_peer_id`** field described under [CROSS-INSTANCE SYNC — PEERS].

[STORE RECIPES]
MUST NOT list, glob, or read MCP tool descriptor/schema files for chat, attachment, or entity-extraction flows. Use only the recipes below. Tool parameters: **`store`** (entities, idempotency_key, relationships, interpretation?, file_path|file_content+mime_type, file_idempotency_key?); create_interpretation(source_id, entities, interpretation_config?, relationships?); create_relationship(relationship_type, source_entity_id, target_entity_id); create_relationships(relationships). Response IDs: combined store → structured.entities[].entity_id and unstructured.asset_entity_id.
Entities **`entities[]` shape:** put every intended observation field at the **top level** of each object **next to** **`entity_type`** (one flat JSON object per entity). FORBIDDEN: legacy **`{"entity_type":"…","attributes":{…}}`** wrappers — post-v0.5 **`/store`** and MCP **`store`** reject them with **`ERR_STORE_RESOLUTION_FAILED`**; flatten before send. Do not nest first-class fields under **`attributes`** to group them; copy the flat objects in every [STORE RECIPES] example below.
MCP tool choice: prefer **`store`** for all writes (entities-only, file-only, or combined). Deprecated MCP aliases `store_structured` and `store_unstructured` call the same handler as **`store`** until removed; use **`store`** in new prompts and configs.
Relationship batching: define ALL known links in the **`store`** `relationships` array whenever possible. Use { relationship_type, source_index, target_index } for entities in the same request, or { relationship_type, source_entity_id, target_entity_id } for existing entities already known by id. If you need to link an existing conversation/message/source entity to newly created entities, prefer re-including the existing entity in `entities` by stable identity fields and linking by index, or use id-based relationship entries in the same store call. If relationships must be created after a store response, use create_relationships with a relationships array; do not issue one create_relationship call per link when batching is possible.
Interpretation batching: when an agent extracts entities from a stored source (file, email, API detail payload, transcript, or other raw artifact), prefer one **`store`** call with `interpretation: { source_id | source_ref, interpretation_config }` plus any `relationships`. Use `source_ref: "unstructured"` for the raw file source in a combined file+entities store, and `source_ref: "structured"` only when intentionally interpreting the generated JSON source. Use `create_interpretation` when the source already exists and you are adding a post-hoc interpretation; use list_interpretations to review interpretation runs for a source. Omit `interpretation` for ordinary user-stated/chat-native structured facts; those observations intentionally keep `interpretation_id = NULL`.
Immutability: observations and sources are immutable after creation. correct creates a new priority-1000 observation that wins in snapshot reduction; it does not edit or delete prior observations.
Turn identity: use host conversation_id/thread_id/session_id and turn_id when available. If the host does not provide them, derive ONE stable synthetic conversation id for the visible chat/thread and reuse it for every turn in that thread; combine it with a monotonically increasing turn index or per-turn timestamp. Put that stable value on the `conversation` entity as `conversation_id`; set message `turn_key = "{conversation_id}:{turn_id}"`; set idempotency_key unique per turn (e.g. conversation-{conversation_id}-{turn_id}-{timestamp_ms}). conversation_id MUST be unique per conversation but stable within that conversation; do not mint a new conversation entity for each user message in the same visible chat. Unscoped turn_keys like "cursor:1" cause cross-conversation entity collisions where unrelated messages merge into the same agent_message entity. When the host provides only a generic session prefix, append a conversation-distinguishing suffix (e.g. timestamp of first message, or a short hash of the conversation title) and keep that suffix stable for the thread.
Session UUID bridge (Claude Code): in Claude Code contexts the SessionStart hook creates a `conversation` entity keyed by the raw session UUID while the MCP agent creates a slug-keyed entity (conversation_id = derived slug). Call `get_session_identity` once per session — at user-phase Step 2 retrieval or Step 3 store time — and include `session_uuid: <UUID returned by get_session_identity>` on the slug-keyed `conversation` entity. This cross-references the hook-created UUID entity and the agent-created slug entity so timeline events, observations, and conversation_turn rows written by hooks can be correlated with the MCP conversation without server-side coalescing. FORBIDDEN: omitting `session_uuid` on the `conversation` entity in Claude Code contexts when `get_session_identity` is available and returns a session UUID.
Fallback IDs: if the host provides no conversation_id/turn_id at all, derive a stable synthetic conversation id from the visible thread context (e.g. "chat-<topic-or-first-message-hash>-<session-start-timestamp>") and reuse it; use idempotency_key "conversation-{synthetic_conversation_id}-<turn>-<timestamp_ms>" and turn_key "{synthetic_conversation_id}:<turn>" (e.g. chat-neotoma-hooks-2026-05-05:5). When no topic or thread context is available, use a session-epoch-scoped fallback: turn_key "chat-<session_epoch_ms>:<turn>" (e.g. chat-1747344000000:1) and idempotency_key "conversation-chat-<session_epoch_ms>-<turn>-<timestamp_ms>", where session_epoch_ms is the millisecond timestamp of the first message in the current session (fixed for the session lifetime). FORBIDDEN: using a fresh fallback conversation id or generic "chat:<turn>" prefix for every message in one UI conversation. FORBIDDEN: using bare "chat:<turn>" (without a session-epoch or other session-distinguishing suffix) as a turn_key fallback — bare chat:N values reuse across sessions and cause cross-session entity merges.
observation_source (write classification): **`store`** (MCP) and HTTP POST `/store` accept an optional `observation_source` describing the KIND of write, orthogonal to numeric `source_priority` and to AAuth attribution. Values: `sensor` (deterministic tool/telemetry emission), `workflow_state` (state machine transitions), `llm_summary` (LLM-authored content — the default; omit the field for ordinary chat extraction and attachment recipes), `human` (direct human edit or acceptance), `import` (batch / ETL ingestion), `sync` (cross-instance peer replication). Optional **`source_peer_id`** on **`store`** stamps observations for loop prevention. The reducer uses `observation_source` as a tie-break after `source_priority` so classified writes beat unclassified legacy rows; `sync` ranks lowest in the default priority order. Set `observation_source` only when the writer is NOT an LLM summary: a sensor emitter, a state-machine step, a human confirmation, a bulk import, or a peer sync replay. One value applies to every observation produced by the request.

[STORE RECIPES] user-phase (one call per user message at turn start)
Shape: **`store`** with entities = [ { entity_type: "conversation", conversation_id: "<stable conversation id>", title? }, { entity_type: "conversation_message", role: "user", sender_kind: "user", content: "<exact message>", turn_key: "{conversation_id}:{turn_id}" }, …optional extracted entities… ]. If bounded retrieval found the existing conversation, include `target_id: "<conversation entity_id>"` on the conversation object instead of relying on title matching; keep `conversation_id` stable when known. Response indices: entities[0] = conversation id, [1] = message id, [2..] = extracted entities. Canonical entity_type is `conversation_message`; the legacy `agent_message` is accepted as an alias and resolved to `conversation_message` at write time for pre-v0.6 clients.
Relationships: always include PART_OF from message (index 1) to conversation (index 0). For each entity the turn created OR updated (indices 2, 3, …), add REFERS_TO from message (index 1) to that entity. "Updated" means any entity that existed before this turn and received a new observation via **`store`** or correct during this turn. Skip the REFERS_TO edge when it already exists from this same agent_message to that target.
Naming: when storing chat artifacts, keep labels human-readable through schema fields, not a payload `canonical_name` field. For `conversation`, keep `title` topical and update it only when the thread's central topic materially changes or expands enough that the old title becomes misleading. For `conversation_message`, use stable `turn_key` for identity and put the actual message body in `content`; do not add `canonical_name` to the payload.
Conversation entity maintenance: when the conversation pivots materially from its initial scope — for example shifting to a different product, system, or objective that the original `title` and `scope_summary` no longer reflect — update both fields on the conversation entity in the same turn via `store` (include `target_id` from the bounded retrieval result to avoid creating a duplicate conversation). A material pivot is one where a user reading only the stored `title` and `scope_summary` would be misled about what the conversation covers. The `conversation_id` value MUST remain unchanged across scope updates; do not generate a new `conversation_id` or create a new conversation entity to represent the new scope — all turns in one visible chat share one conversation entity. FORBIDDEN: leaving `title` and `scope_summary` stale after a material scope change when Neotoma is available. FORBIDDEN: minting a new conversation entity or new `conversation_id` when updating scope on an existing conversation.
Canonical-name scope: `canonical_name` is resolver/entity output, not a general structured payload field. Use schema fields such as `title`, `name`, `subject`, `summary`, `description`, `content`, `turn_key`, `role`, `status`, or notes for human-readable labels and extra semantics. Do not send `canonical_name` unless the target schema explicitly declares it as a field; otherwise it becomes an unknown/raw fragment.
Chat context scope: include bounded, host-provided context when available, but keep it optional and non-identity-bearing. Stable session context MAY go on `conversation` (e.g. `client_name`/`harness`, `workspace_kind`, `repository_name`, `repository_root`, `repository_remote`, `scope_summary`); use `repository_root` for the absolute local checkout path when useful, not a separate `system_path` field. Volatile per-turn context belongs on `conversation_turn` (e.g. `working_directory`, `git_branch`, `active_file_refs`, `context_source`). Prefer linking to a durable `repository` or `project` entity with REFERS_TO when codebase identity matters across chats. Do NOT put repository/workspace fields in `canonical_name_fields`; local paths can move and include usernames. Do NOT store raw git status, terminal output, file contents, secrets, or large open-file lists by default.
Extraction: if the message implies any entity (purchase, task, event, person, place, etc.), append one or more entities with a descriptive entity_type and the properties the message implies — no fixed schema; server accepts arbitrary fields. Do not call list_entity_types before storing; do not reason about schema. See [ENTITY TYPES & SCHEMA] for type reuse rules.
idempotency_key: "conversation-{conversation_id}-{turn_id}-{suffix}" or, on fallback, "conversation-chat-<session_epoch_ms>-<turn>-{suffix}" (where session_epoch_ms is the millisecond timestamp of the first message in the current session).

[STORE RECIPES] attachment (one call; file + entities together)
Step 1 — Parse: if the file is a PDF, call parse_file first to extract readable text; if it is text, CSV, JSON, or Markdown, read it directly; if it is an image, use vision directly. If parse_file yields no usable text or images, store the raw file only. CRITICAL: parse_file is inspect-only — it extracts text but does NOT store anything in Neotoma. Always proceed to Step 3 to persist the binary, even when parse_file succeeds.
Step 2 — Extract: extract entities using one snake_case field per fact, no invention, and a descriptive entity_type that matches the document. Common types when intent is clear: receipt, invoice, note, contract, person, contact, company, task, event, transaction. Put all extracted fields at the top level of the entity object alongside entity_type (flat object); do NOT nest them under raw_fragments or attributes — those are not accepted schema fields on most entity types and will be silently dropped as unknown_fields.
Step 3 — Store: one **`store`** call with entities = [ { entity_type: "conversation", conversation_id: "<stable conversation id>", target_id? when retrieval found an existing conversation, title? }, { entity_type: "conversation_message", role: "user", sender_kind: "user", content: "<exact message or 'Attached: <filename>'>", turn_key: "{conversation_id}:{turn_id}" }, …extracted entities… ], idempotency_key "conversation-{conversation_id}-{turn_id}-{timestamp_ms}", relationships = [ { relationship_type: "PART_OF", source_index: 1, target_index: 0 }, …optional REFERS_TO links from message to extracted entities… ], and file_path: "<absolute file path>" or file_content + mime_type. Path resolution: host @-references (e.g. @/path/to/file.pdf) are display hints, not guaranteed filesystem paths — resolve to an absolute path and verify the file exists (ls or equivalent) before passing to store; if the path does not exist, search nearby for the file before failing.  Optional file_idempotency_key: "file-<short-slug>" (e.g. file-sample-receipt-pdf).
Step 4 — EMBEDS: call create_relationship with relationship_type "EMBEDS", source_entity_id = step3_response.structured.entities[1].entity_id (the user agent_message), target_entity_id = step3_response.unstructured.asset_entity_id.

[STORE RECIPES] screenshot/image
When the user provides a screenshot or image, extract structured data from the visible content (people, messages, dates, criteria, tasks, events, offers, transactions, etc.) and store it using the attachment recipe above: entities = [conversation, user message, …extracted entities], file_path or file_content+mime_type for the image, relationships PART_OF (message→conversation) and REFERS_TO (message→each extracted entity), then EMBEDS (message→file entity). Extract every distinct entity visible in the image before responding.

[STORE RECIPES] chat details and fallbacks
Overwriting between branches is acceptable; users can view historical turns or branches via observation history (list_observations). For reverted turns, optionally call create_relationship(SUPERSEDES, new_message_id, previous_message_id). Supported relationship types: PART_OF, REFERS_TO, EMBEDS, SUPERSEDES, etc. (see MCP spec).
Fallback when inline relationships are not supported by a client: (1) call **`store`** with entities and idempotency_key; (2) call create_relationship(PART_OF, response.entities[1].entity_id, response.entities[0].entity_id); for extracted entities, call create_relationship(REFERS_TO, response.entities[1].entity_id, response.entities[N].entity_id) for each.
create_relationship quick reference (when the target is outside the same store request, e.g. EMBEDS to a file entity): create_relationship(relationship_type, source_entity_id, target_entity_id). Use response.entities[N].entity_id from a prior **`store`** response; no tool-schema read required.

[RETRIEVAL]
Retrieval-first for common query types: when the user asks about tasks, schedule, contacts, notes, issues, events, finances, decisions, or commitments — whether on mobile, desktop, or any harness — MUST run a bounded Neotoma retrieval pass first (retrieve_entities with the relevant entity_type, or retrieve_entity_by_identifier for named items) before answering or falling back to native device integrations (Reminders, Calendar, Contacts, etc.). Do not fall back to other tools or say "I don't have that information" until Neotoma retrieval has been attempted and returned no relevant results.
Entity-type discovery before retrieval: when the user asks about a concept and the entity_type is not one of the well-known types (transaction, task, event, person, contact, company, receipt, note, location, place) and the session has not already cached a type list, call `get_entity_type_counts` or `list_entity_types` (with a relevant keyword) BEFORE issuing retrieve_entities — do not assume a type string from the concept label alone. For example, if the user asks "find my invoices" and no `invoice` type has been seen this session, call `list_entity_types` with keyword "invoice" first; if `invoice` does not appear but `receipt` or `billing_record` does, use the registered type. FORBIDDEN: calling retrieve_entities with an assumed entity_type when the type has not been confirmed to exist in this Neotoma instance and the session type cache is empty or stale.
Query shape: use retrieve_entity_by_identifier for concrete identifiers (names, emails, ids, exact titles). For plural/category or list-intent queries (e.g. "last N transactions", "recent tasks", "latest events"), prefer retrieve_entities scoped by target entity_type with an explicit limit or time window; do not rely on identifier lookup for generic category phrases.
Named entity-type routing: when the user asks about a named entity type — such as "newest plans", "my tasks", "recent notes", "open issues", "latest events" — call retrieve_entities with the matching entity_type parameter directly (e.g. entity_type: "plan", entity_type: "task"). FORBIDDEN: searching conversation history, agent_message rows, or chat context as a substitute for a direct retrieve_entities call when the user is asking about a typed entity. Do not fall back to conversation search until retrieve_entities with the appropriate entity_type has been attempted and returned no relevant results. For recency queries ("newest", "latest", "most recent"), sort by updated_at or created_at descending and apply a reasonable limit (10–25).
Tool reference: use retrieve_entity_snapshot for current or historical entity state (`at` timestamp; `format` markdown by default or json), retrieve_field_provenance for field→observation→source tracing, list_relationships and get_relationship_snapshot for relationship reads/provenance, retrieve_related_entities for n-hop traversal, retrieve_graph_neighborhood for complete graph context, retrieve_file_url for signed source-file URLs, and list_recent_changes for change feeds.
Guardrails and answer grounding: start with small, targeted queries and expand only on ambiguity or low confidence. Avoid broad scans unless necessary. Use retrieved Neotoma facts when relevant; if bounded retrieval finds no relevant context, proceed normally without inventing memory-backed claims.
Publication-recency: for prompts asking for "recently published" or equivalent publication-time ordering, sort by publication timestamp (`published_date` / `published_at`) descending, not by observation recency (`last_observation_at` / `updated_at`). Use a limit/page strategy large enough to avoid subset bias and deduplicate by entity_id before answering.
Entity-type cardinality: when the user asks how many entities exist per entity_type (counts by type, histogram, sorted type totals), answer from aggregated stats first — MCP get_entity_type_counts or OpenAPI getStats / HTTP GET /stats → entities_by_type. list_entity_types lists registered schemas; `field_count` there is schema field width — never report it as entity row counts or substitute it for cardinality. Exhaustive per-type totals via retrieve_entities with limit 1 per type are an expensive last resort, not the default presentation for "all types by count". See [ERRORS & RECOVERY] for getStats-unreachable behavior.
Bounded completeness: for list/count answers derived from entity graphs, run a bounded completeness pass before replying — check likely equivalent containers/identifiers and common relationship variants in that domain, deduplicate by entity_id, and report the reconciled total (or clearly note remaining ambiguity).
Schema and entity agent instructions: when `retrieve_entity_snapshot` or `retrieve_entities` returns a `schema_instructions` field, agents MUST treat that string as behavioral context for the entity type and apply it to the current turn — it is a markdown instruction from the schema's `agent_instructions` declaration. When a `entity_instructions` field is present, agents MUST treat that string as behavioral context for the specific entity and apply it, superseding or extending `schema_instructions`. Both fields are optional and may be absent; when absent, no special behavior is required. FORBIDDEN: ignoring `schema_instructions` or `entity_instructions` when present in a retrieval response.

[PROVENANCE]
Source provenance (required): every entity stored in Neotoma MUST carry traceable source data. For file-derived data (PDF, CSV, spreadsheet export, JSON dump), use the combined store path — entities array for structured fields AND file_path (or file_content+mime_type) for the source file in the same store call, so the raw artifact is preserved as the source row; when the agent parsed or interpreted that raw artifact, include an explicit `interpretation` block so observations link to both the raw `source_id` and `interpretation_id`. Include `source_file` with the filename. For API- or MCP-tool-sourced data (balance check, transfer status, calendar event, email), include `data_source` on each entity identifying the tool, endpoint, and date (e.g. "Wise API GET /v1/transfers/12345 2026-03-15"), and store the raw response payload as `api_response_data` (or a meaningful subset when very large) so original values are preserved alongside the extracted fields. When the payload is preserved as a source and entities are extracted from it, use **`store`** with `interpretation` or `create_interpretation` to capture the agent-authored extraction run. When the tool only returned a list/summary row, hydrate via the matching detail endpoint before persisting (see [COMMUNICATION & DISPLAY] `Depth of capture`) and keep both layers under `api_response_data.list` / `api_response_data.detail`. FORBIDDEN: storing entities with no traceable source data (no file, no data_source, no source_file) unless the data is purely user-stated in chat (traced via the conversation entity chain).
Multi-row `data_source` identity (batch stores): When persisting multiple **distinct** external-tool records in one **`store`** call, each entity MUST use a **different** `data_source` string that embeds that row's stable upstream identifier (e.g. email `message_id`, calendar `event_id`, transfer `id`, row primary key). Reusing the **identical** `data_source` on several rows (e.g. five Gmail messages all labeled with the same tool+date string) can trigger heuristic identity resolution and collapse them into one entity. FORBIDDEN: the same `data_source` on different logical rows in a single batch. REQUIRED: per-row `data_source` like `"<Tool> <operation> id=<stable_id> <ISO-date>"`, and duplicate the id in a native payload field (`gmail_message_id`, etc.) when available; alternatively issue one **`store`** call per distinct row. If a merge mistake already landed, repair with per-row stores or `correct` rather than leaving one entity represent many.
Three-layer analysis of a named entity: when a turn analyzes a named entity (person, company, project, account, property, etc.) from source material, persist all three layers in the same turn before responding: (1) the raw source artifact or source entity, (2) the named entity updated with all sourced facts, (3) a separate synthesized note/report entity capturing the derived conclusions used in the answer. Link source, named entity, and synthesis with REFERS_TO or EMBEDS as appropriate.
Reuse pre-existing sources: if the raw source already exists in Neotoma (e.g. an earlier meeting transcript, email, or uploaded document), retrieve that source entity and explicitly link the current conversation-derived entities to it in the same turn; do not rely on the earlier store remaining discoverable without a relationship.
Source content retrieval: raw source files stored via the combined store path are downloadable via `GET /sources/:id/content`. Observations carry `source_id` linking to the source row. When building UIs or reports that display source labels, provide a link or action to view the raw source content via this endpoint so users can inspect the original artifact (PDF, CSV, JSON, image, etc.) that backs any data point. The endpoint serves the file inline for browser-viewable types (PDF, text, images) and as a download for others.
Unstructured payload retention: user-provided files and unstructured payloads (paths, @-references, attachments, uploads, pasted file or binary content) MUST be persisted in Neotoma in the same turn via the unstructured path (file_path or file_content+mime_type) together with [STORE RECIPES] attachment (conversation + agent_message + EMBEDS when a file entity is returned). Host-only copies (moves under Desktop/Downloads, local checklists, or repo-adjacent folders) are NOT sufficient retention when Neotoma is in scope; always ingest the artifact into Neotoma unless the user explicitly waives file retention. Structured entities inferred from the content (any entity_type) are additive and SHOULD still be stored in the same turn when relevant; they do not replace unstructured preservation of the user-supplied file or blob. parse_file is NOT sufficient retention — it is a read-only inspection tool; retention requires a store call with file_path or file_content+mime_type (see [CONVENTIONS] parse_file vs store).
Synthesized deliverables: for reviews, reports, plans, audits, comparative analyses, legal research, competitive analysis, market research, technical investigation, or any multi-step research combining multiple external sources, store the synthesized result as a structured entity (e.g. legal_research, competitive_analysis, market_research, technical_research, report) with title, subject, conclusion, key_findings, sources, caveats, and research_date. Link related existing entities with REFERS_TO. Do not respond with research findings without storing them in the same turn.
Analysis/briefing durability: when the user asks for analysis, insights, or a briefing about a named entity based on source material, do not rely only on chat agent_message rows for durable capture; persist a structured note/report/research entity detailed enough to reconstruct the substance of the answer, then link it to the analyzed entity and the source before responding.
Agent-authored deliverable artifacts: when the agent creates or materially edits a markdown, text, JSON, CSV, or similar file that is itself the substantive deliverable of the turn (report, plan, analysis, draft, audit, brief, post draft, execution report, checklist), store the file itself in Neotoma in the same turn via the combined store path, persist a structured note/report/research entity summarizing why it matters, and link the file asset, the structured deliverable entity, and the originating conversation/user message. Repo-only or working-tree-only copies are not sufficient durable storage.
Session-derived chat artifacts: when you store any entity created from the current conversation in a separate **`store`** call (after the conversation entity already exists), link provenance in the same turn — create_relationship(REFERS_TO, source_entity_id=<derived entity>, target_entity_id=<conversation entity_id>), or create_relationship(REFERS_TO, source_entity_id=<user agent_message that prompted the save>, target_entity_id=<derived entity>) to match the unified recipe. FORBIDDEN: leaving session-derived entities with no relationship to the conversation or originating message. CLI and follow-up batches (`neotoma store` with `--file`/`--entities` and `--file-path` in offline or API backup mode) are equivalent to a separate MCP store for this rule — after persisting file_asset or any other structured rows the chat prompted, use create_relationship in the same turn so nothing is orphaned (prefer REFERS_TO from the prompting user agent_message to each new entity). Multi-file loops MUST NOT end the turn until every new entity is linked.
Per-turn linkage invariant: every non-bookkeeping entity touched in a turn (created, updated, or materially cited by the reply) MUST carry a REFERS_TO edge from either the user agent_message (creates/updates) or the assistant agent_message (reply-cited) of that turn. observations.source_id is not a substitute for this edge when the entity is graph-queried for turn provenance.

[TASKS & COMMITMENTS]
Base rule: create a task when the user expresses intent, obligation, or future action ("I need to", "remind me", deadlines). Unless the user says no reminders/tasks, create a task with due_date when available and link it to the relevant person or entity.
Outreach and reply-drafting: when you produce or refine outbound text (email, DM, social reply, thread reply) that commits the user to a future step with a named counterparty (e.g. "I'll reach out when…", "I'll send X after Y", "I'll loop back once…"), create a task in the same turn describing that follow-up, set due_date when inferable otherwise capture timing and blockers in notes, and link the task to the counterparty contact with REFERS_TO (reuse contact after bounded retrieval, create if missing). FORBIDDEN: omitting the task when that commitment is explicit or clearly implied. Do not treat "draft only" or copy edits as exempt. Closers with no concrete follow-up do not require a task.
Awaiting-reply task rule (outbound email/message without inbox response): during external-tool scans (inbox triage, sent-folder review, thread summarization), when an outbound email or message to a named recipient has no inbound reply visible in the inbox or thread within a reasonable window (default: outbound sent ≥ 3 days ago with no subsequent inbound from the same recipient in the same thread), create an `awaiting_reply` task in the same turn: `entity_type: "task"`, `kind: "awaiting_reply"` (or equivalent tag), `subject` summarizing what was asked, `counterparty_contact_id` linking to the recipient, `outbound_message_id` linking to the source email/message, `sent_at` from the outbound, `due_date` set to a follow-up date when the outbound named one (otherwise null), `status: "open"`. Link the task to the counterparty contact via REFERS_TO and to the outbound `email_message` (or equivalent) via REFERS_TO. Reuse the existing awaiting_reply task on subsequent scans of the same thread — do not mint a new one per scan; close it (or set status `resolved`) when an inbound reply arrives. FORBIDDEN: completing an inbox triage or sent-folder review without surfacing awaiting_reply tasks for outbound messages that meet the window. Acceptable exclusions: outbound newsletter/automated sends, one-line confirmations the user explicitly does not want tracked, or recipients the user has marked do-not-follow.
Awaiting-reply task rule (email thread ingestion — outbound-only threads): when ingesting an email thread that contains ONLY outbound messages with no inbound reply from the recipient, immediately create a `task` entity with `entity_type: "task"`, `status: "awaiting_reply"`, `description: "Awaiting reply to: <subject>"`, and `due_date: null` (unless a follow-up date is stated in the outbound body). Apply this rule regardless of how long ago the outbound was sent — the absence of any inbound message in the thread is sufficient signal; no time-window check is required. Link the task to the recipient contact and the source email thread via REFERS_TO. FORBIDDEN: ingesting an outbound-only email thread without creating this awaiting-reply task entity in the same turn.
Scheduling cues in correspondence: when email, chat, screenshot, or pasted message text implies arranging a future meeting or call with a named person (e.g. pencil in, another for [month], book next, sync again, catch up later), create a task in the same extraction/store turn to follow up and schedule it, set due_date when a month or date is inferable (otherwise capture the timeframe in notes), and link the task to the relevant contact or person (REFERS_TO from the user agent_message to the task when batching in one store, plus task→contact if the recipe supports it, or create_relationship after store). FORBIDDEN: omitting a task when this scheduling obligation is explicit or clearly implied.
TodoWrite is session-local: the host tool `TodoWrite` (Claude Code session task list) exists only for in-turn tracking of the current session's work. It does NOT satisfy the Neotoma store protocol. When a turn produces follow-up tasks — actions to take in a future session, commitments to carry forward, or work items the user should be able to query later — MUST also store each task via `store` with `entity_type: "task"` in Neotoma in the same turn. FORBIDDEN: using `TodoWrite` alone at the end of a turn to record persistent follow-up tasks when Neotoma is available; those tasks must be stored in Neotoma or they will be lost when the session ends.

[ENTITY TYPES & SCHEMA]
Schema-agnostic for chat: for storage from chat, use a descriptive entity_type and whatever properties the message implies; server accepts arbitrary fields and infers schema. For the well-known types listed here, do NOT call list_entity_types before storing — proceed directly. Examples of entity_type (not fixed shapes): transaction, task, event, person, contact, company, receipt, note, location, place, issue, organization, project, outreach_interaction, legal_research, competitive_analysis, market_research, technical_research, report. For any other entity_type not in this list and not already in the session's cached type list, call `get_schema_recommendations` (with the candidate entity_type) or `list_entity_types` with a relevant keyword BEFORE the first store call that uses that type — this catches declared schemas and prevents type proliferation. For non-chat flows (imports, workflow automation, structured data extraction), always call `get_schema_recommendations` or `list_entity_types` before the first store for a given entity_type regardless of whether it appears in the common list above.
Schema-check before storing known entity types: before storing with an entity_type that has a registered schema (i.e. it appears in `list_entity_types` results, or a prior store for that type returned schema-field metadata, or the type was retrieved from Neotoma this session), check its declared field names first — use `describe_entity_type` with the entity_type for the full SchemaDefinition (fields, canonical_name_fields, temporal_fields, reference_fields, aliases, merge_policies), or use `get_schema_recommendations` with the entity_type, or retrieve one existing entity of that type via `retrieve_entity_by_identifier` or `retrieve_entities` to inspect the snapshot's field names. Use declared fields where they fit the data exactly. When the data has no declared home, invent additional snake_case fields for that content — do not omit data because no declared field matches. FORBIDDEN: storing entities of a known registered type using entirely invented field names without first checking what declared fields exist.
Schema-first store for unfamiliar types: before the FIRST `store` for an entity_type that the agent has not used this session AND that is not in the common-types short list under "Schema-agnostic for chat" above, call `list_entity_types` with a `keyword` matching the type name to discover whether a registered schema exists. If a registered schema is found, follow the schema-check rule above; if no registered schema exists, proceed schema-agnostic. This is additive to the schema-check rule: schema-check covers "you already know it's registered"; schema-first store covers "you don't yet know whether it's registered." FORBIDDEN: storing with an unfamiliar entity_type via intuited fields without a schema check when one introspection call would have caught the existing schema. Skip for common chat types (transaction, task, event, person, contact, company, receipt, note) where schema-agnostic write is the documented default.
Full data fidelity: every field of source data MUST land in a stored field — either a declared schema field or a clearly named invented field. Do not silently drop data. If `unknown_fields_count > 0` in a store response, it means fields were not projected onto declared schema columns; immediately re-store those entities using the correct declared field names (or correct the entities via `correct`) before proceeding to the closing assistant store. FORBIDDEN: treating a store response with `unknown_fields_count > 0` as complete and proceeding without repair.
Entity-type reuse check: before storing with an entity_type that is not one of the common types above and not in the agent's cached type list for this session, check for semantic equivalents among existing types — singular vs plural (place/places), synonyms (person/contact), prefix variants (social_post/social_media_post). If a match exists, use the established type. When uncertain, prefer the type with more existing entities. Call list_entity_types with a keyword search if the cached list is stale or unavailable. FORBIDDEN: creating a new entity_type when a semantically equivalent one already exists — this prevents type proliferation and duplicate schemas.
Schema evolution: schemas evolve via update_schema_incremental. Both adding fields (minor version bump) and removing fields (major version bump) are supported. Use fields_to_remove to prune noisy or inappropriate fields accumulated via auto-enhancement or early inference that do not represent the entity type. Removed fields are excluded from snapshots via schema-projection filtering but all observation data is preserved; re-adding a removed field restores it in snapshots. At least one field must remain after removal.
Existing-entity correction: when fixing a previously stored entity's inaccurate values or normalizing ad hoc fields into an established schema, update the existing entity via `correct` rather than creating a replacement duplicate. Prefer canonical schema fields when a type already has an established shape; fold extra detail into those fields or evolve the schema explicitly if needed.
Entity-type consistency within a workflow: within one import or workflow, pick one canonical entity_type and keep it consistent across all records in that batch. If the user asks for generic transactions or the source is statement-like financial rows (bank/card/account), default to entity_type "transaction" and store source-specific details as fields (provider, account_suffix, value_date, concept, row ids). Use narrower subtypes only when the user explicitly requests them or when existing records for that same workflow already use that subtype; do not mix generic and subtype entity types in the same batch.
Instruction scope (for /learn and future edits): keep MCP interaction instructions generalized and workflow-level (ordering, required steps, safety, idempotency, retrieval/store patterns). Avoid embedding domain- or dataset-specific modeling rules (e.g. instructions tailored to one roster, one customer list, or one content collection). When /learn updates MCP instructions, prefer reusable interaction guidance that applies across entities and workflows instead of case-specific data logic.
conversation_message sender semantics: for every `conversation_message` write (canonical entity_type as of v0.6; pre-v0.6 `agent_message` payloads still resolve via alias), always set `sender_kind` (one of `user` | `assistant` | `agent` | `system` | `tool`) alongside the legacy `role` field. `role` stays for backward compatibility but readers should prefer `sender_kind`. For agent-to-agent (A2A) traffic, set `sender_kind: "agent"` and include `sender_agent_id` + optional `recipient_agent_id` (stable identifiers; derive from AAuth thumbprint / clientInfo / agent_sub when available — see [ATTRIBUTION & AGENT IDENTITY]). For the parent `conversation`, set `thread_kind` to `human_agent` (default), `agent_agent`, or `multi_party` to describe the participant topology.

[ENTITY & RELATIONSHIP LIFECYCLE]
Soft delete: delete_entity(entity_id, entity_type, reason?) and delete_relationship(relationship_type, source_entity_id, target_entity_id, reason?) create deletion observations so the entity/relationship is excluded from active snapshots and queries while preserving audit history. Do not treat delete as physical removal.
Restore: restore_entity and restore_relationship create restoration observations that make previously deleted entities/relationships visible again. Restoration is immutable and auditable, not an in-place edit.
Merge and duplicate repair: use list_potential_duplicates(entity_type, threshold?, limit?) as a read-only detector; never auto-merge. Confirm candidate pairs with the user or a repair plan, then call merge_entities(from_entity_id, to_entity_id) to rewrite observations from the duplicate into the target and mark the source merged.
Split over-merges: use split_entity to re-point a predicate-selected subset of an entity's observations onto a new or existing entity when a prior merge or heuristic resolution collapsed distinct entities. split_entity is the inverse of merge_entities, is idempotent via (user_id, idempotency_key), preserves observation content, and leaves typed relationships bound to the source until rebuilt with create_relationship.

[COMMUNICATION & DISPLAY]
Silent storage default: do not mention storage, memory, or linking unless the user asked, except when a turn created, updated, or retrieved Neotoma entities and you are required to show them per the display rule below. Do not describe internal persistence in thought or reply (e.g. "Persisting this turn, then replying", "Storing the conversation first"). When confirming something was stored, use memory-related language ("remember", "recall", "stored in memory") and include one of those phrases.
Proactive storage: use MCP actions proactively. Store when the user states relevant information; store first, then respond. Do not skip store because the user did not ask to save.
Artifact store triggers: when a concrete artifact is approved or finalized in conversation — including but not limited to a plan, schema_design, architectural_decision, decision_record, feature_spec, policy, design_doc, runbook, or migration_guide — store it in the same turn without waiting for an explicit user instruction to save it. Use a descriptive entity_type matching the artifact kind (e.g. `plan`, `schema_design`, `architectural_decision`, `decision_record`, `feature_spec`). Include a `title`, `content` or `summary`, and any relevant metadata fields. Link the stored artifact entity to the active conversation with REFERS_TO from the user agent_message to the artifact entity, consistent with the Session-derived chat artifacts provenance rule (see [STORE RECIPES] `Session-derived chat artifacts`). FORBIDDEN: ending a turn in which an artifact was approved or finalized without storing it when Neotoma is available.
Repo canon is additive, not a replacement: when a user asks to capture a durable principle, tenet, standing rule, mission element, or other strategy-layer canon in a repo document, persist the durable fact in Neotoma in the same turn when Neotoma is available, then update the canonical repo document if the repo is also the requested or established source of truth. Do not treat "this belongs in the repo" as a reason to skip Neotoma.
External tool store-first: when you pull data from any external source (email, calendar, search, web fetch, web scrape, API, file read, or any other tool), extract and store people, companies, locations, events, tasks, notifications, device status, and relationships in the same turn and BEFORE responding. Create tasks for action items (see [TASKS & COMMITMENTS]). Link events/tasks to locations and people. Do not respond with external data until storage is complete.
Per-record extraction checklist (REQUIRED for every external-source record, not just peek/list queries): for each record encountered — every email opened, every calendar event read, every search result hydrated, every transaction listed — run the full extraction pass before moving to the next record. Do not batch "I'll extract later" or skip extraction because the user only asked for a count or summary. The minimum per-record scan covers, in order: (1) **people and organizations** named in the record (sender, recipient, mentioned third parties, employer, vendor) → `contact` / `person` / `organization` / `company` entities; (2) **temporal commitments** (dates, deadlines, scheduled meetings, follow-ups) → `event` and/or `task` entities; (3) **transactional facts** (amounts, currencies, charges, refunds, transfers) → `transaction` entities with normalized amount/currency/date; (4) **locations** (addresses, venues, places mentioned) → `place` / `location` entities; (5) **referenced artifacts** (linked issues, projects, documents, threads) → see "GH issue / org / project extraction" below; (6) **outreach implications** (outbound replies still awaiting response, follow-up commitments) → see "Awaiting-reply task rule" below. Each extracted entity carries its own `data_source` per the rule above and a `source_quote` per "Embedded entity extraction" below when the extraction is inferred from body content. FORBIDDEN: skipping the per-record scan for any reason — including "I just want to count my emails", "show me my last N events", or any other query-only or peek-style intent. Extraction is mandatory on every external tool call regardless of intent; there are no exemptions for summary, count, or peek queries.
GH issue / org / project extraction (extends the per-record scan): when an external-source record (email, calendar invite, chat message, document, web page, or other tool response) names or references a GitHub issue, organization, multi-message project thread, or recurring outreach pattern with a counterparty, store the corresponding entity in the same turn: (1) **GitHub issues** mentioned by URL, number (e.g. "#123"), or title — use `entity_type: "issue"` with `github_number`, `github_url`, `repo` per the canonical identity rule in [ISSUE REPORTING] "GitHub issue URL extraction"; do NOT create a generic `note` placeholder when the canonical fields are recoverable. (2) **Organizations** named as senders' employers, vendors, partners, or sponsors — use `entity_type: "organization"` or `entity_type: "company"` (reuse the established type per "Entity-type reuse check"); link via REFERS_TO from the source `email_message` / `event` / `note` to the organization. (3) **Multi-message projects** — when two or more external records share a project context (same project name in subject, same client engagement, same recurring topic), create a `project` entity with `name`, `start_date`, `status`, and link all member records via REFERS_TO. Reuse the project across turns using bounded retrieval before creating a duplicate. (4) **Outreach interactions** — when an inbound or outbound message represents a substantive interaction with a contact beyond bookkeeping (not a one-line confirmation), create an `outreach_interaction` entity capturing direction (`inbound` | `outbound`), interaction kind (email, call, meeting, dm), contact, summary, and link to the source record via REFERS_TO. Used for relationship-cadence analysis and follow-up tracking.
Depth of capture: list/summary tool responses (e.g. Gmail search_emails, calendar list_events, CRM list_contacts, search result pages, HTTP index listings) are index rows, not the final payload. Before persisting each item you intend to keep, call the corresponding detail endpoint (e.g. read_email, get_event, get_contact, fetch page) and store the richest stable fields it returns (for email: `body_text` / `body_html`, attachment metadata; for events: full description, attendees; for web pages: parsed content). Preserve both provenance layers by keeping the list-row JSON under `api_response_data.list` and the detail-row JSON under `api_response_data.detail`.
Depth of capture — scope cap: hydrate up to ~10 items per turn unless the user explicitly asks for a larger ingestion; surface the rest as a list and offer to continue.
Depth of capture — size cap: if a detail body exceeds ~100 KB, persist it via the unstructured path (`file_content`+`mime_type` or `file_path` on **`store`**) and link the structured entity to the file entity with EMBEDS instead of inlining the raw body into the snapshot.
Depth of capture — sensitivity: for email/DM/document/message-like sources, persist the body but do not echo it back into chat beyond what answering requires. All queries — including peek-style questions ("what are my last 5 emails?") — hydrate for storage; keep the reply at summary level.
Depth of capture — idempotent upgrade: when the entity already exists from a prior summary-only store, hydrate via `correct` on the same `entity_id` rather than creating a duplicate (see [ENTITY TYPES & SCHEMA] `Existing-entity correction`).
Depth of capture — tool-capability awareness: only hydrate when a detail endpoint actually exists and is cheap. Otherwise persist what the list tool returned and set `capture_depth: "summary_only"` on the entity so a later turn can enrich.
Embedded entity extraction: once an external-tool or file payload is in hand (list row, hydrated detail, or full file body per Depth of capture), scan the content for first-class entities embedded in it — e.g. a subscription charge inside a billing email → `transaction` / `subscription`; a meeting proposal inside a message → `event` + `task`; an order inside a receipt → one `order_item` per line; a person mentioned in a thread → `contact`; a location in an itinerary → `place`. Store each embedded entity alongside the container in the SAME **`store`** call as the container (not a deferred turn). Each embedded entity carries its own `data_source`, a `source_quote` (verbatim snippet from the container body supporting the extraction), and normalized fields where present (amount + currency, due_date, counterparty, recurrence, start_time, end_time). Link container→embedded with `REFERS_TO`; use `EMBEDS` only when the embedded entity is itself a file/media asset. Apply existing-entity correction (see [ENTITY TYPES & SCHEMA] `Existing-entity correction`) when an embedded entity matches a prior record by identifier (same merchant + billing period, same event start_time + title, etc.) — do not mint a duplicate. Respect Depth-of-capture scope and sensitivity clauses: cap embedded extractions per container at a reasonable total (e.g. ~20 line items) and surface the rest as a list; do not echo extracted body content back into chat beyond what answering requires.
Automated sender extraction: for every sender of an external-source record (email, notification, invoice, statement, webhook), create or match a `contact` entity for the sender address AND an `organization` or `company` entity for the underlying service — even when the sender is automated or transactional (e.g. EMEA_Invoicing@email.apple.com, confirmar-envio@amazon.es, notifications@capitalone.com, no-reply@stripe.com, noreply@github.com). Mark automated sender contacts with `sender_kind: "automated"` (or `kind: "automated"`) so they are distinguishable from relational human contacts in queries. Link the sender `contact` to its parent `organization` with REFERS_TO (contact → organization). Link the originating record (transaction, notification, receipt, statement) to both the sender contact and the parent organization with REFERS_TO. Reuse existing org/contact entities via bounded retrieval before creating new ones. FORBIDDEN: extracting only the transactional payload (invoice, shipping notification, statement) without also persisting the `organization` and sender `contact` that produced it.
User identity: when the user provides or implies their identity (name, email, "me"/"myself"), store as contact or person in the same turn when you have enough to identify.
Extract-all: extract and store all relevant entities from the user — people, tasks, events, commitments, preferences, possessions, relationships, places. Store every distinct entity. Places: store as location/property/place and link to tasks or other entities. Implied relationships: use create_relationship or relationship fields per schema. Container+asset: EMBEDS with source=container, target=asset; if the file is in Neotoma, store file, create image/media entity, then create_relationship(EMBEDS, container_id, asset_id); if the asset is elsewhere, store only a reference on the container.
Display rule — section render: when a turn creates, updates, or retrieves Neotoma entities other than chat bookkeeping (`conversation`, `conversation_message`, legacy alias `agent_message`), the user-visible reply MUST render a section headed `🧠 Neotoma — [<full conversation canonical_name or title>](<active Neotoma origin>/inspector/conversations/<conversation entity_id>)` with a horizontal rule immediately above it, and use bullet points with no per-entity tables. Use the full, untruncated conversation canonical name returned by the user-phase store (or the conversation title if canonical_name is unavailable), link that text to the Inspector conversation page, and do not show the conversation entity_id as plain parenthetical text in the heading. Use the active Neotoma origin for the current session/tooling context (for local defaults, `http://localhost:3080` for dev and `http://localhost:3180` for prod). Treat `conversation`, `conversation_message`, and the legacy `agent_message` alias as internal chat bookkeeping: exclude them from counts and from the output groups.
Display rule — groups: within the section, show only non-empty groups — `Created (N)` when there are created entities to show, `Updated (N)` when there are observation-updated entities to show, `Retrieved (N)` when listing entities read from existing Neotoma state this turn without persisting a new observation for them in this turn, and `Ambiguous (N)` when the structured-store response included batch-level `warnings[]` with `code: "HEURISTIC_MERGE"` (R3) — one bullet per warned entity. Use one bullet per entity in each group.
Display rule — ambiguous group: entities in the `Ambiguous (N)` group come from the `warnings[]` aggregate of the last **`store`** call (or the per-entity `entities[].warnings`). The bullet follows the same emoji + title + `(entity_type)` format as other groups, plus a trailing `— heuristic match via identity_rule "<identity_rule>"` suffix so the user can see why resolution was ambiguous. If the same entity would also appear under `Created`/`Updated`, it MUST appear in `Ambiguous` instead (do not double-list). Keep this disclosure short; do not repeat the warning shape or canonical_name verbatim.
Display rule — store disambiguation: non-bookkeeping entities created or observation-updated via Neotoma store in this turn — including ingesting external tool payloads into Neotoma — MUST appear under `Created` or `Updated`, never under `Retrieved`.
Display rule — bullet format: each bullet MUST start with a single emoji that connotes the entity's schema type (e.g. ✅ task, 👤 contact or person, 🏢 company, 📅 event, ✉️ email_message, 🧾 receipt or invoice, 💸 transaction, 📝 note, 📍 location or place, 📎 file_asset, 🔍 research or analysis, 🐛 issue) followed by a human-readable description of the entity using a short primary label such as title or name from stored fields. Do not repeat verbs like created, updated, or retrieved in the bullet text because the group header already states the action. End every entity bullet with the schema entity_type rendered as a markdown link to that entity's Inspector record using the active Neotoma origin, e.g. `([contact](http://localhost:3180/inspector/entities/ent_123))`; if the entity_id is unavailable, fall back to a short no-id note rather than inventing a link. Avoid opaque ids and mechanical `entity_type "label"` body text unless needed for clarity. Pick one emoji per bullet; if no obvious match exists, use 🗂️. Retrieved bullets follow the same formatting contract as Created/Updated bullets.
Display rule — empty state: before rendering the empty-state message or any `Suggestions` block, run a final capture pass. If a candidate suggestion corresponds to a concrete entity or artifact that can be safely persisted in the current turn (for example a synthesized report/note, a task implied by the completed work, or an authored markdown/json/text artifact), store it now and render it under `Created` or `Updated` instead of suggesting it. FORBIDDEN: rendering a `Suggestions` bullet for a concrete note/report/task/artifact that the agent already has enough information to persist safely in this turn. FORBIDDEN: rendering `Created`, `Updated`, `Retrieved`, or `Suggestions` groups from intent, memory, or draft text rather than actual retrieval/store results from this turn. Use `Suggestions` only for items that still require user confirmation, additional source material, or a later follow-up; when suggestions remain, render the empty-state message plus the suggestion bullets as future capture categories rather than skipped same-turn writes.
Display rule — override scope: the display rule overrides the silent-storage default and the no-emoji communication style for this disclosure only; do not narrate internal store sequencing.
Weekly value surfacing: when the conversation is the first of the day or the user has not interacted for several days, proactively run a bounded retrieval (list_recent_changes, retrieve_entities with a recent time window, or list_timeline_events for the past 7 days) and surface a brief summary: "You have N entities in Neotoma. Here is what changed this week: [2 new contacts, 3 tasks completed, 5 observations added]." Keep it to 1–2 sentences. Do not surface this more than once per day.

[ATTRIBUTION & AGENT IDENTITY]
Identify yourself: every write to Neotoma (observations, relationships, timeline events, sources, interpretations) is attributed per row. Attribution shows up in `/stats`, entity and relationship views, and audit trails. Anonymous writes are accepted but flagged as `anonymous` tier.
Preferred — AAuth: sign requests with AAuth (RFC 9421 HTTP Message Signatures plus an `aa-agent+jwt` agent token). Use `@aauth/local-keys` or equivalent. Successful verification records the public-key thumbprint, algorithm, and JWT subject/issuer, and renders the agent with a `hardware` or `software` trust badge (ES256/EdDSA → `hardware`; other algorithms → `software`). AAuth is honoured on every HTTP surface — `/mcp`, direct write routes (`/store`, `/observations/create`, `/create_relationship`, `/correct`, …), and `/session` — and the same identity is threaded into the write-path services regardless of transport (HTTP `/mcp`, MCP stdio, CLI-over-MCP, CLI-over-HTTP).
MCP over stdio: for stdio-only harnesses (Cursor, Claude Code, Codex) that need verified attribution, use the MCP identity proxy with AAuth (`neotoma mcp proxy --aauth`) or the signed dev shim `scripts/run_neotoma_mcp_signed_stdio_dev_shim.sh`. For an unsigned stdio→HTTP launcher, use `scripts/run_neotoma_mcp_unsigned_stdio_dev_shim.sh` (same port-file resolution as the signed shim; legacy alias `scripts/run_neotoma_mcp_unsigned_stdio_proxy.sh`). `neotoma mcp config` preset A emits proxy entries that sign when keys exist under `~/.neotoma/aauth/`; agents do not implement RFC 9421 signing in the harness. For Cursor against a repo-local HTTP API whose port can move (watch scripts), operators can set `NEOTOMA_MCP_USE_LOCAL_PORT_FILE=1` and `NEOTOMA_MCP_LOCAL_HTTP_PORT_PROFILE` (`dev` / `prod`) so the signed shim reads `<repo>/.dev-serve/local_http_port_<profile>` (plus legacy `local_http_port` for dev) and probes before connecting. See `docs/developer/mcp/proxy.md`.
CLI and custom HTTP clients: CLI-over-HTTP and `neotoma mcp proxy --aauth` use the same `~/.neotoma/aauth/` keypair generated by `neotoma auth keygen` when available. Use `@aauth/local-keys` or equivalent only for non-`neotoma` HTTP clients that call Neotoma endpoints directly.
Fallback — clientInfo: when AAuth is unavailable, set `clientInfo.name` and `clientInfo.version` on the MCP `initialize` handshake to a recognisable human-readable identifier (for example `cursor-agent` + build version, `claude-code` + release, `custom-ingest-pipeline@myco` + git sha). These values render as the `unverified_client` tier. Forbidden values: generic strings like `mcp`, `client`, `mcp-client`, `unknown`, `anonymous` — Neotoma normalises those to `anonymous` tier.
Optional free-form label: scripts and CI jobs that cannot set `clientInfo` meaningfully may include `agent_label` or `agent_id` on the payload; it is copied to provenance but never used for authorization.
Do not spoof: copying another agent's `clientInfo`, reusing another agent's public key/thumbprint, or inventing `agent_sub` / `agent_iss` pairs is a policy breach. Future releases will enforce per-tier ACLs; the attribution contract already treats impersonation as a breach.
Attribution surfaces: Neotoma clients expose agent identity on entities, observations, relationships, sources, timeline events, and interpretations (column/filter where supported); configuration UI summarises attribution coverage. Self-identification is a user-facing contract — it directly improves every downstream view.
Preflight your session: before enabling writes from a new client or proxy, call the MCP tool `get_session_identity` (or HTTP `GET /session`, or `neotoma auth session`) and verify that `attribution.tier` is `software`/`hardware` and `eligible_for_trusted_writes` is true. The response also includes a diagnostic `attribution.decision` block that explains why a tier resolved the way it did — see `docs/subsystems/agent_attribution_integration.md` for the full wiring + diagnostics guide.

[CONVENTIONS]
Transport precedence: when both neotoma (prod) and neotoma-dev MCP servers are available, default to neotoma for retrieval/store/instruction precedence; use neotoma-dev only when the user explicitly requests dev behavior or the task is clearly dev-only. Dev vs prod are separate SQLite databases (neotoma.db vs neotoma.prod.db) — data written to one is not visible in the other. When the active environment is ambiguous, confirm with the user or check `neotoma env` before storing.
Avoid get_authenticated_user unless the next action needs it.
Pre-check before storing: check for existing records before storing to avoid duplicates; use the existing entity_id for relationships if a match is found.
Include all fields from source when storing.
MCP resources: read-only resources under `neotoma://` (entities, entity_types, timeline, relationships, sources, individual entity/source URIs) are available through MCP list_resources and read_resource flows. Use resources for discovery and browsing; use actions/tools for mutations and complex operational queries.
Deterministic identity: entity IDs are hash-based and deterministic. The same entity_type plus schema-defined canonical identity resolves to the same entity_id across stores; use canonical identity fields rather than ad hoc names for session-scoped entities.
User scope vs CLI: MCP tools infer `user_id` from authentication; omit `user_id` on tool calls unless you are in a documented server-side override flow. Operators who use the Neotoma CLI with `NEOTOMA_API_ONLY` scope reads via `--user-id` or `NEOTOMA_USER_ID` (CLI-only; see `docs/developer/cli_reference.md`).
Combined-store remote files: when Neotoma is reachable over HTTP but not on the same host as the attachment, use `file_content` + `mime_type` (and `original_filename` when known) in **`store`**; `file_path` only works when the server can read that path (local dev or shared filesystem). Remote/tunnel/hosted APIs cannot read the agent host disk—mirrors `neotoma ingest` auto-upload for non-localhost base URLs.
Store resolution errors: on `ERR_STORE_RESOLUTION_FAILED`, inspect `issues[].hint` for payload-shape guidance (for example flattening a pre-0.5.0 `attributes` wrapper). If the error says a `conversation` resolved to an existing entity via heuristic title/name matching, repair the caller payload by reusing the bounded-retrieval entity with `target_id` (or by supplying the schema-required `conversation_id`) and retry once before surfacing the error. Surface the error envelope to the user only after the repaired retry fails. Do not treat `entities_created=0` as success when `replayed` is not true and no entities were matched unless the user explicitly waived persistence.
Structured vs unstructured path: use the entities array (structured path) for conversation-sourced data; omit `original_filename`. For tool-sourced or file-derived data, prefer the combined store path (entities + file_path or file_content) so the raw source is preserved; set `original_filename` to the source file basename when a file is involved. Use the unstructured path (file_content+mime_type or file_path) for user attachments or files to preserve; pass raw file, do not interpret.
parse_file vs store: **`parse_file`** is a read-only inspection tool — it extracts text and page images from a file but writes nothing to Neotoma (no source row, no entity, no observation). **`store`** with `file_path` or `file_content+mime_type` is the write path — it uploads the binary to the sources bucket, creates a `file_asset` entity, and returns `source_id` + `asset_entity_id`. FORBIDDEN: treating a successful `parse_file` call as sufficient file persistence. Always follow parse_file with a `store` call that includes the file when the user's file needs to be retained.
Omit user_id (inferred from authentication).
CLI parity: in CLI backup mode, `entities search` accepts a positional identifier or `--identifier`; structured `store` accepts preferred `--entities` / `--file` and the compatibility alias `--json=<json>`.
CLI backup transport: when MCP is available or when reconciling counts/results with MCP, run CLI reads/writes via API transport (`--api-only` or explicit `--base-url`) and do not rely on offline-default transport unless the user explicitly requests local/offline data.
Summarization after MCP actions: follow the [COMMUNICATION & DISPLAY] display rule exactly (horizontal rule, linked `🧠 Neotoma — [<conversation name>](<active Neotoma origin>/inspector/conversations/<conversation entity_id>)` heading, non-empty Created/Updated/Retrieved bullet groups, or the empty-state Suggestions fallback). Do not dump raw snapshot keys or internal ids into prose, and do not surface the conversation entity_id as plain header text when it is already encoded in the Inspector link. Do not repeat the same phrase as both thought and section heading.
Update check: at session start, optionally call npm_check_update with packageName "neotoma" and the client version; if updateAvailable, prompt user to upgrade. Pass include_release_notes true when the user should see truncated release notes, a breaking-changes excerpt, and release_url (best-effort; enrichment_error if enrichment fails).

[ISSUE REPORTING]
Issues are the primary feedback channel for iterative Neotoma improvement based on agentic usage. Filing happens in two scenarios: (1) **reactive** — when you hit friction using Neotoma (a failing tool call, an opaque error, a missing surface, or a doc gap); (2) **QA-driven** — when the per-turn self-audit (see [QA REFLECTION] "Auto-file threshold" and "QA-driven issue filing") produces file-worthy findings indicating a Neotoma-side bug or an instruction/interface improvement opportunity. Both scenarios are gated by **`issues.reporting_mode`**: **`proactive`** — file immediately without asking (`neotoma issues config --mode proactive` or `NEOTOMA_ISSUES_REPORTING_MODE=proactive`); **`consent`** (default) — confirm with the user before each `submit_issue`; **`off`** — only submit when the user explicitly asks. Mode discovery: if `reporting_mode` has not been configured (no prior `neotoma issues config --mode` and no `NEOTOMA_ISSUES_REPORTING_MODE` env var) and an issue-filing situation arises, first check for a stored preference entity before prompting: call `retrieve_entities` with `entity_type: "preference"` and filter for `title: "issue_filing_consent"`; if a matching entity exists, treat its `value` field as the resolved mode (`always` → `proactive`, `ask` → `consent`, `never` → `off`) and act accordingly without re-prompting the user. If no `issue_filing_consent` preference entity exists, ask the user once: "I noticed a potential issue to file. How would you like me to handle issue filing going forward? (always — file automatically, ask — confirm each time, never — skip filing)". Map the response to the reporting mode (`always` → `proactive`, `ask` → `consent`, `never` → `off`) and persist it two ways: (1) store a `preference` entity with `entity_type: "preference"`, `title: "issue_filing_consent"`, `value: "<always|ask|never>"` so future sessions can retrieve the preference without re-prompting; (2) run `neotoma issues config --mode <proactive|consent|off>` to set the runtime config flag. Do not re-ask once the preference entity exists or the mode is configured.
Proactive mode contract (auto-file mandate): when **`issues.reporting_mode`** is **`proactive`** (set explicitly via `neotoma issues config --mode proactive`, the `NEOTOMA_ISSUES_REPORTING_MODE=proactive` env var, or a future `auto_file_issues` standing-rule preference resolving to true), the agent MUST call `submit_issue` for every reactive friction event AND every file-worthy QA finding in the same turn, with no user confirmation prompt and no "want me to file this?" question. FORBIDDEN: pausing to ask "should I file this?" when proactive mode is set; deferring filing to a later turn; bundling multiple findings into a chat summary instead of separate `submit_issue` calls when their root causes differ. REQUIRED: complete the PII stripping checklist above for every auto-filed issue; run the dedup-before-filing search before each call; render every auto-filed issue in the `Issues` group of the turn report with its `entity_id` / `github_number`. Proactive mode does NOT bypass risk-management hold points (auth, schema migrations, foundation docs, destructive data repair) — for those, file the issue describing the problem but do not auto-execute the fix.
Dedup before filing: before calling `submit_issue`, search open GitHub issues to avoid creating duplicates. Use `sync_issues` (with `state: "open"`) to pull current issues into local Neotoma, then `retrieve_entities` with `entity_type: "issue"` and a relevant search term to find matches. If a matching or closely related open issue exists: (1) use `add_issue_message` on the existing issue instead of creating a new one — include the new finding, reproduction context, and any additional detail; (2) if the new finding is related but distinct enough to warrant its own issue, proceed with `submit_issue` but reference the existing issue by GitHub number (e.g. "Related: #42") in the body so both are cross-linked. FORBIDDEN: filing a new issue that duplicates an open one when a search would have caught it. When `sync_issues` is unavailable or stale, a `gh issue list --search "..." --state open` shell command is an acceptable fallback for the search step.
PII and secrets: for **`visibility: "public"`** (default when mirrored to GitHub), redact emails, phone numbers, API tokens, UUIDs, and home-directory path fragments with `<LABEL:hash>` placeholders before `submit_issue` so public GitHub text is safe. For **`visibility: "private"`** (`submit_issue` stores Neotoma-only; no GitHub create), still redact the same classes when the user does not want **operators** on the configured `issues.target_url` instance to see them in the issue body or thread — private means no GitHub mirror, not unlimited disclosure to maintainers; omit or generalise fields the user marks sensitive unless they explicitly want them in the report.
Include relevant context in the issue body: Neotoma version, client name, OS, tool name, error class, error message, and invocation shape when applicable. `reporter_app_version` is auto-populated by the server from the running package version when not supplied — you do not need to call `npm_check_update` first to obtain the version, though you may pass it explicitly if you have a more specific value (git SHA, build tag).
Issue–conversation linking: immediately after `submit_issue` returns (or after any `get_issue_status`, `add_issue_message`, or `sync_issues` call that touches an `issue` entity in the current turn), create a REFERS_TO relationship from the active conversation entity to the issue entity (`create_relationship(REFERS_TO, source_entity_id=<conversation_entity_id>, target_entity_id=<issue_entity_id>)`). Also create REFERS_TO from the user or assistant `conversation_message` of the current turn to the issue entity so the exact turn of creation or interaction is traceable. FORBIDDEN: ending a turn in which an issue was created, updated, or retrieved without these two REFERS_TO edges in place. This applies equally to issues retrieved from Neotoma and interacted with (e.g. `add_issue_message`) — not only newly filed ones.
Remote submission failure: when `submit_issue` returns `remote_submission_error` containing "AUTH_REQUIRED", no Bearer token or agent grant is configured for this agent identity on the operator instance. Surface the full error message to the user with the actionable hint (create an agent grant via Inspector → Agents → Grants, or configure a Bearer token). Do not silently swallow the error; `sync_pending=true` on the local entity means it can be retried later once authentication is resolved.
Immediately after `submit_issue` returns, the local `issue` row already exists (with `github_number` / `github_url` when a GitHub mirror was created). When the configured operator instance accepts the issue, the response may include **`guest_access_token`**; treat it as a credential and use it for token-scoped remote read-back / append when the local issue snapshot does not already carry that token. Track status with **`get_issue_status`**: required **`entity_id`** is the `issue` `entity_id` from the `submit_issue` response. Optional **`guest_access_token`** — pass when read-through to a remote operator row needs a token and the issue snapshot does not already carry one; optional **`skip_sync`** — when true, skips mirror refresh for that call (GitHub sync when mirrored; remote read-through when `issues.target_url` applies). Snapshot fields such as **`remote_entity_id`**, **`remote_conversation_id`**, and stored **`guest_access_token`** are used as defaults when you omit overrides. Append thread messages with **`add_issue_message`**: required **`entity_id`** and **`body`**; same optional **`guest_access_token`** semantics as `get_issue_status` for operator read-through / remote append when mirroring a remote issue.
When `pushed_to_github` is false for a public issue, the `submit_issue` response includes `github_mirror_guidance` with recommended next steps (authenticate GitHub, create the mirror issue, update `github_number`/`github_url` on the stored entity).
When upgrade guidance appears as a comment on the GitHub issue (posted by the upgrade guidance bot on release), treat it as the actionable payload: run or propose install commands, follow verification steps, then re-attempt the original invocation and post a verification message on the issue.
Issue-driven work updates: when a turn uses an existing issue as the work driver (for example the user asks to investigate, fix, implement, verify, or document based on an `issue` entity id, GitHub issue number, or issue URL), the agent MUST update that issue thread before the final reply. If the GitHub issue body, title, comments, user prompt, or retrieved issue data names a Neotoma `issue` entity id, first attempt **`add_issue_message`** against that Neotoma entity (passing `guest_access_token` when required and available). If no usable Neotoma issue entity exists, lookup fails, or Neotoma issue tooling is unavailable, update the GitHub issue directly with the matching comment/update path instead. The issue update MUST include the outcome, key files or surfaces changed, verification commands/results, and unresolved caveats or blockers. Do not mark or close the issue automatically unless the user explicitly requests closure or repository instructions explicitly allow verified fixes to be closed. FORBIDDEN: completing issue-driven work with only a chat summary while leaving the driving issue unmodified.
GitHub issue URL extraction: when a user message, retrieved issue body, or external source contains a GitHub issue URL, do not create or update a generic `issue` entity using ad hoc fields such as `github_issue_number`, `repository`, or `url`. Normalize the URL before storing: use `entity_type: "issue"`, `github_number`, `github_url`, and `repo` (`owner/name`) so the schema's `github_number + repo` identity rule can match the existing row. If an `issue` entity for that GitHub number already exists, use that entity via retrieval or `target_id`; do not create a title-keyed placeholder such as "Debug GitHub issue 42". If only partial GitHub issue context is available and the canonical fields cannot be populated, store the reference as a `note` or `technical_research` entity instead of an `issue` until the canonical fields are known.

[QA REFLECTION]
Scope: when operating inside a Neotoma source checkout (repo root or configured NEOTOMA_PROJECT_ROOT points at the Neotoma repo), run the full QA reflection below and resolve turn-based Neotoma issues directly in the repo when safe. When operating outside the Neotoma repo, including packaged/npm consumer contexts, do not edit the surrounding project to fix Neotoma behavior; use `submit_issue` to report product/tooling/doc issues via GitHub and apply only safe local data repairs for the current turn.
Session-start health check (source checkout only): on the first turn of a session, call list_entity_types and health_check_snapshots and cache the result; check for near-duplicate entity types, production test-artifact schemas (`test_type_*`, `test_activate_*`, `cross_layer_schema_*`), stale snapshots, and a small orphan-node sample from recent entities. Respect consistency tiers: core reads/writes are strongly consistent; search may take ~5s and embeddings ~10s, so do not report bounded-eventual delay as a bug. Surface only findings that materially affect the current turn.
Per-turn self-audit: before finalizing the reply, classify the turn against four tiers: Tier 1 interaction efficiency (one user-phase store, 0–2 bounded retrievals, one closing store, 0–1 separate relationship call unless an out-of-store EMBEDS is needed); Tier 2 data quality (entity-type reuse, scoped turn_key, idempotency key, unknown_fields_count [see below], correction protocol, ErrorEnvelope handling, source fields); Tier 3 interpretation fidelity for source material (complete entity extraction, accurate amounts/dates/names/statuses, raw source preservation, schema consistency, dedup awareness); Tier 4 database health from the session-start check. Tier 2 — mandatory unknown_fields repair: if any entity in a store response has `unknown_fields_count > 0`, this is a mandatory inline repair — immediately re-store or `correct` those entities using declared schema field names; do not proceed to the closing assistant store until all entities in the turn report `unknown_fields_count: 0`. Do not treat `unknown_fields_count > 0` as informational; it means data was silently dropped to `raw_fragments` and is not in the entity snapshot.
Severity classification: minor gaps are auto-fixed when safe and otherwise noted briefly; significant gaps (missing conversation-turn persistence, missing required relationship, orphaned entity, raw source not preserved, skipped source entities, wrong key field value, duplicate entity-type risk, ignored ErrorEnvelope, or recurring product weakness) must be repaired in-turn when safe, or surfaced as an issue with immediate meaning, risk if unresolved, and recommended resolution.
Auto-file threshold: a QA finding is **file-worthy** when it plausibly indicates (a) a Neotoma-side bug (server, reducer, resolver, schema projection, transport), or (b) a gap in Neotoma's instructions or interface that, if improved, would help agents avoid the same class of usage mistake in future sessions. File-worthy findings include: `unknown_fields` that persist after schema enrichment (schema-projection drift), `ERR_STORE_RESOLUTION_FAILED` caused by ambiguous identity rules, heuristic merges that collapse distinct entities, missing or misleading instruction text that directly caused a wrong agent action this turn, and recurring product weaknesses observed across multiple turns or sessions. NOT file-worthy (to avoid noisy tickets): one-off agent mistakes correctable via `correct()` in-session with no underlying product cause, transient network/retry failures that self-healed, and minor cosmetic or formatting gaps in the turn footer. When in doubt, err toward filing — the issue system is the primary feedback channel for iterative Neotoma improvement based on agentic usage.
QA-driven issue filing: when the per-turn self-audit produces one or more file-worthy findings, the agent MUST attempt to file each via `submit_issue` in the same turn, gated by `issues.reporting_mode`: (1) **`proactive`** — file immediately without asking; (2) **`consent`** (default) — present each file-worthy finding to the user with a one-line summary and ask "File this as a Neotoma issue? (yes/no, public/private)"; batch multiple findings into a single prompt when possible; (3) **`off`** — do not file; render the finding in the `Issues` group only. If `reporting_mode` has not been configured and a file-worthy finding is detected, follow the mode discovery flow in [ISSUE REPORTING] — check for a stored `issue_filing_consent` preference entity first, then ask the user once if none exists, and persist both the preference entity and the runtime config flag before proceeding. Before filing, run the dedup-before-filing search from [ISSUE REPORTING]; if an existing open issue covers the same root cause, use `add_issue_message` on that issue instead of creating a new one. Inside a Neotoma source checkout, file the issue AND attempt a local repo fix (instruction edit, schema evolution, etc.) in the same turn when safe; the issue still gets filed so it is tracked cross-session and across non-checkout consumers. Combine multiple related findings into a single issue when they share a root cause; file separate issues when root causes are distinct.
Issues and repairs: feed remaining gaps, applied fixes, and filed/skipped issues into the `🧠 Neotoma` turn report. Do not claim a clean turn while a required persistence, relationship, provenance, or error-handling invariant failed. If a repair is itself durable knowledge, store a `neotoma_repair` or suitable note entity and link it to the prompting message.
Compliance-pass diagnosis and auto-fix (source checkout only): when a compliance pass is triggered (by the stop-hook follow-up or by manual user request), the agent MUST (1) diagnose WHY the prior turn missed stores — classify as one of: tooling_unavailable_or_failed, instruction_delivery_missing_or_stale, agent_ignored_available_instructions, hook_state_incomplete, or false_positive_or_no_material_content; cite the signals that led to the classification (reminder injected? tool failures? connection failure? model?); (2) after completing the backfill store, attempt to resolve the underlying repo-level root cause in the same turn — e.g. update client instructions (`docs/developer/mcp/instructions.md`, `docs/developer/cli_agent_instructions.md`, `.cursor/rules/neotoma_cli.mdc`), fix hook delivery paths, clarify ambiguous recipes, or add a missing reminder gate — guided by the stop-hook's `recommended_repairs` when present; (3) render the diagnosis and any applied fix in the `🧠 Neotoma` section under an `Issues` or `Repairs` group. FORBIDDEN: completing a compliance pass with only a backfill store and no diagnosis; completing a compliance pass inside the Neotoma repo without attempting the root-cause fix when `proactive_remediation_required` is true. Outside the Neotoma repo, report the diagnosis via `submit_issue` instead of editing consumer-project files.
Durable enhancement ladder: for recurring product-level weaknesses encountered while operating inside the Neotoma repo, update agent instructions first, re-test the nearest scenario, then consider the smallest schema-agnostic Neotoma-side helper/validation/repair enhancement. Ask before schema-specific behavior, schema evolution, or domain-specific modeling changes. Outside the Neotoma repo, report the issue with `submit_issue`, preserve enough redacted reproduction detail, and follow upgrade guidance when available instead of editing consumer-project files.
Issue reporting: every file-worthy `Issues` item MUST have a same-turn `submit_issue` call (gated by `reporting_mode` per QA-driven issue filing above). Inside a Neotoma source checkout, a local repo fix is additive — the issue is still filed so it is tracked. If an issue is not filed because the user declined (consent mode), mode is `off`, or the issue already exists, render that decision in the `Issues (N)` group so the problem is not orphaned. Whenever `submit_issue` or `get_issue_status` is called, or a check is deliberately deferred, render one traffic-light bullet per issue: green for resolved/closed, yellow for open/in-progress/deferred/skipped/local-fix, red for blocked/transport error. Show at most a short `entity_id` or `github_number` (when present) and the issue URL.

[ERRORS & RECOVERY]
Store retry policy: if **`store`** fails, (1) retry once with the same payload; (2) if it fails again, surface the error to the user ("Storage failed: [error message]") before responding with any retrieved data; (3) do not silently skip storage and respond as if it succeeded.
SQLite corruption: if any local Neotoma operation fails with `database disk image is malformed`, `SQLITE_CORRUPT`, `btreeInitPage`, or failed SQLite integrity checks, tell the user the local SQLite file is likely corrupted and suggest `neotoma storage recover-db` first, then `neotoma storage recover-db --recover` after they stop Neotoma. Do not auto-swap the recovered DB without explicit user approval.
getStats unreachable: if getStats is unreachable when answering entity-type cardinality questions, state that explicitly rather than substituting an expensive per-type count or a schema-width value. See [RETRIEVAL] entity-type cardinality rule.

[ONBOARDING]
Discovery flow: when Neotoma has little or no data (first run or empty state), follow the onboarding sequence from install.md — (1) ask the user which data types matter most (project files, chat transcripts, meeting notes, notes/journals, code context, email, financial docs, custom paths) and which mode they prefer (quick win, guided, power user); (2) discover high-value local files by scanning shallowly — rank by entity density, temporal signals, recency, and relationship potential per docs/foundation/file_ranking_heuristic.md; (3) group results into domains (not file counts) and explain why each was selected and what timeline value it could unlock; (4) let the user confirm per-folder or per-file with an expected reconstruction preview; (5) ingest confirmed files and reconstruct the strongest timeline with provenance (each event traced to a specific source file); (6) show the timeline immediately — not a file count; (7) offer one follow-up query specific to the reconstructed entity and 2–4 leveraged next actions; (8) demonstrate correction. See docs/developer/agent_onboarding_confirmation.md for the full specification.
Output rule (Installation Aha): after first-run ingestion, the first visible output MUST be a reconstructed timeline with provenance, not a file count or entity count. Format: "[Entity name] — Timeline reconstructed from [N] sources" followed by dated events each with "Source: [filename], [location]". This is the Installation Aha that creates the referral moment.
Chat transcript discovery: during onboarding, check for chat transcript exports (ChatGPT JSON, Slack exports, Claude history, meeting transcripts). These are the highest-signal ingestion source. When found, explain their value: "Chat transcripts encode decisions, commitments, and project discussions with timestamps. They are one of the best sources for timeline reconstruction." See docs/developer/transcript_ingestion.md.
```

## One-call chat persistence

Optional `relationships` on **`store`** is an array of relationship entries. Use `{ relationship_type, source_index, target_index }` for indices into the request's `entities` array, or `{ relationship_type, source_entity_id, target_entity_id }` for existing entity IDs. The server creates/entities resolves first, then creates relationships in one request. There is no separate `store_chat_turn` action; the generic store covers chat.

## Design rationale

The instruction block is tuned so agents can complete a turn (retrieval → user-phase store → attachment EMBEDS → assistant reply → closing store) without opening tool schemas or exploring the MCP tool set:

1. **Labelled sections.** Bracket-prefixed labels ([TURN LIFECYCLE], [DATA MODEL], [GUEST ENTITY SUBMISSION] (includes PII stripping checklist before issue filing), [CROSS-INSTANCE SYNC — PEERS], [SUBSTRATE SUBSCRIPTIONS], [STORE RECIPES], [RETRIEVAL], [PROVENANCE], [TASKS & COMMITMENTS], [ENTITY TYPES & SCHEMA], [ENTITY & RELATIONSHIP LIFECYCLE], [COMMUNICATION & DISPLAY], [ATTRIBUTION & AGENT IDENTITY], [CONVENTIONS], [ISSUE REPORTING], [QA REFLECTION], [ERRORS & RECOVERY], [ONBOARDING]) let agents locate rules by topic and cross-reference from one section to another without restating them.
2. **No exploration.** One prominent line under [STORE RECIPES] forbids listing, globbing, or reading MCP tool descriptor/schema files for chat, attachment, and entity-extraction flows. All parameter names and response paths used by the recipes (`structured.entities[].entity_id`, `unstructured.asset_entity_id`) are inline, so agents never need to open schemas.
3. **Unified store shape.** The user-phase recipe covers chat, extraction, and attachments as a single entities list [conversation, message, …extracted entities] with one invariant (PART_OF from message to conversation) plus REFERS_TO per extracted entity. Each list entry is a **flat** object (fields beside `entity_type`); the legacy **`attributes`** wrapper is forbidden (see first bullets under [STORE RECIPES]). Attachment turns use the same shape plus file_path/file_content and a single follow-up EMBEDS call.
4. **Turn-ordered rules.** [TURN LIFECYCLE] encodes the five-step ordering (retrieval, user-phase store, other actions, reply, closing store) once; other sections reference those step numbers instead of re-describing the ordering.
5. **Atomic display rule.** The user-visible entity display rule is split into six sub-rules (section render, groups, store disambiguation, bullet format, empty state, override scope), each on its own line, reducing drift on the most-violated rule.
6. **Explicit fallback IDs.** One line gives a concrete fallback when the host provides no conversation_id/turn_id (`conversation-chat-<turn>-<timestamp_ms>`, `turn_key "chat:<turn>"`), so agents do not re-derive the pattern.
7. **Explicit error policy.** [ERRORS & RECOVERY] defines the store retry policy, SQLite corruption handling, and getStats-unreachable behavior so agents do not silently skip storage on failure.

Keeping the recipes in sync with server response shapes (`structured.entities[].entity_id`, `unstructured.asset_entity_id`) reduces overhead. Attachment turns use parse → extract → store → EMBEDS: one store request for structured and unstructured data, then one EMBEDS relationship call.

## Related documents

- `docs/specs/MCP_SPEC.md` — Action catalog and entity type rules
- `docs/developer/mcp_overview.md` — Overview and setup
- `docs/developer/mcp/unauthenticated.md` — Unauthenticated instructions
- `docs/developer/mcp/tool_descriptions.yaml` — Per-tool descriptions
- `docs/developer/cli_agent_instructions.md` — Thin harness layer (MCP vs CLI transport + CLI cheat sheet; no duplicate of this fenced block)
- `docs/developer/agent_instructions.md` — Index for canonical vs harness files
- `docs/developer/agent_instructions_sync_rules.mdc` — Maintainer contract (canonical-first)
- `src/server.ts` — Loads this file via `getMcpInteractionInstructions()`
