---
name: x-import
description: Ingest an extracted X (Twitter) archive directory into the graph as one :KnowledgeDocument (tweet stream) plus one :ConversationArchive per DM participant set. Triggers when the operator drops an `Archive of <handle>` directory or `twitter-<handle>-archive.zip` path into chat. Owner + every distinct participant confirms against existing `:AdminUser` / `:Person` nodes before any write. Idempotent on archive content hash.
---

# X Import

**Invoked from `specialists:archive-ingest-operator`** — the admin agent dispatches the archive-ingest-operator when the operator hands over an X archive, and the archive-ingest-operator loads this skill. The admin agent never calls this skill directly.

One X archive becomes:

- **One `:KnowledgeDocument`** (`source='x'`) carrying the verbatim chronological transcript of the operator's authored content — root tweets, replies, quote-tweet authorship, deletion markers. `memory-classify` (document mode) splits the body into topic-bounded `:Section` children. Threading is a within-body concept rendered in the transcript; per-thread KDs are explicitly rejected.
- **N `:ConversationArchive` nodes** (`source='x-dm'`) — one per distinct DM participant set, MERGEd on the existing `conversationIdentity = sha256(accountId + ":" + sortedParticipantElementIds)` key. DM media attaches to the archive node via `HAS_ENCLOSURE`.
- **Person resolution** — every `@handle` mentioned, replied-to, or quoted in the tweet stream MERGEs against `:Person` on lowercased `xHandle` scoped by `accountId`. Existing `:Person` nodes gain `xHandle` on `ON CREATE SET` without overwriting operator edits.

## Archive shape

See [archive-shape.md](../../references/archive-shape.md) for the `.js` wrapper grammar and which files in the archive carry which data. The grammar is verified at parse — unwrapped JSON or a missing `window.YTD.*.partN = ` prefix errors loudly, never silently zero-tweets.

## Transcript rendering

See [transcript-shape.md](../../references/transcript-shape.md) for the exact rendering of the tweet-stream KD body — chronological separators, quote-tweet indentation, deletion markers, media references.

## Pipeline (deterministic, four phases)

### Phase 0 — Owner + participant confirmation

Before any classify or ingest call:

1. **Identify the owner.** Read `data/account.js` to extract the archive owner's `accountId`, `username` (the handle), and `displayName`. Surface to the operator: `"X archive owner: @<handle> (<displayName>). Confirm this maps to :AdminUser <name> ({userId})."`. Refuse to proceed without confirmation.

2. **Enumerate distinct participants from the tweet stream.** Read `data/tweets.js` and `data/tweet-headers.js`. Extract every `@handle` that appears as a mention, reply target (`in_reply_to_screen_name`), or quote-tweet author. Emit a single-line histogram per distinct handle: `Handle '@<handle>' (<mentions> mentions, <replies> replies, <quotes> quotes)`.

3. **Enumerate distinct DM participants.** Read `data/direct-messages.js` and `data/direct-messages-group.js` (when present). For each `dmConversation`, extract the distinct senderIds. Senders map to existing `:Person` / `:AdminUser` by either prior `xUserId` property or operator confirmation against numeric ID + recent-message preview.

4. **One operator question per distinct handle / senderId.** Mirror the conversation-archive flow: `"Handle '@<h>' (<n> mentions) — pick existing :Person/:AdminUser or block?"`. The operator either picks an elementId or names "block" (refuses to map). **Never auto-create a `:Person`** — the operator must confirm a canonical node.

5. **Persist resolved IDs.** Build two maps: `handleToElementId` (handle → :Person elementId, for the tweet-stream KD edges) and `senderIdToElementId` (numeric senderId → :Person/:AdminUser elementId, for DM participant lists). Both maps are passed to the writers in subsequent phases.

### Phase 1 — Tweet-stream transcript + media staging

1. **Walk the tweet stream chronologically.** Merge `data/tweets.js` (operator-authored tweets) and `data/tweet-headers.js` (metadata). For each tweet: extract `id_str`, `created_at`, `full_text`, `in_reply_to_status_id_str`, `in_reply_to_screen_name`, `entities.user_mentions`, `entities.urls`, `entities.media`, `quoted_status_id_str`. Sort by `created_at` ascending.

2. **Render the transcript.** Apply the rules in [transcript-shape.md](../../references/transcript-shape.md). The output is a single UTF-8 text file written to `{accountDir}/archive/x/<importId>/tweet-stream.md`, where `importId = sha256(archive bytes)` truncated to 16 hex chars.

3. **Stage media enclosures.** For each `entities.media` entry: copy from `data/tweet_media/<filename>` to `{accountDir}/archive/x/<importId>/media/<sha256>-<filename>` (perms 0o600). Same for DM media (`data/direct_messages_media/`) — staged into the same `media/` dir with the same hash-prefixed naming so re-import is content-addressed.

4. **Render deletion markers.** When `tweet-headers.js` references an id that has no body in `tweets.js`, render the slot in the transcript as `[deleted on <iso>]` so context boundaries survive classification.

### Phase 2 — Tweet-stream :KnowledgeDocument ingest

Dispatch the existing librarian / document-ingest pipeline against the rendered transcript file. The attachment metadata block is constructed deterministically:

- `filename = "tweet-stream-@<handle>.md"`
- `attachmentId = sha256(transcript-bytes)` — guarantees idempotency on identical content
- `storagePath = {accountDir}/archive/x/<importId>/tweet-stream.md`
- `mimeType = "text/markdown"`

The dispatch brief names:
- **Anchor:** `:AdminUser` (the confirmed owner from Phase 0). Adam Mackay / Joel Smalley etc. resolved in Phase 0 appear in the **key entities to connect** list — `wire-brief-entities` writes the `MENTIONED` / `REPLIED_TO` / `QUOTED` edges from the new KD to each `:Person`.
- **`sourceUrl`:** `https://x.com/<handle>`
- **`sourceType`:** `'x'` (carried into `:KnowledgeDocument.source`)
- **`scope`:** `'admin'` (the tweet stream is the operator's authored content)

After document-ingest returns, write the inverse `REPLIED_TO` edges from each inbound-reply author (`:Person`) to the tweet-stream KD — one Cypher MERGE per distinct inbound-replier resolved in Phase 0. Edges carry `accountId`, `sourceDocumentId = attachmentId`, and `createdByAgent = 'x-import'` so re-ingest cleans them up.

### Phase 3 — DM `:ConversationArchive` ingests (one per dmConversation)

For each `dmConversation` in `data/direct-messages.js` (and `data/direct-messages-group.js` when present):

1. **Split.** Write a per-conversation slice file at `{accountDir}/archive/x/<importId>/dm-<sessionId>.json` — a wrapper-stripped JSON array of length 1 whose single element is the dmConversation envelope. This is the file the x-dm normaliser reads.

2. **Resolve participants.** Take the `senderIdToElementId` map from Phase 0; the participant set for this conversation is the distinct senderIds in the slice. The owner is the operator's confirmed `:AdminUser` elementId.

3. **Invoke the bash.** Run in the background (heartbeat protocol per `conversation-archive` SKILL.md):

   ```bash
   bash platform/plugins/memory/bin/conversation-archive-ingest.sh \
     {accountDir}/archive/x/<importId>/dm-<sessionId>.json \
     --source x-dm \
     --participant-person-ids <ownerEid>,<otherEid>,... \
     --scope admin \
     --session-id x-import-<importId>-dm-<sessionId>
   ```

4. **Surface the three operator messages** (parse → classify → write) from each invocation per the existing conversation-archive contract.

### Phase 4 — Media enclosures

After both the KD and all `:ConversationArchive` writes complete:

- For each tweet-media file, write `(:KnowledgeDocument {tweetStreamKd})-[:HAS_ENCLOSURE]->(:ImageObject)` (image media) or write `(:KnowledgeDocument)-[:HAS_ENCLOSURE { contentUrl }]->(...)` skipping body ingest for binary-only video.
- Document-shaped media (OCR-able images, PDFs from quote-tweeted articles when archived) recurse through `document-ingest` to land a child `:KnowledgeDocument`, attached via `HAS_ENCLOSURE` from the parent tweet-stream KD.
- DM media: same `HAS_ENCLOSURE` shape but attaches from the relevant `:ConversationArchive`.

## Diagnostic line

After the full ingest completes, emit a single grep-able line:

```
[x-import] importId=<id16> tweets=<n> kdCreated=<0|1> kdUpdated=<0|1> people=<p> enclosures=<e> dmConversations=<d> ms=<ms>
```

Surfaces to the operator as one chat line. `kdCreated + kdUpdated == 1` per tweet-stream ingest (one or the other, never both, never zero on a valid archive).

## Idempotency

- Re-running the same archive bytes is a no-op for the tweet-stream KD — `attachmentId = sha256(transcript-bytes)` matches the prior write; `document-ingest` skips on identical `attachmentId`.
- A newer archive containing additional tweets produces a different transcript and therefore a different `attachmentId` — the prior KD is replaced (sections regenerated under the same `sourceDocumentId`).
- DM `:ConversationArchive` re-ingest follows the conversation-archive cursor model: same `conversationIdentity` → delta-append. No `--rebuild` is ever issued autonomously.
- Person `xHandle` writes are `ON CREATE SET` — re-import never overwrites operator edits.

## Wrapper-grammar defence

Every `.js` shim read by this skill is gated through the wrapper regex documented in [archive-shape.md](../../references/archive-shape.md). Unwrapped-JSON input (missing the `window.YTD.<feed>.part0 = ` prefix on `tweets.js`, `direct-messages.js`, or `tweet-headers.js`) errors at parse with `phase=parse reason="missing window.YTD wrapper on <filename>"`. Silent zero-tweet imports are the failure mode this defence exists to prevent.

## What this skill does not do

- **Per-thread KDs.** One archive = one tweet-stream KD. Thread boundaries are section boundaries the classifier owns, not graph objects.
- **Bespoke labels.** No `:Post`, `:Tweet`, `:DirectMessage`, `:DMThread`, `:Attachment`. Everything maps to the existing `:KnowledgeDocument` + `:ConversationArchive` + `:ImageObject` shapes.
- **Tweet-level typed edges.** `:REPLIES_TO` / `:QUOTES` as graph edges off individual tweets are rejected; document-level `REPLIED_TO` / `QUOTED` from the KD to each `:Person` covers the same traversals at the right granularity. The Task 305 typed-edge pass enumerates the new KD and writes any further `COMMITTED_TO` / `MENTIONS` / `REFERENCES` edges from in-body references.
- **Retweets-without-comment.** Not persisted reliably in X archives; dropped at parse.
- **Live X API.** Archive-only.
