# Engagement events aggregation

Substack engagement export carries per-subscriber-per-post events (opens, clicks). This skill aggregates them to per-subscriber totals folded onto the `:MENTIONS` edge between the roster KD and each `:Person` — per-event detail is intentionally dropped.

## Detection

Per [archive-shape.md](archive-shape.md), the skill probes for the events file in this order:

1. `email_activity.csv`
2. `subscriber_activity.csv`
3. `emails.csv`

The first match binds `$engagementFile`. None match: emit `[substack-import:warn] no-engagement-file` once and proceed with an empty aggregates map.

## Column tolerance

Substack has used at least three column-naming schemes across export versions. Probe for each value in the order listed; first match wins:

| Aggregate field | Candidate column names |
|---|---|
| subscriber email | `email`, `subscriber_email`, `user_email` |
| event type | `event`, `event_type`, `action` |
| event timestamp | `event_at`, `timestamp`, `time` |
| post identifier (optional, for cross-check) | `post_id`, `email_id` |

If the email or event-type column cannot be resolved, emit `[substack-import:error] file=<engagementFile> reason="cannot resolve email or event-type column"` and proceed with empty aggregates — do not abort the whole import.

## Aggregation

Walk the file once, accumulating per email:

```
emailToAggregates: Map<email-lowercased, {
  totalOpens: int,           // count of rows where event matches /^open$/i
  totalClicks: int,          // count of rows where event matches /^click$/i
  lastOpenedAt: ISO|null,    // max timestamp where event matched open
  lastClickedAt: ISO|null,   // max timestamp where event matched click
  engagementWindowDays: int  // (now - earliest event timestamp for this email) in days, ceil
}>
```

Event types other than open / click are tallied separately and reported in the completion log as `otherEvents=<n>` if non-zero, but are not folded onto the edge (no `:MENTIONS.totalForwards` etc. — would inflate the property set without a concrete query motivating it).

## Cardinality bound

A worst-case substack archive: 2000 subscribers × 500 posts × 2 event types = 2,000,000 events. Reading this twice (once to count, once to aggregate) wastes I/O; the aggregation pass runs streaming with a single map allocation. Map size is bounded by distinct emails (≤ subscriber count), not by event count — so the in-memory cost is O(subscribers), not O(events).

If the events file exceeds 500 MB, emit `[substack-import:warn] engagement-file-large size=<bytes>` so the operator knows the parse pass will take a few minutes; proceed anyway.

## Re-import semantics

Each Substack export carries totals-to-date, not deltas. On re-import:

- The aggregation walk recomputes the per-email totals from the new file.
- The per-subscriber MERGE's unconditional `SET` block overwrites engagement properties on the edge.
- Subscriber 1 with 3 opens in import A and 5 opens in import B: edge ends with `totalOpens=5` (the latest export's total), not 8.
- A subscriber who appears in import A's events file but not import B's: their edge's engagement properties are set to `null` in import B (Neo4j removes them), so the edge stops showing engagement. This is correct — if Substack's export omits a subscriber, they have no recorded engagement and the edge should reflect that.

## Why not per-event KDs

Per-event detail (which specific day of which specific open) is high-volume and low-query-value at v1. The aggregates answer the "warm vs cold" question that the operator actually asks. If a downstream feature needs per-event detail (e.g., "show me a timeline of when subscriber X engaged with my posts"), it lands as a separate task — likely as `:KnowledgeDocument {kind:'substack-email-event'}` children of the per-post KD with `:MENTIONS` to the `:Person`. That schema choice is deferred until a concrete query motivates it.
