# PostHog AI Tracing Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add optional `$ai_span` event emission to the PostHog exporter for tool call events, integrating MCPCat with PostHog's AI observability pipeline.

**Architecture:** A new `enableAITracing` flag on `PostHogExporterConfig` controls whether tool call events additionally emit a `$ai_span` event in the same batch. A `toUUID()` helper generates deterministic UUIDs from MCPCat KSUIDs. A `buildAISpanEvent()` method maps MCPCat event fields to PostHog's `$ai_*` property schema. Customer tags/properties are spread directly (not namespaced) and can override MCPCat defaults.

**Tech Stack:** TypeScript, Vitest, Node.js `crypto` module

**Spec:** `docs/superpowers/specs/2026-03-20-posthog-ai-tracing-design.md`

---

### Task 1: Add `enableAITracing` config and `toUUID()` helper

**Files:**

- Modify: `src/modules/exporters/posthog.ts:1-9` (config interface), add import for `createHash`, add `toUUID()` method
- Modify: `src/tests/posthog-exporter.test.ts` (add UUID determinism test)

- [ ] **Step 1: Write the failing test for deterministic UUID generation**

Add this test at the end of the `describe("PostHogExporter", ...)` block in `src/tests/posthog-exporter.test.ts`:

```typescript
it("should generate deterministic UUIDs for $ai_span trace and span IDs", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  // Export event A (evt_aaa, ses_xxx)
  await exporter.export(makeEvent({ id: "evt_aaa", sessionId: "ses_xxx" }));
  const bodyA = JSON.parse(fetchSpy.mock.calls[0][1].body);
  const spanA = bodyA.batch.find((e: any) => e.event === "$ai_span");

  // Export event B (evt_bbb, ses_xxx) — same session, different event
  fetchSpy.mockClear();
  await exporter.export(makeEvent({ id: "evt_bbb", sessionId: "ses_xxx" }));
  const bodyB = JSON.parse(fetchSpy.mock.calls[0][1].body);
  const spanB = bodyB.batch.find((e: any) => e.event === "$ai_span");

  // Export event A again — verify determinism
  fetchSpy.mockClear();
  await exporter.export(makeEvent({ id: "evt_aaa", sessionId: "ses_xxx" }));
  const bodyC = JSON.parse(fetchSpy.mock.calls[0][1].body);
  const spanC = bodyC.batch.find((e: any) => e.event === "$ai_span");

  // Same sessionId → same $ai_trace_id (trace_id derived from sessionId)
  expect(spanA.properties.$ai_trace_id).toBe(spanB.properties.$ai_trace_id);

  // Different eventId → different $ai_span_id (span_id derived from eventId)
  expect(spanA.properties.$ai_span_id).not.toBe(spanB.properties.$ai_span_id);

  // Same eventId → same $ai_span_id (deterministic)
  expect(spanA.properties.$ai_span_id).toBe(spanC.properties.$ai_span_id);

  // Verify trace_id != span_id (guards against swapped inputs)
  expect(spanA.properties.$ai_trace_id).not.toBe(spanA.properties.$ai_span_id);

  // Valid UUID format (8-4-4-4-12 with version=5 and variant=8-b)
  const uuidRegex =
    /^[0-9a-f]{8}-[0-9a-f]{4}-5[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
  expect(spanA.properties.$ai_trace_id).toMatch(uuidRegex);
  expect(spanA.properties.$ai_span_id).toMatch(uuidRegex);
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: FAIL — `enableAITracing` not recognized in config type, no `$ai_span` event in batch.

- [ ] **Step 3: Implement config change and `toUUID()` helper**

In `src/modules/exporters/posthog.ts`:

1. Add import at line 1:

```typescript
import { createHash } from "crypto";
```

2. Add `enableAITracing` to the config interface:

```typescript
export interface PostHogExporterConfig {
  type: "posthog";
  apiKey: string;
  host?: string;
  enableAITracing?: boolean; // Default: false. Emits $ai_span events for tool calls.
}
```

3. Store config in the class (it's currently only storing `batchUrl` and `apiKey`). Add a private field:

```typescript
private config: PostHogExporterConfig;
```

And in the constructor, add:

```typescript
this.config = config;
```

4. Add the `toUUID()` private method to the class:

```typescript
private toUUID(id: string): string {
  const hash = createHash("sha256").update(id).digest("hex");
  return [
    hash.substring(0, 8),
    hash.substring(8, 12),
    "5" + hash.substring(13, 16),
    ((parseInt(hash[16], 16) & 0x3) | 0x8).toString(16) + hash.substring(17, 20),
    hash.substring(20, 32),
  ].join("-");
}
```

**Note:** Do NOT add `buildAISpanEvent()` or the `export()` conditional yet — that's Task 2. This step just adds the config and UUID helper. The test from Step 1 will still fail because the export method doesn't emit `$ai_span` yet.

- [ ] **Step 4: Commit config + UUID helper (test still failing — that's expected, implementation continues in Task 2)**

No commit yet — the test depends on `buildAISpanEvent()` which is Task 2. Proceed directly.

---

### Task 2: Add `buildAISpanEvent()` and export conditional

**Files:**

- Modify: `src/modules/exporters/posthog.ts` (add `buildAISpanEvent()`, add conditional in `export()`)
- Modify: `src/tests/posthog-exporter.test.ts` (add emission test)

- [ ] **Step 1: Write the failing test for $ai_span emission on tool calls**

```typescript
it("should emit $ai_span alongside regular event for tool calls when enableAITracing is true", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  await exporter.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
      resourceName: "get_weather",
      duration: 250,
      parameters: { city: "London" },
      response: { temp: 15 },
    }),
  );

  const body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  expect(body.batch).toHaveLength(2); // regular + $ai_span

  const regular = body.batch[0];
  expect(regular.event).toBe("mcp_tool_call");

  const span = body.batch[1];
  expect(span.event).toBe("$ai_span");
  expect(span.type).toBe("capture");
  expect(span.distinct_id).toBe("ses_session456");
  expect(span.timestamp).toBe("2025-01-15T10:00:00.000Z");

  // Core $ai_* properties — full property schema verification
  expect(span.properties.$ai_trace_id).toBeDefined(); // deterministic UUID from sessionId
  expect(span.properties.$ai_span_id).toBeDefined(); // deterministic UUID from eventId
  expect(span.properties.$ai_trace_id).not.toBe(span.properties.$ai_span_id); // different inputs
  expect(span.properties.$ai_span_name).toBe("get_weather");
  expect(span.properties.$ai_latency).toBeCloseTo(0.25); // 250ms → 0.25s
  expect(span.properties.$ai_is_error).toBe(false);
  expect(span.properties.$ai_input_state).toEqual({ city: "London" });
  expect(span.properties.$ai_output_state).toEqual({ temp: 15 });
  expect(span.properties.$session_id).toBe("ses_session456");
  expect(span.properties.source).toBe("mcpcat");
  expect(span.properties.server_name).toBe("weather-server");
  expect(span.properties.client_name).toBe("claude-desktop");
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: FAIL — batch has length 1, no `$ai_span` event.

- [ ] **Step 3: Implement `buildAISpanEvent()` and export conditional**

Add the `buildAISpanEvent()` method to the `PostHogExporter` class:

```typescript
private buildAISpanEvent(event: Event): PostHogCaptureEvent {
  const distinctId =
    event.identifyActorGivenId || event.sessionId || "anonymous";
  const timestamp = event.timestamp
    ? event.timestamp.toISOString()
    : new Date().toISOString();

  const properties: Record<string, any> = {
    $ai_trace_id: this.toUUID(event.sessionId),
    $ai_span_id: this.toUUID(event.id),
    $ai_span_name: event.resourceName || "unknown_tool",
    $ai_is_error: event.isError || false,
    $session_id: event.sessionId,
    source: "mcpcat",
  };

  if (event.duration !== undefined) {
    properties.$ai_latency = event.duration / 1000;
  }
  if (event.isError && event.error) {
    properties.$ai_error = event.error;
  }
  if (event.parameters !== undefined) {
    properties.$ai_input_state = event.parameters;
  }
  if (event.response !== undefined) {
    properties.$ai_output_state = event.response;
  }
  if (event.serverName) properties.server_name = event.serverName;
  if (event.clientName) properties.client_name = event.clientName;

  // Spread customer tags directly (can override MCPCat defaults)
  if (event.tags) {
    for (const [key, value] of Object.entries(event.tags)) {
      properties[key] = value;
    }
  }

  // Spread customer properties directly (can override MCPCat defaults)
  if (event.properties) {
    for (const [key, value] of Object.entries(event.properties)) {
      properties[key] = value;
    }
  }

  return {
    event: "$ai_span",
    distinct_id: distinctId,
    properties,
    timestamp,
    type: "capture",
  };
}
```

Add the conditional in `export()`. Insert it after the existing `$exception` block (lines 39-41 of posthog.ts) and before the `writeToLog` call (line 43). The surrounding code looks like:

```typescript
// Send $exception event alongside if this is an error
if (event.isError && event.error) {
  batch.push(this.buildExceptionEvent(event));
}

// >>> INSERT HERE <<<
// Send $ai_span for tool calls when AI tracing is enabled
if (
  this.config.enableAITracing &&
  event.eventType === PublishEventRequestEventTypeEnum.mcpToolsCall
) {
  batch.push(this.buildAISpanEvent(event));
}

writeToLog(`PostHogExporter: Sending ${batch.length} event(s) for ${event.id}`);
```

- [ ] **Step 4: Run tests to verify Task 1 + Task 2 tests pass**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: PASS for both new tests and all existing tests.

- [ ] **Step 5: Commit**

```bash
git add src/modules/exporters/posthog.ts src/tests/posthog-exporter.test.ts
git commit -m "feat: add $ai_span emission for PostHog AI tracing"
```

---

### Task 3: Test $ai_span NOT emitted when flag is off

**Files:**

- Modify: `src/tests/posthog-exporter.test.ts`

- [ ] **Step 1: Write the test**

```typescript
it("should NOT emit $ai_span when enableAITracing is false or unset", async () => {
  // Unset (default)
  const exporter1 = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
  });
  await exporter1.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
    }),
  );
  let body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  expect(body.batch).toHaveLength(1);
  expect(body.batch[0].event).toBe("mcp_tool_call");

  // Explicitly false
  fetchSpy.mockClear();
  const exporter2 = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: false,
  });
  await exporter2.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
    }),
  );
  body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  expect(body.batch).toHaveLength(1);
  expect(body.batch[0].event).toBe("mcp_tool_call");
});
```

- [ ] **Step 2: Run test to verify it passes (this is a guard test — implementation already handles this)**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: PASS

- [ ] **Step 3: Commit**

```bash
git add src/tests/posthog-exporter.test.ts
git commit -m "test: verify $ai_span not emitted when enableAITracing is off"
```

---

### Task 4: Test $ai_span NOT emitted for non-tool-call events

**Files:**

- Modify: `src/tests/posthog-exporter.test.ts`

- [ ] **Step 1: Write the test**

```typescript
it("should NOT emit $ai_span for non-tool-call events even with enableAITracing", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  const nonToolCallTypes = [
    PublishEventRequestEventTypeEnum.mcpInitialize,
    PublishEventRequestEventTypeEnum.mcpToolsList,
    PublishEventRequestEventTypeEnum.mcpResourcesRead,
    PublishEventRequestEventTypeEnum.mcpResourcesList,
    PublishEventRequestEventTypeEnum.mcpPromptsGet,
    PublishEventRequestEventTypeEnum.mcpPromptsList,
  ];

  for (const eventType of nonToolCallTypes) {
    fetchSpy.mockClear();
    await exporter.export(makeEvent({ eventType }));
    const body = JSON.parse(fetchSpy.mock.calls[0][1].body);
    const aiSpan = body.batch.find((e: any) => e.event === "$ai_span");
    expect(aiSpan).toBeUndefined();
  }
});
```

- [ ] **Step 2: Run test to verify it passes**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: PASS

- [ ] **Step 3: Commit**

```bash
git add src/tests/posthog-exporter.test.ts
git commit -m "test: verify $ai_span only emitted for tool call events"
```

---

### Task 5: Test customer tags/properties spread directly on $ai_span

**Files:**

- Modify: `src/tests/posthog-exporter.test.ts`

- [ ] **Step 1: Write the test**

```typescript
it("should spread customer tags and properties directly on $ai_span (not namespaced)", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  await exporter.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
      tags: { env: "production", region: "us-east" },
      properties: { feature_flag: "new_ui", count: 42 },
    }),
  );

  const body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  const span = body.batch.find((e: any) => e.event === "$ai_span");

  // Tags spread directly (NOT as mcpcat_tag_*)
  expect(span.properties.env).toBe("production");
  expect(span.properties.region).toBe("us-east");

  // Properties spread directly (NOT as mcpcat_properties)
  expect(span.properties.feature_flag).toBe("new_ui");
  expect(span.properties.count).toBe(42);

  // Verify regular event still uses namespaced format
  const regular = body.batch.find((e: any) => e.event === "mcp_tool_call");
  expect(regular.properties.mcpcat_tag_env).toBe("production");
  expect(regular.properties.mcpcat_properties).toEqual({
    feature_flag: "new_ui",
    count: 42,
  });
});
```

- [ ] **Step 2: Write test for customer tags overriding MCPCat defaults**

```typescript
it("should allow customer tags to override $ai_* defaults on $ai_span", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  const customTraceId = "custom-trace-uuid-from-customer";
  await exporter.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
      tags: { $ai_trace_id: customTraceId, $ai_span_name: "custom_name" },
    }),
  );

  const body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  const span = body.batch.find((e: any) => e.event === "$ai_span");

  // Customer tag overrides MCPCat's generated $ai_trace_id
  expect(span.properties.$ai_trace_id).toBe(customTraceId);
  expect(span.properties.$ai_span_name).toBe("custom_name");
});
```

- [ ] **Step 3: Run tests to verify they pass**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: PASS

- [ ] **Step 4: Commit**

```bash
git add src/tests/posthog-exporter.test.ts
git commit -m "test: verify tags/properties spread directly on $ai_span with override support"
```

---

### Task 6: Test error tool call with AI tracing (3 events in batch)

**Files:**

- Modify: `src/tests/posthog-exporter.test.ts`

- [ ] **Step 1: Write the test**

```typescript
it("should emit regular + $exception + $ai_span for error tool calls with enableAITracing", async () => {
  const exporter = new PostHogExporter({
    type: "posthog",
    apiKey: "phc_test_key",
    enableAITracing: true,
  });

  await exporter.export(
    makeEvent({
      eventType: PublishEventRequestEventTypeEnum.mcpToolsCall,
      isError: true,
      error: {
        message: "Tool execution failed",
        type: "ExecutionError",
      },
    }),
  );

  const body = JSON.parse(fetchSpy.mock.calls[0][1].body);
  expect(body.batch).toHaveLength(3);

  // Event order: regular, $exception, $ai_span
  expect(body.batch[0].event).toBe("mcp_tool_call");
  expect(body.batch[1].event).toBe("$exception");
  expect(body.batch[2].event).toBe("$ai_span");

  // Verify $ai_span error properties
  const span = body.batch[2];
  expect(span.properties.$ai_is_error).toBe(true);
  expect(span.properties.$ai_error).toEqual({
    message: "Tool execution failed",
    type: "ExecutionError",
  });
});
```

- [ ] **Step 2: Run test to verify it passes**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: PASS

- [ ] **Step 3: Commit**

```bash
git add src/tests/posthog-exporter.test.ts
git commit -m "test: verify error tool calls emit 3 events with AI tracing"
```

---

### Task 7: Run full test suite and final commit

**Files:**

- No new changes — verification only

- [ ] **Step 1: Run the full PostHog exporter test file**

Run: `pnpm test -- --run src/tests/posthog-exporter.test.ts`
Expected: All tests PASS (existing + 7 new)

- [ ] **Step 2: Run typecheck**

Run: `pnpm tsc --noEmit`
Expected: No errors

- [ ] **Step 3: Run full test suite**

Run: `pnpm test -- --run`
Expected: All tests PASS

- [ ] **Step 4: Verify no unintended changes**

Run: `git diff --stat`
Expected: Only `src/modules/exporters/posthog.ts` and `src/tests/posthog-exporter.test.ts` modified.
