# @sentinel-ai-solutions/agent-kit/proactive-surfacing

Proactive Surfacing Primitive — the generic infrastructure that every proactive feature in an agent-kit consumer inherits.

Build it once. Wire in a new TriggerSource. Everything else — scoring, threshold filtering, channel delivery, calibration — is free.

## The four abstractions

| Interface | Role |
|---|---|
| `TriggerSource` | Emits `CandidateEvent` objects from a data domain (intel scan, pipeline stall, AR overdue, school calendar) |
| `RelevanceScorer` | Scores each candidate against the current `OperatorContext` — returns a score (0–1), urgency, and rationale |
| `SurfacingChannel` | Formats a scored candidate into a channel-specific message and delivers it |
| `CalibrationLearner` | Records operator responses (`acted`, `dismissed`, `more_like_this`, `less_like_this`) and adjusts per-source multipliers |

## Pipeline flow

```
TriggerSources.fetchCandidates()
        ↓
  filter expired
        ↓
RelevanceScorer.score()
        ↓
  apply operator perSourceCalibration (Batch 4)
        ↓
  filter < threshold (operator override or pipeline default)
        ↓
  pick channel (preferredChannels → fallback "telegram")
        ↓
SurfacingChannel.format()
        ↓
SurfacingChannel.send()
        ↓
  return sent messages
```

Operator response (from Telegram callback or dashboard action):

```
pipeline.recordResponse(operatorId, sourceName, candidate, response)
        ↓
CalibrationLearner.record()   (adjusts future multipliers)
```

## Reference implementations

| Class | Batch | Role |
|---|---|---|
| `RuleBasedRelevanceScorer` | 1 | Fast deterministic scorer |
| `LLMAugmentedRelevanceScorer` | 3 | Wraps rule scorer + Sonnet sanity-check |
| `TelegramSurfacingChannel` | 1 | HTML messages + inline CTA keyboard |
| `DashboardSurfacingChannel` | 4 | Markdown messages enqueued to a web dashboard |
| `MemoryCalibrationLearner` | 1 | In-memory calibration stub |
| `DummyTriggerSource` | 1 | Canned events for testing |
| `PipelineCoachingTriggerSource` | 3 | Reference shape for CRM-backed sources |
| `Debouncer` + `InMemoryDebounceStore` | 3 | Per-operator rate limits + quiet hours |
| `SurfacingAggregator` | 3 | Batch non-urgent candidates into a single message |
| `InMemoryOperatorPreferencesStore` | 4 | Per-operator pipeline overrides (in-memory) |

## Example usage

```typescript
import {
  DummyTriggerSource,
  MemoryCalibrationLearner,
  ProactiveSurfacingPipeline,
  RuleBasedRelevanceScorer,
  TelegramSurfacingChannel,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";
import { createTelegramBot } from "@sentinel-ai-solutions/agent-kit/telegram";

const bot = createTelegramBot({ token: process.env.TELEGRAM_BOT_TOKEN! });
const calibration = new MemoryCalibrationLearner();
const scorer = new RuleBasedRelevanceScorer(calibration);

const telegramChannel = new TelegramSurfacingChannel({
  bot,
  defaultChatId: Number(process.env.TELEGRAM_CHAT_ID),
});

const pipeline = new ProactiveSurfacingPipeline(
  [new DummyTriggerSource("intel", [...])],
  scorer,
  new Map([["telegram", telegramChannel]]),
  calibration,
  { scoreThreshold: 0.6 },
);

// Run once (cron, webhook, morning briefing trigger)
const operatorContext = {
  operatorId: "brad",
  priorities: ["KPMG", "gatha", "pipeline"],
  preferredChannels: ["telegram"],
};

const sent = await pipeline.run(operatorContext, new Date(Date.now() - 24 * 60 * 60 * 1000));
```

Handling the operator's Telegram response:

```typescript
import { parseSurfacingCallback } from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

// In your Telegram callback handler:
const parsed = parseSurfacingCallback(callbackData);
if (parsed) {
  await pipeline.recordResponse(
    operatorId,
    sourceName,       // store sourceName in candidate.sourceName
    candidate,        // retrieve from your candidate store by shortId
    parsed.response,
  );
}
```

## CandidateEvent payload conventions

Scorers and formatters read well-known keys from `payload`:

| Key | Type | Used by |
|---|---|---|
| `baseRelevance` | `number` (0–1) | `RuleBasedRelevanceScorer` — base score |
| `title` | `string` | `TelegramSurfacingChannel`, `DashboardSurfacingChannel` — message header |
| `summary` | `string` | `TelegramSurfacingChannel`, `DashboardSurfacingChannel` — message body |
| `tags` | `string[]` | Optional — for future tag-based scoring |

All other keys are passed through unchanged and available to custom implementations.

## Telegram callback_data format

Buttons produced by `TelegramSurfacingChannel` use `ps_<action>|<shortId>`:

| Button | `callback_data` | `OperatorResponse` |
|---|---|---|
| ✓ Acted | `ps_acted|<8chars>` | `"acted"` |
| ▲ More like this | `ps_more|<8chars>` | `"more_like_this"` |
| ▼ Less like this | `ps_less|<8chars>` | `"less_like_this"` |
| ✕ Dismiss | `ps_dismissed|<8chars>` | `"dismissed"` |

`<8chars>` is the first 8 non-hyphen characters of `candidate.id`. Use `parseSurfacingCallback(callbackData)` to parse these in your callback handler.

## Dashboard channel (Batch 4)

`DashboardSurfacingChannel` writes to a `SurfacingMessageQueue` instead of calling Telegram. The Command Centre web app reads from the queue to render an operator inbox.

### Quick start

```typescript
import {
  DashboardSurfacingChannel,
  InMemorySurfacingMessageQueue,
  ProactiveSurfacingPipeline,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const queue = new InMemorySurfacingMessageQueue(); // swap for Prisma in production
const dashboardChannel = new DashboardSurfacingChannel(queue);

const pipeline = new ProactiveSurfacingPipeline(
  sources, scorer,
  new Map([["dashboard", dashboardChannel]]),
  calibration,
);

// Messages are enqueued automatically when pipeline.run() fires
await pipeline.run(
  { operatorId: "brad", preferredChannels: ["dashboard"] },
  since,
);

// Dashboard reads the inbox:
const messages = await queue.list("brad", 20);
// Mark a message read when the operator views it:
await queue.markRead("brad", messages[0].id);
```

### Dashboard CTA action format

Dashboard buttons use the same `ps_<action>|<shortId>` format as Telegram, so a single callback handler can service both channels:

| Button label | `action` | `OperatorResponse` |
|---|---|---|
| Act | `ps_acted|<8chars>` | `"acted"` |
| Dismiss | `ps_dismissed|<8chars>` | `"dismissed"` |
| More like this | `ps_more|<8chars>` | `"more_like_this"` |
| Less like this | `ps_less|<8chars>` | `"less_like_this"` |

### Prisma-backed queue

For production, implement `SurfacingMessageQueue` against your Prisma schema. Add this model to your consumer's `schema.prisma`:

```prisma
model SurfacingMessage {
  id               String    @id
  operatorId       String
  sourceName       String
  body             String    @db.Text
  ctaJson          Json?
  score            Float
  urgency          String
  candidatePayload Json
  enqueuedAt       DateTime  @default(now())
  readAt           DateTime?

  @@index([operatorId, enqueuedAt])
}
```

Skeleton implementation:

```typescript
import type { PrismaClient } from "@prisma/client";
import type {
  SurfacingMessageQueue,
  SerializedSurfacingMessage,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

export class PrismaSurfacingMessageQueue implements SurfacingMessageQueue {
  constructor(private readonly prisma: PrismaClient) {}

  async enqueue(message: SerializedSurfacingMessage): Promise<void> {
    await this.prisma.surfacingMessage.create({
      data: {
        id: message.id,
        operatorId: message.operatorId,
        sourceName: message.sourceName,
        body: message.body,
        ctaJson: message.cta ?? null,
        score: message.score,
        urgency: message.urgency,
        candidatePayload: message.candidatePayload,
        enqueuedAt: message.enqueuedAt,
      },
    });
  }

  async list(
    operatorId: string,
    limit = 50,
  ): Promise<SerializedSurfacingMessage[]> {
    const rows = await this.prisma.surfacingMessage.findMany({
      where: { operatorId },
      orderBy: { enqueuedAt: "desc" },
      take: limit,
    });
    return rows.map((r) => ({
      id: r.id,
      operatorId: r.operatorId,
      sourceName: r.sourceName,
      body: r.body,
      cta: r.ctaJson as { label: string; action: string }[] | undefined,
      score: r.score,
      urgency: r.urgency as SerializedSurfacingMessage["urgency"],
      candidatePayload: r.candidatePayload as Record<string, unknown>,
      enqueuedAt: r.enqueuedAt,
      readAt: r.readAt ?? undefined,
    }));
  }

  async markRead(operatorId: string, messageId: string): Promise<void> {
    await this.prisma.surfacingMessage.updateMany({
      where: { id: messageId, operatorId },
      data: { readAt: new Date() },
    });
  }
}
```

## Operator preferences (Batch 4)

`OperatorPreferencesStore` stores per-operator overrides that the pipeline applies before each run. Useful for:

- Raising or lowering the score threshold for a specific operator
- Redirecting a noisy operator to "dashboard" while keeping "telegram" for others
- Silencing an over-active source for a specific operator without touching global config

### Quick start

```typescript
import {
  InMemoryOperatorPreferencesStore,
  ProactiveSurfacingPipeline,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const prefsStore = new InMemoryOperatorPreferencesStore();

// Set per-operator overrides
await prefsStore.upsert("brad", {
  scoreThreshold: 0.7,            // only surface high-confidence signals
  preferredChannels: ["dashboard", "telegram"],
  perSourceCalibration: {
    growth: { enabled: true, multiplier: 1.5 },   // boost growth signals
    intel: { enabled: false, multiplier: 1.0 },   // silence intel for this operator
  },
});

const pipeline = new ProactiveSurfacingPipeline(
  sources, scorer, channels, calibration,
  { operatorPreferencesStore: prefsStore },
);
```

### Preferences applied by the pipeline automatically

| Preference | How applied |
|---|---|
| `scoreThreshold` | Replaces `options.scoreThreshold` for this operator's run |
| `preferredChannels` | Replaces `operatorContext.preferredChannels` for channel selection |
| `perSourceCalibration.enabled = false` | Source's candidates are silently skipped |
| `perSourceCalibration.multiplier` | `effectiveScore = originalScore × multiplier` (clamped to [0, 1]) |

### Preferences that require custom wiring

`debounceConfig` and `quietHours` are stored in `OperatorPreferences` but the default pipeline does not automatically apply them, because the `Debouncer` is constructed with a fixed config at startup.

To apply per-operator debounce settings, build a preference-aware Debouncer subclass or wrapper that reads from your `OperatorPreferencesStore` inside `shouldSurface()`.

### Prisma schema for operator preferences

```prisma
model OperatorSurfacingPreferences {
  operatorId           String   @id
  scoreThreshold       Float?
  preferredChannelsJson Json?
  debounceConfigJson   Json?
  perSourceCalJson     Json?
  quietHoursJson       Json?
  updatedAt            DateTime @updatedAt
}
```

### Upsert merge semantics

`InMemoryOperatorPreferencesStore.upsert()` performs:
- **Scalar fields** (`scoreThreshold`, `preferredChannels`, `quietHours`): replaced when present in the patch
- **Nested objects** (`perSourceCalibration`, `debounceConfig`): shallow-merged — existing source keys are preserved; new keys are added or overridden

```typescript
await store.upsert("brad", { perSourceCalibration: { growth: { enabled: true, multiplier: 1.5 } } });
await store.upsert("brad", { perSourceCalibration: { intel: { enabled: false, multiplier: 1.0 } } });
// Result: { growth: ..., intel: ... } — both keys preserved
```

## LLMAugmentedRelevanceScorer (Batch 3)

Wraps any `RelevanceScorer` and adds a Sonnet sanity-check for candidates that
land in the "uncertain" middle band. Clearly irrelevant (`< 0.4`) and clearly
relevant (`> 0.75`) candidates skip the LLM call entirely — no cost, no latency.

```typescript
import Anthropic from "@anthropic-ai/sdk";
import {
  LLMAugmentedRelevanceScorer,
  RuleBasedRelevanceScorer,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const anthropic = new Anthropic();
const baseScorer = new RuleBasedRelevanceScorer(calibration);
const scorer = new LLMAugmentedRelevanceScorer(baseScorer, anthropic);

// Custom bands:
const scorer2 = new LLMAugmentedRelevanceScorer(
  baseScorer, anthropic, "claude-sonnet-4-6",
  { lower: 0.35, upper: 0.80 },
);
```

**Result caching (v0.17.0):** scored results are cached for 60 seconds per
`(operatorId, sourceName, candidate.id)` triple. A pipeline run that re-scores
the same candidate (e.g. retry, debugging, secondary surfacing path) does not
re-invoke the LLM. Cached results from the LLM-skipped bands also stick — the
base scorer is not re-invoked either. Pass a custom TTL as the 5th constructor
argument:

```typescript
const scorer3 = new LLMAugmentedRelevanceScorer(
  baseScorer, anthropic, "claude-sonnet-4-6",
  { lower: 0.4, upper: 0.75 },
  120_000,  // 2-minute cache TTL
);
```

## RateLimiter (Batch 3 — v0.17.0)

Standalone rate-limiting primitive. The Debouncer composes one internally
for its hourly/daily guards; consumers who want rate-limiting without quiet
hours can use it directly.

```typescript
import {
  InMemoryDebounceStore,
  RateLimiter,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const store = new InMemoryDebounceStore();
const rateLimiter = new RateLimiter(store, {
  default: { maxPerHour: 5, maxPerDay: 20 },
  perSource: {
    intel: { maxPerHour: 2 },          // tighter cap for chatty source
    pipeline: { maxPerDay: 50 },       // higher daily ceiling
  },
});

const { allow, reason } = await rateLimiter.allow("op_brad", "intel");
if (allow) {
  // ...deliver...
  await rateLimiter.record("op_brad", "intel");
}
```

Per-source entries are partial — unspecified fields fall back to `default`.

## PrismaDebounceStore (Batch 3 — v0.17.0)

Cross-process persistence for Debouncer / RateLimiter state. Required when
multiple Node processes (web + worker + cron) need to share quotas — the
in-memory store isolates state per process and breaks rate limits.

```typescript
import { PrismaClient } from "@prisma/client";
import {
  Debouncer,
  PrismaDebounceStore,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const store = new PrismaDebounceStore(new PrismaClient());
const debouncer = new Debouncer(store, { maxPerHour: 5, maxPerDay: 20 });
```

Add the `SurfacingEmissionLog` model from
[`src/proactive-surfacing/schema.prisma.example`](./schema.prisma.example)
to your `prisma/schema.prisma` and run `prisma migrate dev`. Attach a
periodic prune job (rows older than 48h) — the store does not auto-prune.

## Debouncer (Batch 3)

Rate-limits surfacing per `(operatorId, sourceName)` to prevent floods.
`urgency === "now"` bypasses quiet hours but still respects hourly/daily limits.

```typescript
import {
  Debouncer,
  InMemoryDebounceStore,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const store = new InMemoryDebounceStore();
const debouncer = new Debouncer(store, {
  maxPerHour: 5,
  maxPerDay: 20,
  quietHours: { start: 22, end: 7, tz: "Australia/Perth" },
});

// Pass to pipeline:
const pipeline = new ProactiveSurfacingPipeline(
  sources, scorer, channels, calibration,
  { debouncer },
);
```

## SurfacingAggregator (Batch 3)

Batches `morning_brief` and `weekly_brief` candidates into a single bundled
message rather than sending them individually. `"now"` urgency always bypasses
the aggregator.

Flush triggers:
- Volume threshold (5 items per operator+urgency)
- Manual `aggregator.flush(operatorId, urgency)` call (e.g. from a scheduled 8am routine)

```typescript
import { SurfacingAggregator } from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const aggregator = new SurfacingAggregator();
const pipeline = new ProactiveSurfacingPipeline(
  sources, scorer, channels, calibration,
  { aggregator },
);

// Manual flush (e.g. from morning cron):
const messages = aggregator.flush(operatorId, "morning_brief");
for (const msg of messages) {
  await channel.send(msg);
}
```

## PipelineCoachingTriggerSource (Batch 3 — reference shape)

A documented example of a CRM-backed `TriggerSource`. Copy into your consumer
project and supply a real `DealFetcher` implementation (HubSpot, Prisma, etc.).

```typescript
import {
  PipelineCoachingTriggerSource,
  type DealFetcher,
} from "@sentinel-ai-solutions/agent-kit/proactive-surfacing";

const fetcher: DealFetcher = {
  async fetchDeals(operatorId, since) {
    return myHubSpotClient.deals.list({ ownerId: operatorId });
  },
};

const source = new PipelineCoachingTriggerSource(fetcher, {
  coldThresholdDays: 14,
});
```

## Deferred to later batches

| Feature | Batch |
|---|---|
| Prisma-backed `CalibrationLearner` | 2 |
| `IntelTriggerSource`, `PipelineTriggerSource`, `FinanceTriggerSource`, `FamilyTriggerSource` | 2 |
| ~~Prisma-backed `DebounceStore`~~ — shipped in 0.17.0 as `PrismaDebounceStore` | 3 |
| Persistent aggregator queue | 5 |
| Email `SurfacingChannel` | 5 |
| WebSocket / SSE push for dashboard | 5 |
| Bulk mark-read / dismiss-all | 5 |
| Preference-aware Debouncer (per-operator quiet hours applied automatically) | 5 |
| Per-operator calibration UI | 5 |
