# Closed-Loop Lead Generation & CRM System — Plan

**Owner:** Kev · **Status:** Proposal · **Drafted:** 2026-05-26

## Executive summary

Today InfiniBot can scrape leads from Google Maps/Instagram/directories and persist them to a per-business CRM. That's where the loop stops — there's no enrichment, no qualification, no outreach, no reply detection, no follow-up cadence. This document specifies the architecture and phased rollout to close the loop end-to-end, so the user runs ONE command ("find me 50 pressure-wash leads in Florida") and the system autonomously enriches → qualifies → emails → follows up → routes replies → reports — all visible live in the InfiniBot UI.

## Current state

| Layer | What exists | Gap |
|---|---|---|
| **Discovery** | `lead_scrape` (Google Maps, Instagram, directories via Playwright) | Returns name + Maps URL only. No website, phone, email, address. |
| **Persistence** | Auto-insert into per-business Leads DB + `business.clients.create` for the Clients page | Two parallel stores (leads DB + clients DB) drift; no normalization. |
| **Enrichment** | `lead_generation` MCP tool with Hunter.io / Apollo / Google Places adapters | All require API keys that aren't configured. No fallback to Playwright-based Maps enrichment. |
| **Qualifier** | None | Lead "quality" field exists but is always `unknown`. |
| **Outreach** | None | No email send, no template store, no per-lead message log. |
| **Reply detection** | None | No IMAP/webhook ingestion, no reply classification. |
| **Follow-up** | None | No scheduler, no cadence policy. Continuous-workflow kind exists in the runtime but no lead-cadence workflow uses it yet. |
| **UI** | Clients page (grid/table/kanban), detail panel, CSV import, Notion sync | No pipeline-stage view, no per-lead activity timeline, no campaign view, no live action feed. |

## Vision: the closed loop

```
   ┌─────────────┐   ┌────────────┐   ┌───────────┐   ┌──────────┐   ┌─────────┐
   │ Lead scrape ├──▶│ Enrichment ├──▶│ Qualifier ├──▶│ Outreach ├──▶│ Inbox / │
   │ (Maps/IG/…) │   │ (web+email)│   │ (score+   │   │ (email,  │   │ replies │
   └─────────────┘   └────────────┘   │  promote) │   │  threads)│   └────┬────┘
          ▲                            └───────────┘   └────┬─────┘        │
          │                                                 │              ▼
          │           ┌──────────────────┐         ┌────────┴────────┐  ┌──────────┐
          └───────────┤ Reporting /      │◀────────┤ Follow-up loop  │◀─┤ Reply    │
                      │ campaign funnel  │         │ (continuous wf) │  │ classifier│
                      └──────────────────┘         └─────────────────┘  └──────────┘
```

Every stage updates the lead's row in real time; the UI streams those changes via the existing event bus so the Clients page, the new Pipeline view, and the agent activity feed all reflect state without page reload.

## Lead lifecycle states

State machine for the `Status` property on each lead row:

| State | Set by | Next states |
|---|---|---|
| `raw` | scraper | `enriching`, `dead` (invalid) |
| `enriching` | enrichment worker | `enriched`, `enrichment_failed` |
| `enriched` | enrichment worker | `scoring` |
| `scoring` | qualifier worker | `qualified`, `disqualified` |
| `qualified` | qualifier | `queued` (auto if campaign accepts) or held for human |
| `queued` | outreach scheduler | `contacted` (after first send) |
| `contacted` | outreach worker | `opened`, `replied`, `bounced`, `unresponsive` |
| `opened` | tracking pixel | `replied`, `unresponsive` |
| `replied` | inbox listener | `engaged`, `not_interested`, `unsubscribed`, `oof` |
| `engaged` | reply classifier | `meeting_booked`, `nurture`, `lost` |
| `meeting_booked` | human or agent | `won`, `lost` |
| `unresponsive` | follow-up scheduler | re-queue OR `dead` after N follow-ups |
| `dead` | any | terminal |

Each transition writes an event to a per-lead `activity` log row (separate child rows on the lead).

## Architecture components

### 1. Lead Generation Module (existing — needs upgrade)

Files: `src/workflows/playwright/scrape-leads.ts`, `src/agents/tools/lead-scrape-tool.ts`.

**Change**: extend the Playwright `google_maps` scraper to click into each result and extract Website, Phone, Address, Rating, ReviewCount BEFORE returning. Roughly +2s/lead but eliminates 100% of post-enrichment trips. Optional `enrich: false` flag for fast list-only scraping.

### 2. Enrichment Module (new)

Files: new `src/workflows/enrichment/`.

- **Tier 1 — Maps detail**: already addressed by upgrading the scraper.
- **Tier 2 — Email finder**: optional. Hunter.io or Apollo. If no API key, fall back to MX-record check + common pattern probes (`firstname.lastname@domain`, `info@`, `contact@`) and use the highest-confidence Hunter-style verifier.
- **Tier 3 — Domain enrichment**: if a website exists, fetch homepage, extract company size signals (employee count phrases, "founded in", press kit links), pull email/phone from contact page.
- Runs in a worker queue. Each lead enriches independently; failures don't block the campaign.

### 3. Qualifier Module (new)

Files: new `src/workflows/qualifier/`.

- **Fast rules**: present in `qualifier-rules.ts`. Examples: no-website disqualifies for "websites" service offers but qualifies for "build-them-a-site" pitches; rating < 3.5 disqualifies; review count < 5 may indicate too-new business.
- **LLM scoring**: a `pi-embedded-runner` agent reads the enriched lead JSON + the campaign's qualifier prompt template, emits a `{score: 0-100, reason: "...", recommended_pitch: "..."}` JSON.
- The score writes back to the lead row. Threshold (configurable per campaign, default 70) promotes to `qualified`.

### 4. Outreach Module (new)

Files: new `src/workflows/outreach/`.

- **Provider**: start with **Resend** (simplest API, generous free tier, decent deliverability). Architect as `OutreachProvider` interface so SendGrid/Postmark/SMTP can swap in later.
- **Templates**: markdown files under `~/.infinibot/outreach/templates/<name>.md` with frontmatter (subject, follow-up cadence in days, retire-after count). Body uses `{{lead.name}}`, `{{lead.website || "your business"}}`, `{{me.first_name}}` mustache interpolation. Agent fills a `personalized_opener` slot via the qualifier's `recommended_pitch`.
- **Send queue**: `outreach_queue` rows in the existing `tasks` DB so it shows up in kanban for free. Each row has `lead_id`, `template_id`, `scheduled_for`, `sent_at`, `status`, `thread_id`, `message_id`.
- **Tracking pixel**: 1×1 transparent gif served at `/api/track/open/:msg_id` writes an `opened` event.
- **Link click tracking**: rewrite outbound URLs through `/api/track/click/:msg_id?url=…`.

### 5. Follow-up Scheduler (new)

Files: new `src/workflows/outreach/cadence.ts`, run as a **continuous workflow** (`kind: "continuous"` — the runtime already supports this, added 2026-05-23 per project memory).

- Daemon loop ticks every 5 minutes.
- For each lead in `contacted` / `opened` / `unresponsive` whose `next_touch_at <= now` and `follow_up_count < template.max_followups`:
  - Send next template variant
  - Increment `follow_up_count`
  - Set `next_touch_at = now + template.cadence_days[follow_up_count]`
- Stop conditions: reply received, unsubscribed, max follow-ups reached, manual hold.

### 6. Inbox / Reply Detection (new)

Files: new `src/workflows/inbox/`.

- **Listener**: IMAP IDLE on a dedicated outreach mailbox (recommend a subdomain like `hello@outreach.kingly.app`). Resend supports inbound webhooks too — that's lower-latency and reduces moving parts.
- **Threading**: match incoming mail by `In-Reply-To` / `References` headers against our stored `message_id`. Fallback: subject + from-address fuzzy match.
- **Classifier**: LLM call with reply text + original outbound message context. Outputs `{class: "interested"|"not_interested"|"unsubscribe"|"oof"|"question", urgency: low|medium|high, suggested_action: "..."}`.
- High-urgency interested replies surface as a notification in the InfiniBot UI activity feed AND a kanban "respond now" task.

### 7. CRM / Kanban surface (existing — extend)

- Add a **Pipeline view** to the Clients page: kanban-board mode grouped by `Status`, columns mirror the lifecycle states.
- Add a **per-lead activity timeline** sub-panel: scrollable list of all events (sent, opened, clicked, replied, classified, follow-up scheduled).
- Existing Notion sync continues to work — extend the property mapping to include the new fields.

### 8. UI surfaces (mostly new)

| Surface | Purpose | Implementation hint |
|---|---|---|
| Pipeline dashboard | Live kanban by Status | Reuse `clients.set_view_mode("kanban") + set_kanban_group_by("Status")` |
| Lead detail timeline | Per-lead event log + email thread | New tab inside the existing client detail panel |
| Campaign view | Funnel KPIs, source breakdown, top templates | New top-level page; one row per `Campaign` row |
| Live agent action feed | What the autonomy is doing right now | Reuse the subagent-activity-panel pattern, point at outreach/enrichment events |
| Reply inbox | Triage view for replies needing human input | New page; filtered list of `engaged` leads sorted by classifier urgency |
| Template editor | CRUD on outreach templates with preview | New page; markdown editor with live `{{lead.*}}` preview against a sample lead |

## Autonomy model

The whole pipeline runs as **one continuous workflow** that ticks every 5 minutes:

```
every 5min:
  1. raw → enriched         (concurrency cap: 10/tick)
  2. enriched → scored      (concurrency cap: 20/tick)
  3. scored → queued        (if campaign auto-promote: true)
  4. queued → sent          (rate-limited per provider plan)
  5. drain inbox            (classify new replies)
  6. schedule follow-ups    (overdue cadence rows)
  7. emit metrics           (write campaign counters)
```

Each step is idempotent and resumable — a gateway restart picks up where it left off because all queues are kanban rows, not in-memory state.

Optional manual hold: any lead can be `paused: true` (a new boolean property) to take it out of the autonomy loop without changing its status.

## Data model

### Lead row (existing + additions)

```
Name, Email, Phone, Website, Address    (existing)
Status                                   (existing — repurpose to lifecycle enum)
Tags                                     (existing)
Notes                                    (existing)
+ Source            (multiSelect: google_maps, instagram, directory, manual, csv)
+ SourceQuery       (text — the original scrape query for traceability)
+ CampaignId        (relation → Campaigns DB)
+ QualifierScore    (number 0-100)
+ QualifierReason   (richText)
+ RecommendedPitch  (text — fed into the personalized_opener slot)
+ FollowUpCount     (number)
+ LastContactedAt   (date)
+ NextTouchAt       (date — when scheduler should act)
+ ReplyCount        (number)
+ Opens / Clicks    (number)
+ Paused            (checkbox)
```

### New tables

**Campaigns**: `name, seedQuery, sourceList[], qualifierTemplate, outreachTemplate, autoPromote, startedAt, ownerAgentId`.

**OutreachMessages**: `leadId, campaignId, templateId, subject, body, sentAt, threadId, messageId, status, openedAt, repliedAt, replyClassification, replyBody`.

**Templates**: `name, subject, body, cadenceDays[], maxFollowups, retiredAt`. Stored as markdown files on disk, indexed in DB.

## Phased rollout

### Phase 1 — Foundation (~1 week)
- Upgrade `lead_scrape` to capture website/phone/address in one pass.
- Add new lead properties (Status enum, QualifierScore, NextTouchAt, etc.).
- Pipeline kanban view in the Clients page (group-by Status).
- Manual "send email" button on lead detail (uses a single hardcoded template). Plumbs the outreach send path end-to-end.

### Phase 2 — Enrichment + qualifier (~1 week)
- Enrichment worker (Maps detail + optional Hunter.io).
- Qualifier worker (LLM + rule pre-filter).
- Auto-promote `qualified` → `queued`.

### Phase 3 — Outreach send + tracking (~1 week)
- Resend integration.
- Template store on disk + DB index.
- Tracking pixel + link click rewrite.
- Per-lead activity timeline UI.

### Phase 4 — Reply loop (~1 week)
- Resend inbound webhook OR IMAP listener.
- LLM reply classifier.
- Auto state transitions + "needs-human" surfacing in the activity feed.

### Phase 5 — Follow-up scheduler (~3-5 days)
- Continuous-workflow daemon ticking the cadence.
- Stop conditions, manual hold, max follow-ups.

### Phase 6 — Campaigns + reporting (~1 week)
- Campaign DB + UI.
- Funnel KPIs (per campaign, per template).
- A/B template variants.
- Compliance: unsubscribe link, sender identification, GDPR check.

## Open questions to resolve before kicking off

1. **Email provider** — Resend (recommended) vs SendGrid vs Postmark vs raw SMTP? Affects Phase 3.
2. **Sending domain** — new subdomain like `outreach.kingly.app` to isolate reputation? DNS records (SPF, DKIM, DMARC) need to be set.
3. **Warm-up plan** — cold sends from a fresh domain land in spam. Manual warm-up (50/day ramping) or Mailwarm-style automation?
4. **Compliance** — CAN-SPAM (US) + CASL (Canada) + GDPR (EU). Unsubscribe in every email, physical address, identifiable sender. Mandatory.
5. **Where do replies land** — single shared inbox vs per-campaign inboxes?
6. **Rate limits** — Resend free = 100/day, paid = up to 50k/mo. Plan for what volume we expect.
7. **Bounce handling** — hard bounces should mark leads `dead`, soft bounces retry once.
8. **Lead deduplication** — same business found in multiple scrapes → merge on website+phone, not on name.

## Success metrics

| Metric | Target |
|---|---|
| Scraped → enriched | 90% within 1 hour |
| Enriched → qualified | 30% (industry varies) |
| Qualified → contacted | 100% within 24h |
| Contacted → reply | 5-15% (cold-email benchmark) |
| Replied → meeting | 30% of replies |
| Cycle: scrape → first email (autonomous) | < 2 hours |
| Agent intervention rate | < 10% of leads need human review |

## Risks

- **Deliverability** — cold outreach without warm-up = inbox graveyard. Must plan warm-up before Phase 3 production use.
- **Compliance** — sending to unverified business emails without opt-in is legally questionable in some jurisdictions. Stick to B2B with clear identifiable purpose, always include unsubscribe.
- **Qualifier accuracy** — first LLM scoring runs will be wrong. Build in a `qualifier_human_override` slot from day 1 so corrections train the prompt.
- **Reply classifier false-positives** — auto-marking someone `not_interested` based on a one-line LLM call could miss interested replies. Confidence threshold + human review queue for low-confidence cases.
- **Lead duplicate explosion** — re-scraping the same query creates dupes. Dedup keyed on (website || phone || maps_place_id).

## Immediate next steps

1. User approves scope (or amends).
2. Decide email provider + sending domain (Open Question #1, #2).
3. Spawn Phase 1 coder agent: scraper-enrichment + Pipeline kanban view + new lead properties.
4. In parallel: register the `outreach.kingly.app` subdomain (or chosen alt), set DNS.
5. Daily standup against the phase checklist until Phase 4 ships (point at which the loop is actually closed).
