# Voice-Call Latency Baseline (Phase 1)

Phase 1 ships per-stage telemetry with **no behavior changes**. This file
records measured latency for the chained inbound path (Twilio → STT → PANIK
agent via chat.send → TTS → Twilio) and defines the rubric that Phase 2 will
use to discard or escalate the A–J optimization proposals.

## How to collect

### Live calls (baseline)

1. On the gateway host (netcup), start the gateway with tracing on:
   ```bash
   VOICE_TRACE=1 systemctl --user restart minion-gateway.service
   ```
2. Place 5 calls from your phone to the Twilio number with these scripted prompts:
   - **greeting** — "Hi" (single short utterance, hang up after reply)
   - **factual** — "What's the weather like in Madrid today?"
   - **conversational** — "Tell me a quick joke."
   - **memory-needing** — "What do you remember about our last conversation?"
   - **multi-turn** — three back-and-forth exchanges on any topic
3. Each call writes one JSONL to `~/.minion/voice-calls/traces/YYYY-MM-DD/<CallSid>.jsonl`.
4. Pull the traces locally and run the reader:
   ```bash
   scp 'bot-prd@netcup:.minion/voice-calls/traces/2026-05-16/*.jsonl' /tmp/traces/
   node --experimental-strip-types extensions/voice-call/scripts/voice-trace.ts /tmp/traces/*.jsonl
   ```

### Synthetic fixtures (regression — for Phase 2)

1. On the gateway, capture one real call's inbound audio:
   ```bash
   VOICE_TRACE=1 VOICE_CAPTURE_INBOUND=$HOME/.minion/voice-calls/fixtures \
     systemctl --user restart minion-gateway.service
   ```
   Then place a call. Raw mu-law is written to
   `~/.minion/voice-calls/fixtures/<CallSid>.mulaw.bin`.
2. Commit the fixture(s) to `extensions/voice-call/fixtures/voice-call/` (skip
   anything PII-sensitive — these are raw audio recordings of your voice).
3. Replay against a running gateway (requires a pre-registered call):
   ```bash
   node --experimental-strip-types extensions/voice-call/scripts/voice-replay.ts \
     --fixture fixtures/voice-call/greeting.mulaw.bin \
     --ws ws://127.0.0.1:3334/twilio-media-stream \
     --call CA-replay-1 --stream MZ-replay-1 --token <token>
   ```

## Stages captured

| Stage                     | When                                                          |
| ------------------------- | ------------------------------------------------------------- |
| `call_start`              | t0 anchor — chained `onConnect` fires                         |
| `media_stream_open`       | same moment (kept distinct for symmetry with future S2S path) |
| `speech_started`          | OpenAI Realtime VAD detects speech onset (also bumps turnIdx) |
| `transcript_partial`      | each delta event from OpenAI Realtime                         |
| `transcript_final`        | full utterance committed                                      |
| `response_dispatch_start` | webhook `handleInboundResponse` begins                        |
| `rpc_open`                | chat.send WS connection opens                                 |
| `rpc_sent`                | chat.send request flushed                                     |
| `agent_first_delta`       | first non-empty `chat` delta arrives                          |
| `sentence_emit`           | each completed sentence emitted to TTS                        |
| `tts_request`             | `synthesizeForTelephony` invoked                              |
| `tts_audio_ready`         | full mu-law buffer ready                                      |
| `twilio_first_chunk`      | first sendAudio call to Twilio                                |
| `agent_final`             | full agent reply assembled                                    |
| `turn_complete`           | speakChain resolved (all TTS for this turn played)            |
| `call_end`                | media stream disconnects                                      |

## Computed deltas (printed by the reader)

| Delta                 | Formula                               | What it measures                                       |
| --------------------- | ------------------------------------- | ------------------------------------------------------ |
| `vad_wait`            | transcript_final − speech_started     | STT wall-clock for the user's utterance                |
| `dispatch_overhead`   | rpc_sent − transcript_final           | Time from final transcript to RPC fired                |
| `agent_ttft`          | agent_first_delta − rpc_sent          | Agent first-token latency (model TTFT + agent prelude) |
| `first_sentence_wait` | sentence_emit − agent_first_delta     | Time spent buffering tokens until a sentence boundary  |
| `tts_synth`           | tts_audio_ready − tts_request         | TTS provider round-trip                                |
| `tts_to_audio`        | twilio_first_chunk − tts_request      | TTS + first frame on the wire                          |
| `first_audio`         | twilio_first_chunk − transcript_final | **Perceived TTFB** — silence end to first audio byte   |
| `turn_total`          | turn_complete − transcript_final      | Full end of user speech → end of bot speech            |

## Baseline numbers

Collected 2026-05-17 — 15 turns across 5 live calls placed by user on the
netcup-deployed gateway (`2026.5.15-dev` + globalThis trace fix).

```
delta                    n      p50      p95     mean
vad_wait                15   2427ms   6739ms   3071ms
dispatch_overhead       13     76ms    131ms     88ms
agent_ttft              12   2827ms   3904ms   2619ms
first_sentence_wait     12    325ms   7361ms    556ms
tts_synth               20   1153ms  11168ms   1305ms
tts_to_audio            20   1170ms  11168ms   1677ms
first_audio             14   3879ms  21401ms   3796ms
turn_total               9   5654ms  17558ms   5909ms
```

**Headline**: `first_audio` p50 ≈ 3.9s, p95 ≈ 21.4s. The p95 outlier is the
long single-sentence case (973 chars before the sentence-boundary regex
fires) — D (clause-boundary emit) targets it directly. `vad_wait` is also
way out of spec — configured 800ms silence threshold but actual STT-side
latency runs 2.4-6.7s (B targets this).

### Rubric application (verdict per proposal)

| #   | Proposal                     | Targets                | p50 measured           | Verdict                             |
| --- | ---------------------------- | ---------------------- | ---------------------- | ----------------------------------- |
| A   | streaming TTS                | tts_synth              | 1153ms                 | KEEP (long-tail p95=11168ms is bad) |
| B   | VAD tune / semantic_vad      | vad_wait               | 2427ms                 | **ESCALATE → Phase 2.B**            |
| C   | persistent chat.send session | dispatch_overhead      | 76ms                   | **DISCARD** (< 100ms threshold)     |
| D   | clause-boundary emit         | first_sentence_wait    | 325ms p50 / 7361ms p95 | **ESCALATE → Phase 2.D** (bimodal)  |
| E   | slim voice-mode prompt       | agent_ttft             | 2827ms                 | KEEP, bundle with G/F               |
| F   | Anthropic prompt caching     | agent_ttft             | 2827ms                 | KEEP, bundle with G/E               |
| G   | Haiku for voice              | agent_ttft             | 2827ms                 | KEEP, bundle with E/F               |
| H   | parallelize speakChain       | gaps between sentences | 0.7-5.5s observed      | KEEP, pair with A                   |
| I   | OpenAI Realtime S2S          | first_audio            | 3879ms                 | DEFER (re-eval after A-H)           |
| J   | US edge for Twilio           | RTT contribution       | unknown                | DEFER (need probe)                  |

### Phase 2 tiering

- **Tier 1**: B + D + K (in-process end_call, new from qualitative reports)
- **Tier 2**: A + H + (G/E/F bundle)
- **Tier 3 (correctness, not latency)**: L (abort agent on barge-in) + M (STT silent-failure watchdog)

## Phase 2 re-eval rubric

Once baseline numbers exist, decide each proposal's fate:

| Proposal                                          | Targets                                               | Discard if                      | Escalate if               |
| ------------------------------------------------- | ----------------------------------------------------- | ------------------------------- | ------------------------- |
| **A** streaming TTS (ElevenLabs/OpenAI streaming) | `tts_synth`, `tts_to_audio`                           | `tts_synth` p50 < 300ms         | `tts_synth` p50 > 800ms   |
| **B** VAD tuning / semantic_vad                   | `vad_wait`                                            | `vad_wait` p50 already < 1000ms | `vad_wait` p50 > 1500ms   |
| **C** persistent chat.send session                | `dispatch_overhead`                                   | p50 < 100ms                     | p50 > 250ms               |
| **D** clause-boundary sentence emit               | `first_sentence_wait`                                 | p50 < 200ms                     | p50 > 700ms               |
| **E** slim voice-mode prompt                      | `agent_ttft`                                          | p50 < 1000ms                    | p50 > 2000ms              |
| **F** Anthropic prompt caching                    | `agent_ttft`                                          | already cached upstream         | as above                  |
| **G** Haiku for voice turns                       | `agent_ttft`                                          | p50 < 1000ms                    | p50 > 2500ms              |
| **H** parallelize TTS with later sentence gen     | `turn_total` − `first_audio` for multi-sentence turns | gap < 1000ms total              | gap > 3000ms total        |
| **I** OpenAI Realtime S2S                         | `first_audio`                                         | already < 2000ms                | > 4000ms after A–H landed |
| **J** US-region edge for Twilio                   | `dispatch_overhead`, `tts_synth` (network legs)       | RTT contribution < 100ms        | otherwise                 |

**Discovery rule**: any stage with p50 > 1500ms that isn't covered by A–J
becomes a new Phase 2 candidate. Add it to the table.

## Notes

- `traceEmit` is fire-and-forget. JSONL writes are serialized per call but
  never block the hot path — measurements include zero overhead from the
  trace layer itself.
- The trace key is `providerCallId` (Twilio CallSid). When the Twilio call
  doesn't go through (e.g., S2S direct path), traces won't anchor — Phase 2
  can extend `traceStart` to additional anchor points if S2S becomes the
  hot path.
- Trace files are gitignored (`.minion/voice-calls/traces/` lives under
  `$HOME`, never inside the repo).
