# Validation Tests

## Purpose
Define concrete, pass/fail test scenarios that prove a prototype works before going live. Validation tests are the bridge between "it's built" and "it's ready" — they catch workflow gaps, edge cases, and integration failures that desk-checking misses.

## When to Use
- After building a prototype, before the live validation sprint
- When a PRD defines a multi-step workflow with cross-party coordination
- When the prototype involves scheduled actions (cron, timeouts, follow-ups)
- When there are gates/guards that must block incorrect states
- Before declaring a sprint build "complete"

## Process

1. **Derive scenarios from the workflow**
   - Walk through each step of the workflow and ask: "What proves this step works?"
   - One test per step for the happy path
   - Additional tests for every branch, timeout, and failure mode

2. **Categorize by risk area**
   - **Core workflow**: The happy path end-to-end. If this fails, nothing works.
   - **Timeouts and expiry**: Scheduled actions that fire without human input. Easy to get wrong.
   - **Escalation and enforcement**: Warning → penalty → removal progressions. Must fire in order.
   - **Gates and guards**: Precondition checks that block invalid actions. Must be airtight.
   - **Cross-session and scope**: Multi-party coordination where state is shared. Most technically fragile.
   - **Information relay**: Passing data between parties while preserving constraints (e.g. anonymity).
   - **Reputation and scoring**: Trust scores, priority tiers, blacklist thresholds. Must be deterministic.
   - **Data compliance**: Access and erasure rights. Legal requirement — must work perfectly.
   - **Scheduled job reliability**: Every cron/timeout type fires and the system acts on it correctly.

3. **Write each test as a numbered scenario**
   - **ID**: Short reference (T1, T2, ... Tn)
   - **Title**: What it tests in plain English
   - **Steps**: Numbered sequence of actions (input → expected behaviour)
   - **Pass criteria**: What "success" looks like — specific state changes, messages sent, records updated
   - Keep it concrete: use example values, names, amounts. Abstract tests are hard to run.

4. **Order by dependency**
   - Core workflow first (can't test edge cases if the happy path is broken)
   - Gates and guards before escalation (guards prevent bad states that escalation handles)
   - Cross-session before information relay (relay depends on scope resolution working)
   - Data compliance last (depends on all other data being created correctly first)

5. **Define the test environment**
   - What test accounts/phones are needed?
   - What seed data must exist before testing? (e.g. pre-created profiles, records)
   - How are cron jobs accelerated for testing? (Shortened timers or manual trigger)
   - How do you verify state after each step? (Check memory files, customer records, session history)

6. **Run and record results**
   - Run each test in order
   - Record: pass/fail, actual behaviour, any deviations from expected
   - If a test fails, fix the issue and re-run from the start of that test (not just the failing step)
   - All tests must pass before declaring the build ready for live use

## Test Count Guidelines

| Prototype complexity | Typical test count |
|---|---|
| Simple (single-party, no scheduling) | 5–10 |
| Moderate (multi-party OR scheduling) | 10–20 |
| Complex (multi-party AND scheduling AND enforcement) | 20–30 |

More tests are not necessarily better. Each test should cover a distinct scenario — if two tests would pass or fail together, merge them.

## Example

**Prototype**: AI booking agent that connects customers with service providers via WhatsApp.

**Test categories and counts:**
- Core workflow (happy path, extension round, both rounds fail): 3 tests
- Quote expiry and timeouts (no response, 2nd expiry warning, 3rd expiry blacklist): 3 tests
- Fee collection escalation (on time, late, full escalation to blacklist, reinstatement): 4 tests
- Gates and guards (outstanding fee blocks booking, location filter, concurrent cap, opt-out, opt-in): 5 tests
- Cross-session and scope (virtual group write access, access revocation, late response): 3 tests
- Information relay (provider requests more info, photo forwarding, anonymity preserved): 1 test
- Trust scores (shared during anonymous phase, hidden after reveal, score updates): 3 tests
- Data compliance (export, deletion): 2 tests
- Cron reliability (all scheduled job types fire correctly): 1 test

**Total: 25 tests** — appropriate for a complex multi-party, scheduled, enforced workflow.

## Common Pitfalls

- **Testing only the happy path.** The happy path is the least likely to break. Timeouts, escalations, and edge cases are where prototypes fail.
- **Not testing cron jobs end-to-end.** Scheduling a cron is easy; verifying the agent acts correctly when it fires is where bugs hide.
- **Assuming cross-session state is visible.** If Party A writes data in their session, verify Party B can actually read it in theirs.
- **Skipping gate tests.** A missing precondition check (e.g. outstanding payment) can let invalid state propagate through the entire workflow.
- **Not re-running after fixes.** A fix to one test can break another. Re-run from the top of the category, not just the failing test.
