# TestDriver Test Recording Architecture

## Overview

This system provides comprehensive test execution tracking, linking test runs with dashcam screen recordings and CI/CD pipelines in the TestDriver dashboard.

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────────┐
│                    Developer's Machine / CI                  │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────────┐         ┌──────────────┐                  │
│  │   Vitest     │────────▶│  TD Vitest   │                  │
│  │  Test Runner │         │   Plugin     │                  │
│  └──────────────┘         └──────┬───────┘                  │
│                                   │                           │
│  ┌──────────────┐                 │                          │
│  │   Dashcam    │                 │                          │
│  │  Recording   │                 │                          │
│  └──────┬───────┘                 │                          │
│         │                         │                           │
│         │ (records screen)        │ (reports results)        │
│         │                         │                           │
└─────────┼─────────────────────────┼───────────────────────────┘
          │                         │
          │                         │
          ▼                         ▼
┌─────────────────────────────────────────────────────────────┐
│                   TestDriver API Server                      │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────────┐         ┌──────────────┐                  │
│  │ Replay API   │         │ Test Run API │                  │
│  │ (dashcam)    │         │ (new)        │                  │
│  └──────┬───────┘         └──────┬───────┘                  │
│         │                        │                           │
│         └────────┬───────────────┘                           │
│                  │                                            │
│                  ▼                                            │
│         ┌────────────────┐                                   │
│         │   MongoDB      │                                   │
│         │                │                                   │
│         │ • TdTestRun    │                                   │
│         │ • TdTestCase   │                                   │
│         │ • Replay       │                                   │
│         └────────────────┘                                   │
│                                                               │
└───────────────────────────┬───────────────────────────────────┘
                            │
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                  TestDriver Web Dashboard                    │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌─────────────────────────────────────────┐                │
│  │        Test Runs View (NEW)             │                │
│  │                                          │                │
│  │  • List all test runs                   │                │
│  │  • Filter by status, date, CI           │                │
│  │  • Show pass/fail statistics            │                │
│  │  • Link to CI/CD runs                   │                │
│  └─────────────────────────────────────────┘                │
│                                                               │
│  ┌─────────────────────────────────────────┐                │
│  │      Test Run Detail View (NEW)         │                │
│  │                                          │                │
│  │  • Test case list with status           │                │
│  │  • Dashcam replay player (embedded)     │                │
│  │  • Error messages and stack traces      │                │
│  │  • Sandbox details                      │                │
│  │  • Git commit info                      │                │
│  └─────────────────────────────────────────┘                │
│                                                               │
└───────────────────────────────────────────────────────────────┘
```

## Data Models

### TdTestRun
Represents a complete test suite execution (e.g., `vitest run`).

**Key Fields:**
- `runId`: Unique identifier
- `suiteName`: Name of the test suite
- `status`: running | passed | failed | cancelled
- `totalTests`, `passedTests`, `failedTests`: Statistics
- `platform`: windows | mac | linux
- CI/CD metadata (provider, runId, url)
- Git metadata (repo, branch, commit)
- `dashcamSessionId`: Links to dashcam recordings

**Relationships:**
- `team`: Owner team
- `testCases`: Collection of TdTestCase
- `replays`: Associated Replay records

### TdTestCase
Represents an individual test within a test run.

**Key Fields:**
- `testName`: Test name from `test('...')`
- `testFile`: Path to test file
- `suiteName`: Describe block name
- `status`: passed | failed | skipped | pending
- `duration`: Test duration in ms
- `errorMessage`, `errorStack`: Failure details
- `replayUrl`: Direct link to dashcam replay
- `replayStartTime`, `replayEndTime`: Timestamps within replay

**Relationships:**
- `testRun`: Parent TdTestRun
- `replay`: Associated Replay recor
### Replay (Extended)
Existing model extended with test run associations.

**New Fields:**
- `tdTestRun`: Associated test run
- `tdTestCase`: Associated test case

## API Endpoints

### POST /api/v1/testdriver/test-run-create
Create a new test run.

**Auth:** Required (Bearer token)

**Request:**
```json
{
  "runId": "vitest-1234567890-abc123",
  "suiteName": "Integration Tests",
  "platform": "windows",
  "sandboxId": "sandbox-xyz",
  "ciProvider": "GitHub Actions",
  "ciRunId": "12345",
  "repo": "myorg/myrepo",
  "branch": "main",
  "commit": "abc123def456"
}
```

**Response:**
```json
{
  "data": {
    "id": "...",
    "runId": "vitest-1234567890-abc123",
    "status": "running",
    "startTime": 1700000000000
  }
}
```

### POST /api/v1/testdriver/test-run-complete
Mark a test run as complete.

**Auth:** Required

**Request:**
```json
{
  "runId": "vitest-1234567890-abc123",
  "status": "passed",
  "totalTests": 25,
  "passedTests": 24,
  "failedTests": 1,
  "skippedTests": 0
}
```

### POST /api/v1/testdriver/test-case-create
Record a test case result (create or update).

**Auth:** Required

**Request:**
```json
{
  "runId": "vitest-1234567890-abc123",
  "testName": "should login successfully",
  "testFile": "tests/auth/login.test.js",
  "suiteName": "Authentication Tests",
  "status": "passed",
  "startTime": 1700000001000,
  "endTime": 1700000002500,
  "duration": 1500,
  "replayUrl": "https://app.dashcam.io/replay/abc123"
}
```

## Components

### Vitest Plugin (`interfaces/vitest-plugin.mjs`)
Automatically integrates with Vitest test runs.

**Features:**
- Auto-detects CI/CD environment (GitHub Actions, GitLab, etc.)
- Extracts Git metadata from environment or git commands
- Creates test run at start
- Records each test case result
- Associates with dashcam session if `DASHCAM_SESSION_ID` is set
- Completes test run with statistics
- Uses plugin architecture for better global state management

**Usage:**
```javascript
// vitest.config.mjs
import testDriverPlugin from './interfaces/vitest-plugin.mjs';

export default {
  plugins: [
    testDriverPlugin({
      apiKey: process.env.TD_API_KEY,
      apiRoot: process.env.TD_API_ROOT || 'https://api.testdriver.ai',
    }),
  ],
}
```

### SDK Methods (`sdk.js`)

#### `client.createTestRun(options)`
Create a test run programmatically.

#### `client.recordTestCase(options)`
Record a test case result.

#### `client.completeTestRun(options)`
Mark test run as complete.

## Integration Flows

### Flow 1: Automated with Vitest Reporter

```
1. Developer runs: vitest run
2. Vitest starts, reporter initializes
3. Reporter creates TdTestRun
4. For each test:
   - Vitest runs test
   - Reporter records TdTestCase (passed/failed)
5. All tests complete
6. Reporter calls completeTestRun()
7. Results visible in dashboard
```

### Flow 2: With Dashcam Recording

```
1. Start dashcam: dashcam start
2. Set session ID: export DASHCAM_SESSION_ID=$(dashcam session-id)
3. Run tests: vitest run
4. Reporter creates test run with dashcamSessionId
5. Tests execute, dashcam records
6. Stop dashcam: dashcam stop
7. Publish: dashcam publish -p PROJECT_ID
8. Replay URL returned
9. Dashboard shows test results + replay link
```

### Flow 3: CI/CD Pipeline (GitHub Actions)

```yaml
jobs:
  test:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v3
      - run: npm install
      - name: Start Dashcam
        run: |
          dashcam start
          echo "DASHCAM_SESSION_ID=$(dashcam session-id)" >> $GITHUB_ENV
      - name: Run Tests
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}
        run: vitest run
      - name: Publish Recording
        if: always()
        run: |
          dashcam stop
          dashcam publish -p ${{ secrets.DASHCAM_PROJECT_ID }}
```

Reporter auto-detects:
- GitHub repo, branch, commit
- Workflow run ID and URL
- Job ID
- Actor (who triggered)

## Dashcam Association Strategies

### Strategy 1: Session ID (Implemented)
- Set `DASHCAM_SESSION_ID` environment variable
- Reporter includes in test run creation
- Dashboard queries replays by session ID
- Shows all replays from that session

### Strategy 2: Explicit URL (Implemented)
- Dashcam publishes, returns URL
- Pass URL to `recordTestCase()`
- Direct 1:1 link between test and replay

### Strategy 3: Timestamp Matching (Future)
- Parse dashcam logs for replay timestamps
- Match test start/end times with replay markers
- Automatically associate without manual linking
- Allows seeking to exact test within long replay

### Strategy 4: Log Parsing (Future)
- Dashcam logs test names/files during recording
- Parse logs to extract test-to-timestamp mapping
- Generate replay URLs with timestamp seek parameters
- Example: `https://app.dashcam.io/replay/abc123?t=45000` (seek to 45s)

## Dashboard Views (To Be Built)

### Test Runs List
- Table of all test runs
- Columns: Suite Name, Status, Tests (passed/failed), Duration, Date, CI Link
- Filters: Status, Date range, CI provider, Platform
- Search: By suite name, repo, branch

### Test Run Detail
- Header: Suite name, status, duration, platform
- Statistics card: Total/passed/failed/skipped
- Test cases table: Name, Status, Duration, Replay link
- Sidebar: Git info, CI info, Sandbox details
- Dashcam replay player (embedded iframe)
- Click test case → seek replay to that test's time range

### Sandbox Management
- List of active/terminated sandboxes
- Lifecycle timeline visualization
- Cost tracking (duration × instance type)
- Associated test runs

## Environment Variables

| Variable | Required | Description |
|----------|----------|-------------|
| `TD_API_KEY` | Yes | TestDriver API key for authentication |
| `DASHCAM_SESSION_ID` | No | Links test run to dashcam session |
| `TD_SANDBOX_ID` | No | Sandbox ID if running in TestDriver sandbox |
| `GITHUB_ACTIONS` | Auto | Detected for GitHub Actions integration |
| `GITLAB_CI` | Auto | Detected for GitLab CI integration |
| `CIRCLECI` | Auto | Detected for CircleCI integration |

## Future Enhancements

1. **Real-time Test Streaming**
   - WebSocket connection from reporter
   - Live test progress in dashboard
   - See tests pass/fail as they run

2. **Flaky Test Detection**
   - Track test history across runs
   - Identify tests that intermittently fail
   - Suggest fixes based on error patterns

3. **Performance Regression Detection**
   - Compare test durations across runs
   - Alert on significant slowdowns
   - Visualize performance trends

4. **Advanced Dashcam Integration**
   - Automatic timestamp extraction from logs
   - AI-powered test failure analysis from replays
   - Highlight exact moment of failure in replay

5. **Multi-Framework Support**
   - Jest reporter
   - Mocha reporter
   - Playwright reporter
   - Cypress plugin

6. **Cost Analytics**
   - Track sandbox costs per test run
   - Optimize instance types
   - Budget alerts

## Security Considerations

- API keys stored securely (environment variables)
- Bearer token authentication for all API calls
- Team-based access control (tests only visible to team members)
- Replay access control (dashcam's existing permissions)
- No sensitive data in test metadata (sanitize error messages)

## Performance Considerations

- Async test case recording (doesn't slow tests)
- Batch updates for large test suites
- Efficient database indexing (runId, testFile, status)
- Replay association is lazy (doesn't block test recording)
- Optional reporter (disable in local development)
