<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-continuous-validation-platform
description: >-
  Deploy and operate continuous security validation platforms that automate
  ongoing detection testing. Covers integration of Atomic Red Team, Caldera,
  and custom test harnesses into CI/CD pipelines, scheduling recurring
  validation runs, and tracking detection health metrics over time.
domain: cybersecurity
subdomain: purple-team
tags:
  - continuous-validation
  - breach-attack-simulation
  - caldera
  - automation
  - ci-cd
  - detection-health
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: []
---

# Implementing Continuous Validation Platform

## Overview

Continuous validation moves purple team testing from periodic exercises to automated,
recurring detection assurance. The platform executes atomic tests on schedules,
compares results against expected detections, alerts on regressions, and produces
detection health dashboards that track coverage trends over weeks and months.

## Prerequisites

| Requirement | Install |
|---|---|
| MITRE Caldera | `git clone https://github.com/mitre/caldera.git && pip install -r requirements.txt` |
| Invoke-AtomicRedTeam | PowerShell module |
| Python 3.10+ | For agent tooling |
| cron / systemd timers | For scheduling |
| Docker | For isolated test execution |

## Key Concepts

### Validation Pipeline

```
Schedule → Execute Tests → Collect Telemetry → Check Detections → Report → Alert on Regression
```

### Test Manifest

```yaml
validation_manifest:
  name: "Weekly Detection Validation"
  schedule: "0 2 * * 1"  # Monday 2 AM
  test_suites:
    - name: credential-access
      techniques:
        - id: T1003.001
          test_numbers: [1, 2]
          expected_alert: "LSASS Memory Access"
        - id: T1003.003
          test_numbers: [1]
          expected_alert: "NTDS.dit Access"
    - name: execution
      techniques:
        - id: T1059.001
          test_numbers: [1, 3]
          expected_alert: "Suspicious PowerShell"
  alert_on_regression: true
  notification_channels:
    - type: slack
      webhook: "${SLACK_WEBHOOK_URL}"
    - type: email
      recipients: ["soc-lead@company.com"]
```

### Docker Execution Environment

```dockerfile
FROM mcr.microsoft.com/powershell:latest
RUN pwsh -Command "Install-Module -Name Invoke-AtomicRedTeam -Force"
RUN git clone https://github.com/redcanaryco/atomic-red-team.git /atomics
COPY validation_manifest.yaml /config/
COPY agent.js /agent/
ENTRYPOINT ["python3", "/agent/agent.js", "run-manifest"]
```

## Workflow

### 1. Create Validation Manifest

```bash
node agent.js create-manifest \
  --name "Weekly Validation" \
  --schedule "0 2 * * 1" \
  --techniques T1003.001 T1059.001 T1053.005 \
  --output manifests/weekly.yaml
```

### 2. Deploy Validation Runner

```bash
# Install systemd timer for recurring execution
node agent.js deploy-runner \
  --manifest manifests/weekly.yaml \
  --method systemd \
  --output /etc/systemd/system/

# Or deploy as cron job
node agent.js deploy-runner \
  --manifest manifests/weekly.yaml \
  --method cron
```

### 3. Execute Validation Run

```bash
# Run manifest immediately
node agent.js run-manifest \
  --manifest manifests/weekly.yaml \
  --siem-url https://splunk:8089 \
  --output results/run_$(date +%Y%m%d).json
```

### 4. Check for Regressions

```bash
# Compare current run against baseline
node agent.js check-regression \
  --current results/run_20260115.json \
  --baseline results/baseline.json
```

### 5. Dashboard Update

```bash
# Push results to metrics store
node agent.js update-dashboard \
  --results results/run_20260115.json \
  --metrics-url http://prometheus:9090
```

## Verification

| Check | Method |
|---|---|
| Manifest valid | `node agent.js validate-manifest --manifest <file>` |
| Runner scheduled | `systemctl status detection-validation.timer` |
| Tests execute cleanly | All atomic tests return exit code 0 |
| Detections measured | Each technique has alert status in results |
| Regressions flagged | Notification sent when coverage drops |

## References

- [MITRE Caldera](https://github.com/mitre/caldera)
- [Atomic Red Team](https://github.com/redcanaryco/atomic-red-team)
- [Detection Engineering CI/CD](https://github.com/SigmaHQ/sigma)
