<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: automating-detection-validation
description: >-
  Build CI/CD pipelines for automated detection validation. Run Atomic Red Team
  tests on schedule, validate Sigma rules on commit, measure MTTD continuously,
  and alert on detection regression. Detection-as-code at scale.
domain: cybersecurity
subdomain: purple-team
tags:
  - detection-as-code
  - ci-cd
  - automation
  - github-actions
  - atomic-red-team
  - sigma
  - regression-testing
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: []
---

# Automating Detection Validation

## Overview

Automated detection validation turns purple team testing from periodic exercises
into continuous assurance. Sigma rules are validated on every commit, atomic tests
run on schedule against lab environments, and MTTD metrics are tracked over time.
Regressions — rules that stop firing — trigger alerts before adversaries exploit gaps.

## Prerequisites

| Requirement | Install |
|---|---|
| GitHub Actions | CI/CD platform (or GitLab CI, Jenkins) |
| sigma-cli | `pip install sigma-cli` |
| Atomic Red Team | Lab environment with test execution capability |
| Python 3.10+ | For orchestration and agent tooling |
| Docker | Containerized test runners |

## Key Concepts

### Pipeline Stages

```
┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
│  Lint    │──▶│ Convert  │──▶│ Unit Test│──▶│ Deploy   │
│  sigma   │   │ to SIEM  │   │ vs data  │   │ to SIEM  │
│  check   │   │ targets  │   │ samples  │   │          │
└──────────┘   └──────────┘   └──────────┘   └──────────┘
                                                   │
┌──────────┐   ┌──────────┐   ┌──────────┐        ▼
│ Report   │◀──│ Measure  │◀──│ Execute  │   ┌──────────┐
│ metrics  │   │ MTTD     │   │ atomics  │◀──│ Schedule │
└──────────┘   └──────────┘   └──────────┘   └──────────┘
```

### GitHub Actions Workflow

```yaml
name: Detection Validation
on:
  push:
    paths: ['rules/**/*.yml']
  schedule:
    - cron: '0 6 * * 1'  # Weekly Monday 6am UTC

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install sigma-cli
        run: pip install sigma-cli pysigma-backend-splunk
      - name: Lint rules
        run: find rules/ -name '*.yml' -exec sigma check {} +
      - name: Convert to Splunk
        run: |
          find rules/ -name '*.yml' | while read rule; do
            sigma convert -t splunk -p splunk_cim "$rule" > /dev/null
          done
      - name: Unit tests
        run: node scripts/agent.js batch-validate --rules-dir rules/ --tests-dir tests/

  atomic-tests:
    runs-on: self-hosted
    if: github.event_name == 'schedule'
    needs: validate
    steps:
      - uses: actions/checkout@v4
      - name: Execute atomic tests
        run: node scripts/agent.js run-scheduled --config schedule.json
      - name: Collect metrics
        run: node scripts/agent.js collect-metrics --output metrics.json
      - name: Upload results
        uses: actions/upload-artifact@v4
        with:
          name: detection-metrics
          path: metrics.json
```

## Workflow

### 1. Setup CI Pipeline

```bash
# Initialize detection-as-code repository
node scripts/agent.js init-pipeline \
  --ci github-actions \
  --siem-targets splunk,elasticsearch \
  --output .github/workflows/detection.yml
```

### 2. Configure Scheduled Tests

```bash
# Create test schedule
node scripts/agent.js create-schedule \
  --techniques T1003.001,T1059.001,T1053.005,T1547.001 \
  --frequency weekly \
  --output schedule.json
```

### 3. Run Validation

```bash
# Validate all rules (CI mode)
node scripts/agent.js batch-validate \
  --rules-dir rules/ \
  --tests-dir tests/ \
  --output validation-results.json

# Run scheduled atomic tests
node scripts/agent.js run-scheduled \
  --config schedule.json \
  --output test-results.json
```

### 4. Monitor for Regressions

```bash
# Compare current results with baseline
node scripts/agent.js check-regression \
  --current test-results.json \
  --baseline baseline-results.json
```

## Verification

| Check | Method |
|---|---|
| Pipeline runs on commit | Push a rule change, CI triggers |
| All rules pass lint | `sigma check` exits 0 for every rule |
| Conversions succeed | All target backends produce output |
| Scheduled tests run | Weekly job completes with results |
| Regressions detected | Previously-passing test that fails triggers alert |

## References

- [GitHub Actions Documentation](https://docs.github.com/en/actions)
- [Detection-as-Code Best Practices](https://github.com/SigmaHQ/sigma)
- [Atomic Red Team CI Integration](https://github.com/redcanaryco/invoke-atomicredteam/wiki)
