<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: measuring-detection-engineering-metrics
description: >-
  Define, collect, and report on detection engineering metrics including MTTD,
  detection coverage ratio, rule fidelity, false positive rate, and detection
  maturity scores. Build dashboards for continuous measurement.
domain: cybersecurity
subdomain: purple-team
tags:
  - detection-engineering
  - metrics
  - mttd
  - coverage-ratio
  - false-positive-rate
  - maturity-model
  - kpi
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: []
---

# Measuring Detection Engineering Metrics

## Overview

Detection engineering metrics quantify how well your security monitoring performs.
Key indicators include Mean Time to Detect (MTTD), technique coverage ratio, rule
fidelity, and false positive rates. These metrics drive prioritization, justify
investment, and track improvement over time.

## Prerequisites

| Requirement | Install |
|---|---|
| Python 3.10+ | For metrics collection and agent tooling |
| SIEM API access | Splunk/Elastic/Sentinel for alert data |
| ATT&CK Navigator | Coverage visualization |
| sigma-cli | `pip install sigma-cli` |

## Key Concepts

### Core Metrics

| Metric | Formula | Target |
|---|---|---|
| MTTD | `avg(alert_time - attack_time)` | < 5 min for critical techniques |
| Coverage ratio | `covered_techniques / relevant_techniques` | > 80% for top threat actors |
| Rule fidelity | `true_positives / (true_positives + false_positives)` | > 95% |
| FP rate | `false_positives / total_alerts` | < 5% per rule |
| Detection maturity | Weighted score across all dimensions | Level 3+ on 5-point scale |
| Rule velocity | `new_rules_deployed / time_period` | Trending upward |
| Gap closure rate | `gaps_closed / gaps_identified` per quarter | > 50% |

### Maturity Model

| Level | Name | Description |
|---|---|---|
| 1 | Initial | Ad-hoc rules, no coverage tracking |
| 2 | Managed | Rules mapped to ATT&CK, basic coverage layer |
| 3 | Defined | Automated validation, MTTD measured, FP tracked |
| 4 | Quantitative | Metrics-driven prioritization, CI/CD pipeline |
| 5 | Optimizing | Continuous improvement, threat-informed coverage |

## Workflow

### 1. Collect Alert Data

```bash
# Export alert data from Splunk
curl -s -k -u admin:changeme \
  "https://splunk:8089/services/search/jobs/export" \
  -d 'search=search index=notable | stats count by rule_name, urgency | head 100' \
  -d 'output_mode=json' > alerts.json

# Export from Elasticsearch
curl -s "http://elastic:9200/.alerts-*/_search?size=1000" \
  -H 'Content-Type: application/json' \
  -d '{"query":{"range":{"@timestamp":{"gte":"now-30d"}}}}' > alerts.json
```

### 2. Calculate Metrics

```bash
# Calculate all metrics
node scripts/agent.js calculate \
  --alerts alerts.json \
  --coverage coverage-layer.json \
  --period 30 \
  --output metrics.json

# Calculate MTTD from exercise data
node scripts/agent.js mttd \
  --exercise-results exercise-report.json

# Calculate rule fidelity
node scripts/agent.js fidelity \
  --alerts alerts.json \
  --verified-incidents incidents.json
```

### 3. Generate Dashboard Data

```bash
# Generate time-series metrics for dashboard
node scripts/agent.js dashboard \
  --metrics-history metrics-history/ \
  --output dashboard.json
```

## Verification

| Check | Method |
|---|---|
| MTTD baseline | Calculated from at least 10 validated detections |
| Coverage accurate | Cross-referenced with Navigator layer |
| FP rate measured | Based on analyst triage verdicts, not estimates |
| Trends tracked | At least 3 data points for trend analysis |
| Maturity scored | Each dimension assessed independently |

## References

- [MITRE D3FEND](https://d3fend.mitre.org/)
- [Detection Engineering Maturity Matrix](https://kyle-bailey.medium.com/detection-engineering-maturity-matrix)
- [Palantir Alerting Detection Strategy Framework](https://github.com/palantir/alerting-detection-strategy-framework)
