<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: building-security-metrics-pipelines
description: >-
  Build automated pipelines that compute security KPIs including MTTD, MTTR,
  detection coverage, false positive rates, and log source health from
  security event data for executive and operational reporting.
domain: cybersecurity
subdomain: security-data-engineering
tags:
  - metrics
  - kpi
  - mttd
  - mttr
  - dashboards
  - reporting
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  frameworks: ["NIST CSF 2.0", "CIS Controls v8"]
  tools: ["python3", "prometheus", "grafana", "duckdb"]
  mitre-attack: []
---

# Building Security Metrics Pipelines

## Overview

Security metrics pipelines transform raw event and incident data into
quantifiable KPIs that track program effectiveness. This skill covers defining
metric taxonomy, building ETL pipelines, computing MTTD/MTTR, measuring
detection coverage, and generating dashboards for operational and executive
audiences.

## Prerequisites

| Requirement | Details |
|---|---|
| Incident tracking system (Jira, ServiceNow, PagerDuty) | Required |
| Security event data with timestamps and severity | Required |
| Detection rule inventory with ATT&CK mappings | Required |
| Visualization platform (Grafana, Kibana, Looker) | Required |

## Key Concepts

- **MTTD (Mean Time to Detect)**: Average time from threat activity to alert
- **MTTR (Mean Time to Respond)**: Average time from alert to containment
- **Detection coverage ratio**: ATT&CK techniques with active detections
- **False positive rate**: Alerts closed as benign vs total alerts
- **Log source health**: Availability and freshness of telemetry sources

## Workflow

### Step 1: Define Metric Taxonomy

```python
from dataclasses import dataclass
from enum import Enum

class MetricCategory(Enum):
    DETECTION = "detection"
    RESPONSE = "response"
    COVERAGE = "coverage"
    OPERATIONAL = "operational"

@dataclass
class SecurityMetric:
    """Security KPI definition."""
    name: str
    category: MetricCategory
    query: str
    unit: str  # "hours", "percent", "count"
    target: float
    frequency: str  # "hourly", "daily", "weekly"
```

### Step 2: Compute Core Metrics

```sql
-- MTTD: Mean Time to Detect
SELECT
    AVG(EXTRACT(EPOCH FROM (alert_time - activity_time)) / 3600) AS mttd_hours
FROM incidents
WHERE created_at >= CURRENT_DATE - INTERVAL '30 days'
  AND alert_time IS NOT NULL
  AND activity_time IS NOT NULL;

-- False Positive Rate
SELECT
    ROUND(
        COUNT(*) FILTER (WHERE disposition = 'false_positive') * 100.0 / COUNT(*),
        2
    ) AS fp_rate_pct
FROM alerts
WHERE created_at >= CURRENT_DATE - INTERVAL '30 days';
```

### Step 3: Build Export Pipeline

```bash
node agent.js compute --metrics mttd,mttr,fp_rate --window 30d --output metrics.json
node agent.js dashboard --input metrics.json --format grafana
```

## References

- [NIST CSF 2.0](https://www.nist.gov/cyberframework)
- [SANS Security Metrics](https://www.sans.org/white-papers/)
- [Grafana Dashboards](https://grafana.com/docs/grafana/latest/)
