<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-security-data-governance
description: >-
  Implement security data governance programs covering data classification,
  lineage tracking, retention policies, access controls, privacy compliance,
  and quality ownership for security telemetry and analytics data.
domain: cybersecurity
subdomain: security-data-engineering
tags:
  - data-governance
  - classification
  - lineage
  - retention
  - privacy
  - compliance
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  frameworks: ["NIST 800-92", "GDPR", "CCPA", "CIS Controls v8"]
  tools: ["python3", "apache-atlas", "datahub"]
  mitre-attack: []
---

# Implementing Security Data Governance

## Overview

Security data governance ensures that telemetry, detection artifacts, and
analytics data are classified, tracked, retained appropriately, and accessed
only by authorized roles. This skill covers classification taxonomy, lineage
tracking from source to detection, retention policy enforcement, PII handling
in security logs, and quality ownership assignment.

## Prerequisites

| Requirement | Details |
|---|---|
| Data catalog platform (Apache Atlas, DataHub, Amundsen) | Required |
| Understanding of data classification requirements | Required |
| Regulatory compliance mapping (GDPR, CCPA, HIPAA) | Required |
| RACI matrix for security data ownership | Required |

## Key Concepts

- **Data classification**: Label data by sensitivity (Public, Internal, Confidential, Restricted)
- **Data lineage**: Track transformations from raw log to detection output
- **Retention policy**: Define keep/archive/delete timelines per data class
- **PII in logs**: Identify and mask/pseudonymize personal data in telemetry
- **Quality ownership**: Assign accountability for data freshness and accuracy

## Workflow

### Step 1: Define Classification Taxonomy

```python
from dataclasses import dataclass
from enum import Enum

class DataClassification(Enum):
    PUBLIC = "public"
    INTERNAL = "internal"
    CONFIDENTIAL = "confidential"
    RESTRICTED = "restricted"

@dataclass
class GovernancePolicy:
    """Data governance policy for a security data source."""
    source_name: str
    classification: DataClassification
    retention_days: int
    pii_fields: list[str]
    owner: str
    review_frequency: str  # "monthly", "quarterly"
```

### Step 2: Map Data Lineage

```yaml
lineage:
  - source: "crowdstrike_raw"
    transforms:
      - step: "normalization"
        output: "ocsf_process_activity"
        pipeline: "cs-normalizer-v2"
      - step: "enrichment"
        output: "enriched_process_activity"
        pipeline: "ti-enrichment"
      - step: "detection"
        output: "alerts"
        rules: ["sigma/process_injection.yml"]
    retention:
      raw: 90
      normalized: 365
      alerts: 730
```

### Step 3: Implement PII Controls

```python
import re

PII_PATTERNS = {
    "email": re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}"),
    "ssn": re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
    "ip_internal": re.compile(r"\b10\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"),
}

def scan_for_pii(event: dict) -> list[str]:
    """Scan event fields for PII patterns."""
    found: list[str] = []
    for field_name, value in event.items():
        if not isinstance(value, str):
            continue
        for pii_type, pattern in PII_PATTERNS.items():
            if pattern.search(value):
                found.append(f"{field_name}:{pii_type}")
    return found
```

### Step 4: Audit Governance Compliance

```bash
node agent.js classify --source auth_events --output classification.json
node agent.js audit --policies governance.yml --report compliance
```

## References

- [Apache Atlas](https://atlas.apache.org/) — Data governance and metadata
- [DataHub](https://datahubproject.io/) — Data catalog
- [GDPR Article 30](https://gdpr-info.eu/art-30-gdpr/) — Records of processing
- [NIST SP 800-92](https://csrc.nist.gov/publications/detail/sp/800-92/final)
