<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-security-data-enrichment
description: >-
  Enrich security events with contextual data from threat intelligence feeds,
  geolocation databases, asset inventories, and identity stores to accelerate
  triage and improve detection fidelity.
domain: cybersecurity
subdomain: security-data-engineering
tags:
  - enrichment
  - threat-intelligence
  - geolocation
  - asset-inventory
  - context
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1590", "T1591"]
  frameworks: ["NIST 800-92", "STIX/TAXII"]
  tools: ["python3", "misp", "maxmind", "shodan"]
---

# Implementing Security Data Enrichment

## Overview

Raw security events lack context needed for efficient triage. Enrichment adds
threat intelligence scores, geolocation data, asset criticality, user identity
context, and domain reputation to events at ingestion time or query time,
reducing analyst mean-time-to-triage.

## Prerequisites

| Requirement | Details |
|---|---|
| Threat intelligence feeds (MISP, OTX, VirusTotal) | Required |
| GeoIP database (MaxMind GeoLite2) | Required |
| Asset inventory / CMDB with criticality ratings | Required |
| Identity provider directory (AD, Okta, Entra ID) | Required |

## Key Concepts

- **Inline enrichment**: Add context during stream processing (low latency)
- **Query-time enrichment**: Join at search time (flexible, higher latency)
- **Enrichment cache**: Local cache of TI/geo lookups to reduce API calls
- **Confidence scoring**: Weight enrichment sources by reliability

## Workflow

### Step 1: Configure Enrichment Sources

```python
from dataclasses import dataclass

@dataclass
class EnrichmentSource:
    """Enrichment data source configuration."""
    name: str
    source_type: str  # "threat_intel", "geolocation", "asset", "identity"
    endpoint: str
    cache_ttl_seconds: int = 3600
    confidence: float = 0.8  # source reliability score
```

### Step 2: Build Enrichment Pipeline

```python
def enrich_event(event: dict, sources: list[EnrichmentSource]) -> dict:
    """Enrich a security event with contextual data."""
    enriched = dict(event)
    enrichments: dict[str, dict] = {}
    src_ip = event.get("src_endpoint", {}).get("ip")
    if src_ip:
        for source in sources:
            if source.source_type == "threat_intel":
                enrichments["threat_intel"] = lookup_ti(source, src_ip)
            elif source.source_type == "geolocation":
                enrichments["geo"] = lookup_geo(source, src_ip)
    enriched["enrichments"] = enrichments
    return enriched
```

### Step 3: Implement Caching Layer

```python
from functools import lru_cache

@lru_cache(maxsize=10000)
def cached_geo_lookup(ip: str) -> dict:
    """Cached GeoIP lookup to reduce external API calls."""
    return {"country": "US", "city": "Unknown", "asn": 0}
```

### Step 4: Validate Enrichment Coverage

```bash
node agent.js enrich --input events.jsonl --sources ti,geo,asset --output enriched.jsonl
node agent.js coverage --input enriched.jsonl --report enrichment-gaps
```

## Detection

```yaml
title: Security Data Enrichment Detection
id: c289d792-a15f-47bf-95c6-866fe2f60d30
status: experimental
description: Detects suspicious activity related to implementing security data enrichment techniques in security data engineering context
logsource:
  category: process_creation
  product: linux
detection:
  selection:
    CommandLine: "*implementing*security*"
  condition: selection
level: medium
tags:
  - attack.t1590
  - attack.t1591
  - attack.collection
falsepositives:
  - Data pipeline ETL processes during scheduled batch windows
```


**Detection Opportunities**

| Indicator | Source | Detection Logic |
|---|---|---|
| Security Data Enrichment Detection | linux/process_creation | Sigma rule (medium) |
| ATT&CK Coverage | MITRE ATT&CK | T1590, T1591 |

## References

- [MISP Project](https://www.misp-project.org/) — Threat intelligence platform
- [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)
- [STIX/TAXII](https://oasis-open.github.io/cti-documentation/)
