<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: performing-detection-data-modeling
description: >-
  Model security data for detection analytics using entity graphs, session
  stitching, temporal aggregation, and behavioral baselines to support
  advanced threat detection beyond single-event rules.
domain: cybersecurity
subdomain: security-data-engineering
tags:
  - data-modeling
  - entity-graph
  - session-stitching
  - behavioral
  - analytics
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1078", "T1059"]
  frameworks: ["MITRE ATT&CK", "NIST CSF 2.0"]
  tools: ["python3", "duckdb", "networkx", "apache-spark"]
---

# Performing Detection Data Modeling

## Overview

Detection data modeling structures security telemetry for analytics beyond
simple field matching. Entity graphs link users, hosts, and processes across
events. Session stitching reconstructs activity timelines. Behavioral baselines
enable anomaly scoring. This skill covers modeling patterns that power advanced
threat detection.

## Prerequisites

| Requirement | Details |
|---|---|
| Normalized security event data (OCSF or ECS) | Required |
| Graph library (NetworkX) or graph database (Neo4j) | Required |
| SQL analytics engine (DuckDB, Spark SQL) | Required |
| Understanding of entity relationships in security data | Required |

## Key Concepts

- **Entity graph**: Nodes (users, hosts, IPs) with edges (connections, auth, access)
- **Session stitching**: Group events into logical sessions by time and entity
- **Temporal aggregation**: Roll up events into time windows for trend analysis
- **Behavioral baseline**: Statistical profile of normal entity behavior
- **Anomaly score**: Deviation from baseline indicating potential threat

## Workflow

### Step 1: Define Entity Model

```python
from dataclasses import dataclass, field

@dataclass
class SecurityEntity:
    """Security entity for graph modeling."""
    entity_type: str  # "user", "host", "ip", "process"
    entity_id: str
    attributes: dict = field(default_factory=dict)
    first_seen: str | None = None
    last_seen: str | None = None

@dataclass
class EntityRelation:
    """Relationship between security entities."""
    source: SecurityEntity
    target: SecurityEntity
    relation_type: str  # "authenticated_to", "connected_to", "executed"
    event_count: int = 0
    first_seen: str | None = None
```

### Step 2: Build Session Model

```sql
-- Stitch authentication events into sessions
WITH ordered_events AS (
    SELECT *,
        LAG(time_dt) OVER (PARTITION BY actor.user.name, src_endpoint.ip ORDER BY time_dt) AS prev_time
    FROM authentication_events
),
session_boundaries AS (
    SELECT *,
        CASE WHEN prev_time IS NULL
             OR time_dt - prev_time > INTERVAL '30 minutes'
             THEN 1 ELSE 0 END AS new_session
    FROM ordered_events
)
SELECT
    actor.user.name AS username,
    src_endpoint.ip AS source_ip,
    SUM(new_session) OVER (ORDER BY time_dt) AS session_id,
    MIN(time_dt) AS session_start,
    MAX(time_dt) AS session_end,
    COUNT(*) AS event_count
FROM session_boundaries
GROUP BY username, source_ip, session_id;
```

### Step 3: Compute Behavioral Baselines

```python
import statistics

def compute_baseline(entity_id: str, metric_values: list[float]) -> dict:
    """Compute statistical baseline for entity behavior."""
    if len(metric_values) < 7:
        return {"entity": entity_id, "status": "insufficient_data"}
    return {
        "entity": entity_id,
        "mean": statistics.mean(metric_values),
        "stdev": statistics.stdev(metric_values),
        "p95": sorted(metric_values)[int(len(metric_values) * 0.95)],
        "sample_size": len(metric_values),
    }
```

### Step 4: Generate Detection Models

```bash
node agent.js model --input events.jsonl --type entity-graph --output model.json
node agent.js baseline --entity-type user --metric auth_count --window 30d
```

## Detection

```yaml
title: Detection Data Modeling Detection
id: 2d6ffe7b-b015-4e52-bf65-0a2f3cd9a581
status: experimental
description: Detects suspicious activity related to performing detection data modeling techniques in security data engineering context
logsource:
  category: process_creation
  product: linux
detection:
  selection:
    CommandLine: "*performing*detection*"
  condition: selection
level: medium
tags:
  - attack.t1078
  - attack.t1059
  - attack.collection
falsepositives:
  - Data pipeline ETL processes during scheduled batch windows
```


**Detection Opportunities**

| Indicator | Source | Detection Logic |
|---|---|---|
| Detection Data Modeling Detection | linux/process_creation | Sigma rule (medium) |
| ATT&CK Coverage | MITRE ATT&CK | T1078, T1059 |

## References

- [MITRE ATT&CK](https://attack.mitre.org/)
- [NetworkX](https://networkx.org/) — Graph library
- [Apache Spark GraphX](https://spark.apache.org/graphx/)
