<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-ai-incident-response
description: >-
  Implement incident response procedures specific to AI/LLM systems including
  prompt injection triage, model compromise containment, data exfiltration
  detection, automated evidence collection, and post-incident analysis with
  model-specific forensic artifacts.
domain: cybersecurity
subdomain: ai-llm-security
tags:
  - incident-response
  - ai-forensics
  - triage
  - containment
  - evidence-collection
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1190", "T1041"]
  owasp-llm: ["LLM01", "LLM02", "LLM06"]
  mitre-atlas: ["AML.T0051", "AML.T0048"]
  tools: ["python3", "json", "datetime", "hashlib"]
---

# Implementing AI Incident Response

## Overview

AI systems require specialized incident response procedures that account for
model-specific attack vectors, non-deterministic behavior, and novel evidence
types. Traditional IR playbooks miss prompt injection artifacts, model behavior
drift, output exfiltration channels, and agent tool abuse. This technique
implements AI-specific triage classification, evidence collection, containment
actions, and post-incident analysis workflows.

## Prerequisites

| Tool / Requirement | Details |
|---|---|
| `python3` | Security tooling |
| `json` | Security tooling |
| `datetime` | Security tooling |
| `hashlib` | Security tooling |
| Incident response plan with AI-specific annexes | Environment requirement |
| Access to LLM application logs, API gateway logs, model telemetry | Environment requirement |
| Communication channels for escalation | Environment requirement |
| Evidence storage with integrity protection | Environment requirement |

## Key Concepts

- **AI incident classification**: Taxonomy of AI-specific incident types
- **Model quarantine**: Isolating a compromised model from production traffic
- **Prompt forensics**: Analyzing prompt/completion logs for attack artifacts
- **Behavioral baseline deviation**: Detecting model behavior changes post-incident
- **Evidence chain of custody**: Preserving model inputs, outputs, and state

## Workflow

### Step 1: AI Incident Classification

```python
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import Any
import hashlib
import json
import uuid


class AIIncidentType(Enum):
    PROMPT_INJECTION = "prompt_injection"
    DATA_EXFILTRATION = "data_exfiltration"
    MODEL_POISONING = "model_poisoning"
    EXCESSIVE_AGENCY = "excessive_agency"
    OUTPUT_MANIPULATION = "output_manipulation"
    MODEL_THEFT = "model_theft"
    TRAINING_DATA_LEAK = "training_data_leak"
    SAFETY_BYPASS = "safety_bypass"


class Severity(Enum):
    CRITICAL = "P1"
    HIGH = "P2"
    MEDIUM = "P3"
    LOW = "P4"


SEVERITY_MAP: dict[AIIncidentType, Severity] = {
    AIIncidentType.DATA_EXFILTRATION: Severity.CRITICAL,
    AIIncidentType.MODEL_POISONING: Severity.CRITICAL,
    AIIncidentType.MODEL_THEFT: Severity.CRITICAL,
    AIIncidentType.PROMPT_INJECTION: Severity.HIGH,
    AIIncidentType.EXCESSIVE_AGENCY: Severity.HIGH,
    AIIncidentType.SAFETY_BYPASS: Severity.HIGH,
    AIIncidentType.TRAINING_DATA_LEAK: Severity.HIGH,
    AIIncidentType.OUTPUT_MANIPULATION: Severity.MEDIUM,
}


@dataclass
class AIIncident:
    incident_id: str
    incident_type: AIIncidentType
    severity: Severity
    title: str
    description: str
    affected_model: str
    reporter: str
    status: str = "open"
    created_at: str = field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )
    evidence: list[dict[str, str]] = field(default_factory=list)
    timeline: list[dict[str, str]] = field(default_factory=list)
    containment_actions: list[str] = field(default_factory=list)

    def to_dict(self) -> dict[str, Any]:
        return {
            "incident_id": self.incident_id,
            "type": self.incident_type.value,
            "severity": self.severity.value,
            "title": self.title,
            "affected_model": self.affected_model,
            "status": self.status,
            "created_at": self.created_at,
            "evidence_count": len(self.evidence),
            "timeline_entries": len(self.timeline),
        }
```

### Step 2: Evidence Collection and Preservation

```python
def collect_evidence(
    evidence_type: str, content: str, source: str, collector: str,
) -> dict[str, Any]:
    evidence_id = str(uuid.uuid4())
    content_hash = hashlib.sha256(content.encode()).hexdigest()
    return {
        "evidence_id": evidence_id,
        "type": evidence_type,
        "source": source,
        "collector": collector,
        "content_hash": content_hash,
        "content_preview": content[:200],
        "collected_at": datetime.now(timezone.utc).isoformat(),
        "chain_of_custody": [
            {"action": "collected", "actor": collector,
             "timestamp": datetime.now(timezone.utc).isoformat()},
        ],
    }


EVIDENCE_TYPES: list[str] = [
    "prompt_log",
    "completion_log",
    "api_gateway_log",
    "model_telemetry",
    "tool_invocation_log",
    "safety_filter_log",
    "user_session_data",
    "model_config_snapshot",
]
```

### Step 3: Containment Actions

```python
CONTAINMENT_PLAYBOOK: dict[str, list[dict[str, str]]] = {
    "prompt_injection": [
        {"action": "Enable enhanced input filtering", "priority": "immediate"},
        {"action": "Increase logging verbosity on affected endpoint", "priority": "immediate"},
        {"action": "Rate limit suspicious source IPs", "priority": "short_term"},
        {"action": "Deploy updated prompt injection classifier", "priority": "short_term"},
    ],
    "data_exfiltration": [
        {"action": "Disable affected model endpoint", "priority": "immediate"},
        {"action": "Revoke API keys associated with exfiltration", "priority": "immediate"},
        {"action": "Enable output content scanning", "priority": "short_term"},
        {"action": "Audit all outputs in affected timeframe", "priority": "short_term"},
    ],
    "model_poisoning": [
        {"action": "Rollback to last known-good model version", "priority": "immediate"},
        {"action": "Quarantine suspected poisoned training data", "priority": "immediate"},
        {"action": "Verify model integrity hashes", "priority": "short_term"},
        {"action": "Re-run evaluation benchmarks", "priority": "short_term"},
    ],
    "excessive_agency": [
        {"action": "Disable agent tool execution", "priority": "immediate"},
        {"action": "Enable human-in-the-loop for all tool calls", "priority": "immediate"},
        {"action": "Audit recent tool invocations", "priority": "short_term"},
        {"action": "Restrict tool permissions to minimum required", "priority": "short_term"},
    ],
}


def get_containment_actions(incident_type: str) -> dict[str, Any]:
    actions = CONTAINMENT_PLAYBOOK.get(incident_type, [])
    return {
        "incident_type": incident_type,
        "total_actions": len(actions),
        "immediate": [a for a in actions if a["priority"] == "immediate"],
        "short_term": [a for a in actions if a["priority"] == "short_term"],
    }
```

### Step 4: Post-Incident Analysis

```python
def generate_post_incident_report(incident: dict[str, Any]) -> dict[str, Any]:
    return {
        "report_type": "AI Incident Post-Mortem",
        "incident_id": incident.get("incident_id", "unknown"),
        "sections": [
            {"name": "Executive Summary", "required": True},
            {"name": "Timeline of Events", "required": True},
            {"name": "Attack Vector Analysis", "required": True},
            {"name": "Impact Assessment", "required": True},
            {"name": "Evidence Summary", "required": True},
            {"name": "Containment Actions Taken", "required": True},
            {"name": "Root Cause Analysis", "required": True},
            {"name": "Detection Gap Analysis", "required": True},
            {"name": "Remediation Plan", "required": True},
            {"name": "Lessons Learned", "required": True},
        ],
        "detection_improvements": [
            "Update prompt injection detection rules",
            "Add behavioral baseline monitoring",
            "Implement output content scanning",
            "Deploy canary tokens in system prompts",
        ],
    }
```

## Detection

```yaml
title: Ai Incident Response Detection
id: c35d2d71-e170-4966-8b35-879dc65d960c
status: experimental
description: Detects suspicious activity related to implementing ai incident response techniques in ai llm security context
logsource:
  category: application
  product: webserver
detection:
  selection:
    EventType: error
  condition: selection
level: medium
tags:
  - attack.t1190
  - attack.t1041
  - attack.initial_access
falsepositives:
  - Automated AI model training and inference pipeline operations
```


**Detection Opportunities**

| Indicator | Source | Detection Logic |
|---|---|---|
| Ai Incident Response Detection | webserver/application | Sigma rule (medium) |
| ATT&CK Coverage | MITRE ATT&CK | T1190, T1041 |

## Verification

- [ ] AI incident classification covers all OWASP LLM Top 10 categories
- [ ] Evidence collection preserves integrity with SHA-256 hashes
- [ ] Containment playbooks exist for each incident type
- [ ] Post-incident report template captures all required sections
- [ ] Escalation procedures tested with tabletop exercise
- [ ] Mean time to contain (MTTC) measured and baselined

## References

- [NIST SP 800-61 — Computer Security Incident Handling Guide](https://csrc.nist.gov/pubs/sp/800/61/r2/final)
- [OWASP LLM Top 10 2025](https://genai.owasp.org/llm-top-10/)
- [MITRE ATLAS — Adversarial Threat Landscape](https://atlas.mitre.org/)
- [AI Incident Database](https://incidentdatabase.ai/)
