<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-llm-output-guardrails
description: >-
  Implement output guardrails for LLM applications to prevent harmful content,
  PII leakage, code injection, and insecure output handling through filtering,
  classification, and sanitization pipelines.
domain: cybersecurity
subdomain: ai-llm-security
tags:
  - guardrails
  - output-filtering
  - llm-security
  - pii-detection
  - content-safety
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1059", "T1005"]
  owasp-llm: ["LLM02", "LLM06"]
  mitre-atlas: ["AML.T0048.004"]
  tools: ["python3", "presidio", "guardrails-ai", "regex"]
---

# Implementing LLM Output Guardrails

## Overview

LLM output guardrails prevent models from returning harmful, sensitive, or
exploitable content. Without output filtering, model responses may contain PII,
executable code, system prompt leaks, or content that enables downstream injection
when rendered in HTML, SQL, or shell contexts. A defense-in-depth approach combines
regex filters, NER-based PII detection, content classifiers, and context-aware
sanitization.

## Prerequisites

| Tool / Requirement | Details |
|---|---|
| `python3` | Security tooling |
| `presidio-analyzer` | Security tooling |
| `guardrails-ai` | Security tooling |
| `bleach` | Security tooling |
| Understanding of model output rendering context (HTML, API, CLI) | Environment requirement |
| PII taxonomy for the application's data domain | Environment requirement |
| Content policy defining prohibited output categories | Environment requirement |

## Key Concepts

- **Insecure output handling (LLM02)**: Model output rendered as HTML/JS/SQL without sanitization
- **PII leakage (LLM06)**: Model reproducing personal data from training or context
- **Output classifier**: ML model scoring output for policy violations
- **Context-aware sanitization**: Escaping output based on rendering context
- **Structured output enforcement**: Constraining model to JSON schema to reduce free-text risk

## Workflow

### Step 1: PII Detection Pipeline

```python
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def redact_pii(text: str, score_threshold: float = 0.7) -> dict:
    results = analyzer.analyze(
        text=text, language="en", score_threshold=score_threshold,
        entities=["PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER",
                  "CREDIT_CARD", "US_SSN", "IP_ADDRESS"],
    )
    anonymized = anonymizer.anonymize(text=text, analyzer_results=results)
    return {
        "redacted_text": anonymized.text,
        "entities_found": [
            {"type": r.entity_type, "score": r.score}
            for r in results
        ],
        "pii_detected": len(results) > 0,
    }
```

### Step 2: Context-Aware Sanitization

```python
import html
import shlex

def sanitize_for_context(text: str, context: str = "html") -> str:
    if context == "html":
        return html.escape(text)
    elif context == "sql":
        return text.replace("'", "''").replace(";", "")
    elif context == "shell":
        return shlex.quote(text)
    return text

def detect_code_injection(text: str) -> dict:
    import re
    patterns = {
        "xss": r"<script[^>]*>|javascript:|on\w+=",
        "sql": r"(?:UNION\s+SELECT|DROP\s+TABLE|;\s*DELETE)",
        "shell": r"(?:\$\(|`[^`]+`|\|\s*(?:bash|sh|curl))",
        "ssrf": r"(?:file://|gopher://|http://169\.254)",
    }
    detected = []
    for attack_type, pattern in patterns.items():
        if re.search(pattern, text, re.IGNORECASE):
            detected.append({"type": attack_type})
    return {"has_injection": len(detected) > 0, "detected": detected}
```

### Step 3: Guardrail Pipeline Integration

```python
def guardrail_pipeline(
    model_output: str, context: str = "html", pii_threshold: float = 0.7
) -> dict:
    pii_result = redact_pii(model_output, pii_threshold)
    cleaned = pii_result["redacted_text"]
    injection = detect_code_injection(cleaned)
    if injection["has_injection"]:
        cleaned = sanitize_for_context(cleaned, context)
    return {
        "final_output": cleaned,
        "pii_redacted": pii_result["pii_detected"],
        "injection_sanitized": injection["has_injection"],
        "safe": not pii_result["pii_detected"] and not injection["has_injection"],
    }
```

## Detection

```yaml
title: Llm Output Guardrails Detection
id: 63c5319a-391c-43fe-9d50-5dc9e543b0dd
status: experimental
description: Detects suspicious activity related to implementing llm output guardrails techniques in ai llm security context
logsource:
  category: application
  product: webserver
detection:
  selection:
    EventType: error
  condition: selection
level: medium
tags:
  - attack.t1059
  - attack.t1005
  - attack.initial_access
falsepositives:
  - Automated AI model training and inference pipeline operations
```


**Detection Opportunities**

| Indicator | Source | Detection Logic |
|---|---|---|
| Llm Output Guardrails Detection | webserver/application | Sigma rule (medium) |
| ATT&CK Coverage | MITRE ATT&CK | T1059, T1005 |

## Verification

- [ ] PII detection covers all applicable entity types
- [ ] Content safety classifier tuned with acceptable false positive rate
- [ ] Context-aware sanitization tested for HTML, SQL, shell, JSON
- [ ] Code injection detection covers XSS, SQLi, command injection, SSRF
- [ ] Pipeline latency measured and within SLA
- [ ] Bypass testing performed against guardrail evasion techniques

## References

- [OWASP LLM02 — Insecure Output Handling](https://genai.owasp.org/llmrisk/llm02-insecure-output-handling/)
- [Microsoft Presidio — PII Detection](https://microsoft.github.io/presidio/)
- [Guardrails AI Framework](https://docs.guardrailsai.com/)
- [NIST AI 100-2 — Adversarial Machine Learning](https://csrc.nist.gov/pubs/ai/100/2/e2023/final)
