<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: detecting-adversarial-example-attacks
description: >-
  Detect adversarial example attacks against ML models including gradient-based
  perturbations, universal adversarial patches, text adversarial attacks using
  character substitution, and transferability-based black-box attacks that
  cause misclassification or safety bypass.
domain: cybersecurity
subdomain: ai-llm-security
tags:
  - adversarial-examples
  - evasion-attacks
  - robustness-testing
  - perturbation-detection
  - model-security
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: ["T1027"]
  owasp-llm: ["LLM01"]
  mitre-atlas: ["AML.T0043", "AML.T0044"]
  tools: ["python3", "numpy", "difflib", "re"]
---

# Detecting Adversarial Example Attacks

## Overview

Adversarial examples are inputs crafted to cause ML models to produce incorrect
outputs while appearing normal to humans. For LLMs, this includes character-level
perturbations (homoglyphs, typosquatting), word-level substitutions that preserve
semantics but evade classifiers, and token-level attacks that exploit tokenizer
behavior. Detection requires multi-layer analysis combining visual similarity
checks, embedding distance monitoring, and statistical anomaly detection.

## Prerequisites

| Tool / Requirement | Details |
|---|---|
| `python3` | Security tooling |
| `numpy` | Security tooling |
| `difflib` | Security tooling |
| `re` | Security tooling |
| Baseline of normal input distributions | Environment requirement |
| Homoglyph and confusable character databases | Environment requirement |
| Reference embeddings for similarity comparison | Environment requirement |

## Key Concepts

- **Homoglyph attack**: Substituting visually similar characters (e.g., Cyrillic 'а' for Latin 'a')
- **Token-level perturbation**: Modifications that change tokenization while preserving visual appearance
- **Universal adversarial trigger**: Short phrases that reliably cause model misbehavior
- **Transferability**: Adversarial examples crafted for one model often work on others
- **Perturbation budget**: Maximum allowed modification (L0/L2/Linf norm) before detection

## Workflow

### Step 1: Homoglyph and Confusable Detection

```python
import re
import unicodedata
from typing import Any

CONFUSABLE_MAP: dict[str, str] = {
    "\u0430": "a", "\u0435": "e", "\u043e": "o", "\u0440": "p",
    "\u0441": "c", "\u0443": "y", "\u0445": "x", "\u0456": "i",
    "\u0455": "s", "\u0460": "O", "\u04bb": "h", "\u0501": "d",
    "\uff41": "a", "\uff42": "b", "\uff43": "c", "\uff44": "d",
    "\u2010": "-", "\u2011": "-", "\u2012": "-", "\u2013": "-",
}


def detect_homoglyphs(text: str) -> dict[str, Any]:
    findings: list[dict[str, Any]] = []
    for i, char in enumerate(text):
        if char in CONFUSABLE_MAP:
            findings.append({
                "position": i,
                "char": char,
                "unicode": f"U+{ord(char):04X}",
                "confusable_with": CONFUSABLE_MAP[char],
                "category": unicodedata.category(char),
            })
    return {
        "total_chars": len(text),
        "homoglyphs_found": len(findings),
        "findings": findings[:20],
        "risk": "HIGH" if len(findings) > 3 else "LOW" if findings else "NONE",
    }
```

### Step 2: Character Perturbation Scoring

```python
import difflib


def score_perturbation(original: str, suspect: str) -> dict[str, Any]:
    visual_ratio = difflib.SequenceMatcher(None, original, suspect).ratio()
    byte_diffs: list[dict[str, Any]] = []
    for i, (a, b) in enumerate(zip(original, suspect)):
        if a != b:
            byte_diffs.append({
                "position": i,
                "original": f"U+{ord(a):04X} ({a})",
                "suspect": f"U+{ord(b):04X} ({b})",
            })
    len_diff = abs(len(original) - len(suspect))
    return {
        "visual_similarity": round(visual_ratio, 4),
        "character_diffs": len(byte_diffs),
        "length_diff": len_diff,
        "diff_details": byte_diffs[:10],
        "is_adversarial": visual_ratio > 0.9 and len(byte_diffs) > 0,
        "confidence": round(visual_ratio * (len(byte_diffs) / max(len(original), 1)), 4),
    }
```

### Step 3: Universal Adversarial Trigger Detection

```python
KNOWN_TRIGGERS: list[str] = [
    "describing.-- Pro",
    ")](/ surely]( certainly",
    "Answer: `json{",
    "\\n\\nHuman:",
    "INST]",
    ">> BEGIN",
]


def detect_triggers(text: str) -> dict[str, Any]:
    matches: list[str] = []
    for trigger in KNOWN_TRIGGERS:
        if trigger.lower() in text.lower():
            matches.append(trigger)
    entropy = _char_entropy(text)
    return {
        "known_triggers_matched": len(matches),
        "matches": matches,
        "char_entropy": round(entropy, 3),
        "low_entropy_anomaly": entropy < 3.0,
        "risk": "CRITICAL" if matches else "LOW",
    }


def _char_entropy(text: str) -> float:
    import math
    from collections import Counter
    if not text:
        return 0.0
    freq = Counter(text)
    length = len(text)
    return -sum((c / length) * math.log2(c / length) for c in freq.values())
```

### Step 4: Embedding Distance Monitoring

```python
import numpy as np


def cosine_similarity(a: list[float], b: list[float]) -> float:
    a_arr = np.array(a)
    b_arr = np.array(b)
    dot = np.dot(a_arr, b_arr)
    norm = np.linalg.norm(a_arr) * np.linalg.norm(b_arr)
    if norm == 0:
        return 0.0
    return float(dot / norm)


def check_embedding_drift(
    original_emb: list[float], suspect_emb: list[float], threshold: float = 0.95,
) -> dict[str, Any]:
    similarity = cosine_similarity(original_emb, suspect_emb)
    return {
        "cosine_similarity": round(similarity, 4),
        "threshold": threshold,
        "drift_detected": similarity < threshold,
        "risk": "HIGH" if similarity < threshold else "NONE",
    }
```

## Detection

```yaml
title: Adversarial Example Attacks Detection
id: 1d4a29dd-dc6a-481b-9d64-b148ca75bc2b
status: experimental
description: Detects suspicious activity related to detecting adversarial example attacks techniques in ai llm security context
logsource:
  category: application
  product: webserver
detection:
  selection:
    EventType: error
  condition: selection
level: medium
tags:
  - attack.t1027
  - attack.initial_access
falsepositives:
  - Automated AI model training and inference pipeline operations
```


**Detection Opportunities**

| Indicator | Source | Detection Logic |
|---|---|---|
| Adversarial Example Attacks Detection | webserver/application | Sigma rule (medium) |
| ATT&CK Coverage | MITRE ATT&CK | T1027 |

## Verification

- [ ] Homoglyph detector catches Cyrillic/fullwidth substitutions
- [ ] Perturbation scorer differentiates adversarial from benign edits
- [ ] Known trigger database updated and matching correctly
- [ ] Embedding drift detection calibrated for target model
- [ ] False positive rate measured on legitimate multilingual input
- [ ] End-to-end pipeline latency under 50ms

## References

- [MITRE ATLAS AML.T0043 — Adversarial ML Evasion](https://atlas.mitre.org/techniques/AML.T0043)
- [NIST AI 100-2 — Adversarial Machine Learning](https://csrc.nist.gov/pubs/ai/100/2/e2023/final)
- [Unicode Confusables — TR39](https://unicode.org/reports/tr39/)
- [Goodfellow et al. — Explaining and Harnessing Adversarial Examples](https://arxiv.org/abs/1412.6572)
