<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->

# API Reference — LLM Input Sanitization

## Unicode Normalization
```python
import unicodedata

text = unicodedata.normalize("NFKC", user_input)
```

## Invisible Character Detection
```python
import re

INVISIBLE = re.compile(r"[\u200b-\u200f\u2028-\u202f\u2060-\u2069\ufeff]")
cleaned = INVISIBLE.sub("", text)
```

## Token Budget Enforcement
```python
import tiktoken

enc = tiktoken.get_encoding("cl100k_base")
tokens = enc.encode(text)
if len(tokens) > 4096:
    text = enc.decode(tokens[:4096])
```

## Delimiter Injection Detection
```python
import re

DELIMITERS = re.compile(r"<\|?(system|assistant|user)\|?>", re.IGNORECASE)
if DELIMITERS.search(user_input):
    print("ALERT: Delimiter injection attempt")
```

## Base64 Payload Detection
```python
import base64, re

for match in re.finditer(r"[A-Za-z0-9+/]{20,}={0,2}", text):
    try:
        decoded = base64.b64decode(match.group()).decode("utf-8", errors="ignore")
        print(f"Decoded payload: {decoded[:100]}")
    except Exception:
        pass
```
