<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->
# Log Parsing Pipelines — API Reference

## Common Log Formats

### Syslog (RFC 5424)
```
<priority>version timestamp hostname app-name procid msgid structured-data msg
```

### Auth.log (Debian/Ubuntu)
```
Mar 17 08:53:57 hostname sshd[12345]: Failed password for root from 10.0.0.1 port 22 ssh2
```

### Apache Combined
```
127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://example.com" "Mozilla/4.0"
```

### Nginx JSON
```json
{"time": "2024-01-01T00:00:00+00:00", "remote_addr": "1.2.3.4", "request": "GET / HTTP/1.1", "status": 200}
```

## Python Regex Patterns

```python
import re

# IP address
IP = re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b')

# Email address
EMAIL = re.compile(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b')

# MD5 hash
MD5 = re.compile(r'\b[a-fA-F0-9]{32}\b')

# SHA256 hash
SHA256 = re.compile(r'\b[a-fA-F0-9]{64}\b')

# URL
URL = re.compile(r'https?://[^\s<>"{}|\\^`\[\]]+')

# Domain
DOMAIN = re.compile(r'\b(?:[a-zA-Z0-9-]+\.)+[a-zA-Z]{2,}\b')

# Timestamp ISO 8601
ISO_TS = re.compile(r'\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?')
```

## CLI Log Analysis Commands

```bash
# Count failed SSH logins by IP
grep "Failed password" /var/log/auth.log | grep -oP 'from \K[\d.]+' | sort | uniq -c | sort -rn | head -20

# Extract unique IPs from any log
grep -oP '(?:\d{1,3}\.){3}\d{1,3}' logfile | sort -u

# Top HTTP status codes
awk '{print $9}' access.log | sort | uniq -c | sort -rn

# Request rate per minute
awk '{print $4}' access.log | cut -d: -f1-3 | sort | uniq -c | sort -rn | head

# 4xx/5xx errors
awk '$9 >= 400' access.log | head -50

# JSON log parsing with jq
cat json.log | jq -r 'select(.status >= 400) | [.time, .remote_addr, .status, .request] | @tsv'

# Journalctl structured output
journalctl -u sshd --since "1 hour ago" -o json | jq '.MESSAGE'
```

## Python datetime Parsing

```python
from datetime import datetime

# Syslog timestamp (no year)
ts = datetime.strptime("Mar 17 08:53:57", "%b %d %H:%M:%S").replace(year=2026)

# Apache timestamp
ts = datetime.strptime("10/Oct/2000:13:55:36 -0700", "%d/%b/%Y:%H:%M:%S %z")

# ISO 8601
ts = datetime.fromisoformat("2024-01-01T00:00:00+00:00")
```

## Exit Codes

| Code | Meaning |
|------|---------|
| 0 | Parsing completed successfully |
| 1 | Security events detected (brute force, anomalies) |
| 2 | Parse error or file not found |
