<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->

# Secure Infrastructure Deep Dive

> NERF Training Module — Network Architecture, Segmentation, Firewalling, VPN, and Infrastructure Hardening
>
> Sources: OWASP Cheat Sheets, NIST 800-series, CIS Benchmarks, WireGuard whitepaper, fail2ban/CrowdSec docs, nginx-admins-handbook, trimstray/iptables-essentials, imthenachoman/How-To-Secure-A-Linux-Server, awesome-security-hardening

---

## 1. Network Segmentation Architecture

### 1.1 The Three-Tier Model (OWASP)

Network segmentation prevents lateral movement by enforcing trust boundaries between functional zones. The OWASP Network Segmentation Cheat Sheet defines three primary tiers:

| Tier | Contents | Trust Level |
|------|----------|-------------|
| **FRONTEND** | Load balancers, WAF, web servers, caches | Low — Internet-facing |
| **MIDDLEWARE** | Application logic, auth services, message queues, analytics, stream processing | Medium — internal only |
| **BACKEND** | SQL databases, LDAP/AD, key stores, file servers | High — most restricted |

**Permitted flows:**
- FRONTEND --> MIDDLEWARE (same service only)
- MIDDLEWARE --> BACKEND (same service only)
- FRONTEND/MIDDLEWARE --> external networks (outbound, controlled)

**Prohibited flows:**
- Cross-service FRONTEND-to-MIDDLEWARE access (service A frontend cannot reach service B middleware)
- MIDDLEWARE access to foreign BACKEND segments (prevents database bypass across service boundaries)
- Direct FRONTEND --> BACKEND (no skipping the middleware tier)

### 1.2 DMZ Architecture

Two-DMZ design separating inbound and outbound exposure:

```
                    ┌─────────────────┐
    Internet ───────┤  Edge Firewall   ├───────┐
                    └────────┬────────┘        │
                             │                 │
                    ┌────────▼────────┐  ┌─────▼──────┐
                    │  DMZ Inbound    │  │ DMZ Outbound│
                    │  (Web, Mail,    │  │ (Proxies,   │
                    │   Public APIs)  │  │  outbound   │
                    │  + WAF          │  │  services)  │
                    └────────┬────────┘  └─────┬──────┘
                             │                 │
                    ┌────────▼─────────────────▼──┐
                    │     Internal Firewall        │
                    └────┬────┬────┬────┬─────────┘
                         │    │    │    │
                    ┌────▼┐┌──▼─┐┌─▼──┐┌▼────────┐
                    │APPS ││ DB ││ AD ││  LOGS    │
                    │VLAN ││VLAN││VLAN││  VLAN    │
                    └─────┘└────┘└────┘└─────────┘
```

- **DMZ Inbound**: Services accessible from the Internet. Must have WAF protection.
- **DMZ Outbound**: Services that initiate external connections but accept no inbound Internet traffic.
- **Internal VLANs**: Separated by function (applications, databases, AD/directory services, logging).

### 1.3 VLAN Segmentation

VLANs (802.1Q) provide Layer 2 isolation. Key design principles:

- **One function per VLAN** — do not mix databases and application servers on the same VLAN.
- **Inter-VLAN routing through a firewall** — never use a Layer 3 switch for inter-VLAN routing without ACLs. All inter-VLAN traffic must traverse a stateful firewall or router with explicit permit rules.
- **Management VLAN isolation** — IPMI/iLO/iDRAC, switch management, and hypervisor management interfaces belong on a dedicated management VLAN with no user access.
- **VLAN hopping prevention**:
  - Disable DTP (Dynamic Trunking Protocol) on all access ports.
  - Set native VLAN to an unused VLAN ID (not VLAN 1).
  - Explicitly configure all ports as access or trunk (no auto-negotiation).
  - Prune VLANs from trunks — only allow VLANs that are needed on each trunk link.

### 1.4 Microsegmentation

Microsegmentation operates at Layer 7 / workload level, enforcing policy between individual hosts or containers rather than subnets.

**Implementation approaches:**
- **Host-based firewalls** (iptables/nftables per host with centralized policy management)
- **Service mesh** (Istio, Linkerd — mTLS between all services, authorization policies per endpoint)
- **Software-defined networking** (VMware NSX, Calico for Kubernetes — policy attached to workload identity, not IP)
- **Zero Trust Network Access (ZTNA)** — every connection is authenticated and authorized regardless of network location

**Microsegmentation rules for containerized environments:**
```
# Kubernetes NetworkPolicy example — deny all, then allow specific
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-api
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: api-server
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080
```

### 1.5 Segmentation for Specific Use Cases

**CI/CD Pipeline Isolation:**
- CI/CD systems get a dedicated VLAN with firewall-controlled access to code repositories, artifact stores, and deployment targets.
- Build agents should not have direct database access.
- Secrets injection happens at deploy time from a vault on a separate segment.

**Centralized Logging:**
- Log collectors on a dedicated VLAN.
- **Write-only access pattern**: sources push logs via syslog/TCP or agent; log VLAN accepts inbound syslog but initiates no outbound connections to source VLANs.
- SIEM/analytics on the log VLAN can pull from the log store but cannot reach production VLANs.

**Monitoring (Zabbix/Prometheus):**
- Monitoring VLAN can poll agents across VLANs (specific ports only: 10050/TCP for Zabbix, 9090/TCP for Prometheus).
- Agents should not accept arbitrary connections — only from the monitoring server IP.

---

## 2. Firewall Rule Design

### 2.1 iptables Fundamentals

iptables operates on tables (filter, nat, mangle, raw) and chains (INPUT, OUTPUT, FORWARD, plus user-defined chains).

**Core principle: Default deny, explicit allow.**

```bash
# Set default policies — DROP everything
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# Allow established/related connections (stateful tracking)
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Drop invalid packets
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow SSH from management subnet only
iptables -A INPUT -p tcp -s 10.0.100.0/24 --dport 22 -j ACCEPT

# Allow HTTP/HTTPS inbound
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Allow DNS outbound
iptables -A OUTPUT -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -p tcp --dport 53 -j ACCEPT

# Allow NTP outbound
iptables -A OUTPUT -p udp --dport 123 -j ACCEPT

# Allow HTTPS outbound (for package updates)
iptables -A OUTPUT -p tcp --dport 443 -j ACCEPT
```

### 2.2 Anti-DDoS and Attack Mitigation Rules

```bash
# SYN flood protection with SYN cookies (kernel-level)
sysctl -w net.ipv4.tcp_syncookies=1

# Rate limit new SSH connections (max 4 per 60 seconds per source IP)
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW \
  -m recent --set --name SSH
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW \
  -m recent --update --seconds 60 --hitcount 4 --name SSH -j DROP

# Drop bogus TCP flag combinations
iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP
iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP
iptables -A INPUT -p tcp --tcp-flags ALL FIN,PSH,URG -j DROP
iptables -A INPUT -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
iptables -A INPUT -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
iptables -A INPUT -p tcp --tcp-flags FIN,RST FIN,RST -j DROP

# Drop packets with uncommon MSS values (scanner fingerprint)
iptables -t mangle -A PREROUTING -p tcp -m conntrack --ctstate NEW \
  -m tcpmss ! --mss 536:65535 -j DROP

# Block port scanning (detect sequential port probes)
iptables -N port-scanning
iptables -A port-scanning -p tcp --tcp-flags SYN,ACK,FIN,RST RST \
  -m limit --limit 1/s --limit-burst 2 -j RETURN
iptables -A port-scanning -j DROP
```

### 2.3 Kernel Hardening (sysctl) for Network Stack

```bash
# /etc/sysctl.d/99-network-hardening.conf

# Reverse path filtering — drop packets with spoofed source IPs
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Log martian packets (impossible source addresses)
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1

# Disable ICMP redirects (prevents route table poisoning)
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0

# Disable source routing (attacker-controlled path selection)
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0

# SYN flood protection
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 5

# Ignore broadcast pings (Smurf attack mitigation)
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Ignore bogus ICMP error responses
net.ipv4.icmp_ignore_bogus_error_responses = 1

# Disable IPv4 forwarding (unless this is a router/firewall)
net.ipv4.ip_forward = 0

# Harden TCP timestamps (information leakage reduction)
net.ipv4.tcp_timestamps = 0
```

### 2.4 nftables (iptables Successor)

nftables is the modern Linux firewall framework, replacing iptables with a cleaner syntax and better performance.

```bash
#!/usr/sbin/nft -f

flush ruleset

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # Connection tracking
        ct state established,related accept
        ct state invalid drop

        # Loopback
        iifname "lo" accept

        # ICMP (limited)
        ip protocol icmp icmp type { echo-request } \
            limit rate 5/second accept

        # SSH from management only
        ip saddr 10.0.100.0/24 tcp dport 22 accept

        # HTTP/HTTPS
        tcp dport { 80, 443 } accept

        # Log and drop everything else
        log prefix "[nftables-drop] " counter drop
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
    }

    chain output {
        type filter hook output priority 0; policy drop;

        ct state established,related accept
        oifname "lo" accept

        # DNS, NTP, HTTPS outbound
        udp dport { 53, 123 } accept
        tcp dport { 53, 443, 80 } accept

        log prefix "[nftables-out-drop] " counter drop
    }
}
```

### 2.5 UFW (Uncomplicated Firewall)

UFW wraps iptables for simpler management. Suitable for single-host hardening:

```bash
# Default deny
ufw default deny incoming
ufw default deny outgoing

# Allow SSH from management subnet
ufw allow from 10.0.100.0/24 to any port 22 proto tcp

# Allow HTTP/HTTPS inbound
ufw allow in 80/tcp
ufw allow in 443/tcp

# Allow DNS, NTP, HTTPS outbound
ufw allow out 53
ufw allow out 123/udp
ufw allow out 443/tcp
ufw allow out 80/tcp

# Rate limit SSH (auto brute-force protection)
ufw limit 22/tcp

# Enable
ufw enable
```

### 2.6 Persistence

```bash
# Debian/Ubuntu — iptables
apt install iptables-persistent
netfilter-persistent save

# RHEL/CentOS — iptables
service iptables save

# nftables — config already in /etc/nftables.conf
systemctl enable nftables

# UFW — persists automatically when enabled
```

---

## 3. Reverse Proxy Hardening (NGINX)

### 3.1 SSL/TLS Configuration

```nginx
# /etc/nginx/conf.d/ssl-hardening.conf

# TLS 1.2 and 1.3 only — no SSLv3, TLS 1.0, TLS 1.1
ssl_protocols TLSv1.2 TLSv1.3;

# Strong cipher suites (AEAD only for TLS 1.2)
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305';

# TLS 1.3 cipher suites (set separately)
ssl_conf_command Ciphersuites TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256;

# Server cipher preference
ssl_prefer_server_ciphers on;

# ECDH curve
ssl_ecdh_curve X25519:secp384r1;

# DH parameters (generate: openssl dhparam -out /etc/nginx/dhparam.pem 4096)
ssl_dhparam /etc/nginx/dhparam.pem;

# Session settings
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;  # Disable for perfect forward secrecy

# OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;
resolver 1.1.1.1 8.8.8.8 valid=300s;
resolver_timeout 5s;

# Certificate files
ssl_certificate /etc/nginx/certs/fullchain.pem;
ssl_certificate_key /etc/nginx/certs/privkey.pem;
ssl_trusted_certificate /etc/nginx/certs/chain.pem;
```

### 3.2 Security Headers

```nginx
# /etc/nginx/conf.d/security-headers.conf

# HSTS — force HTTPS for 1 year, include subdomains, allow preload list
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;

# Prevent MIME type sniffing
add_header X-Content-Type-Options "nosniff" always;

# Clickjacking protection
add_header X-Frame-Options "DENY" always;

# XSS filter (legacy browsers)
add_header X-XSS-Protection "1; mode=block" always;

# Referrer policy
add_header Referrer-Policy "strict-origin-when-cross-origin" always;

# Permissions policy (restrict browser features)
add_header Permissions-Policy "camera=(), microphone=(), geolocation=(), payment=()" always;

# Content Security Policy (customize per application)
add_header Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self'; frame-ancestors 'none'; base-uri 'self'; form-action 'self';" always;

# Remove server version disclosure
server_tokens off;
more_clear_headers Server;
```

### 3.3 Rate Limiting and Connection Controls

```nginx
# Define rate limit zones
limit_req_zone $binary_remote_addr zone=general:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
limit_req_zone $binary_remote_addr zone=api:10m rate=30r/s;

# Connection limits
limit_conn_zone $binary_remote_addr zone=addr:10m;

server {
    # Apply rate limits
    location / {
        limit_req zone=general burst=20 nodelay;
        limit_conn addr 10;
    }

    location /login {
        limit_req zone=login burst=3 nodelay;
        limit_conn addr 2;
    }

    location /api/ {
        limit_req zone=api burst=50 nodelay;
        limit_conn addr 20;
    }

    # Custom error pages for rate limiting (don't leak info)
    error_page 429 /429.html;
    error_page 503 /503.html;
}
```

### 3.4 Reverse Proxy Security

```nginx
server {
    listen 443 ssl http2;
    server_name app.example.com;

    # Block access to hidden files (.git, .env, .htaccess)
    location ~ /\. {
        deny all;
        return 404;
    }

    # Block common backup/config file extensions
    location ~* \.(bak|config|sql|fla|psd|ini|log|sh|inc|swp|dist|old|orig)$ {
        deny all;
        return 404;
    }

    # Proxy to backend
    location / {
        proxy_pass http://backend_upstream;

        # Pass original client info
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Prevent host header attacks
        proxy_set_header Host $host;

        # Timeouts (prevent slowloris)
        proxy_connect_timeout 5s;
        proxy_send_timeout 10s;
        proxy_read_timeout 30s;

        # Buffer limits
        proxy_buffering on;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;

        # Max request body size
        client_max_body_size 10m;
    }

    # Strip sensitive headers from upstream responses
    proxy_hide_header X-Powered-By;
    proxy_hide_header X-AspNet-Version;
    proxy_hide_header X-Runtime;
}

# Catch-all server block — reject requests with undefined Host header
server {
    listen 80 default_server;
    listen 443 ssl default_server;
    server_name _;
    ssl_certificate /etc/nginx/certs/default.pem;
    ssl_certificate_key /etc/nginx/certs/default-key.pem;
    return 444;  # Close connection without response
}
```

### 3.5 WAF Integration

**ModSecurity with NGINX:**
```nginx
# Load ModSecurity module
load_module modules/ngx_http_modsecurity_module.so;

http {
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;

    # Use OWASP Core Rule Set (CRS)
    # Include /etc/nginx/modsecurity/crs/crs-setup.conf
    # Include /etc/nginx/modsecurity/crs/rules/*.conf
}
```

**NAXSI (lightweight alternative):**
```nginx
# Basic NAXSI rules — deny SQL injection and XSS patterns
location / {
    SecRulesEnabled;
    DeniedUrl "/RequestDenied";
    CheckRule "$SQL >= 8" BLOCK;
    CheckRule "$RFI >= 8" BLOCK;
    CheckRule "$TRAVERSAL >= 4" BLOCK;
    CheckRule "$EVADE >= 4" BLOCK;
    CheckRule "$XSS >= 8" BLOCK;
}
```

### 3.6 Web Server Backdoor Awareness (htshells)

Defenders must monitor for `.htaccess`-based backdoors that enable:

| Technique | Method | Detection |
|-----------|--------|-----------|
| PHP shell via .htaccess | `AddHandler` / `AddType` to execute arbitrary extensions as PHP | Alert on .htaccess creation/modification; audit `AddHandler` directives |
| SSI command execution | Server-Side Include directives in .htaccess | Disable `mod_include` if unused |
| CGI handler abuse | Bind bash/python as CGI handler for file extensions | Monitor for unexpected CGI registrations |
| Credential forwarding | Redirect basic auth credentials to attacker server | Review `AuthBasicProvider` and `RewriteRule` in .htaccess |
| Info disclosure | Bind `mod_status`, `mod_info` to accessible paths | Restrict status pages to localhost only |
| Directory traversal | Abuse `mod_hitlog`, `mod_layout` to read `/etc/passwd` | Disable unused modules; file integrity monitoring |

**Prevention:**
- Set `AllowOverride None` globally; only enable specific directives where needed.
- Disable unused Apache modules (`mod_include`, `mod_cgi`, `mod_status`).
- Monitor `.htaccess` files with file integrity monitoring (AIDE, OSSEC, Wazuh).
- Block direct access to `.htaccess` and `.htpasswd` in NGINX/Apache config.

---

## 4. VPN Architecture

### 4.1 WireGuard

WireGuard is a modern VPN protocol designed around minimal attack surface and cryptographic simplicity.

**Cryptographic primitives:**
- **Key exchange**: Noise protocol framework with Curve25519 (ECDH)
- **Symmetric encryption**: ChaCha20-Poly1305 (AEAD)
- **Hashing**: BLAKE2s
- **Key derivation**: HKDF

**Design philosophy:**
- ~4,000 lines of kernel code (vs. ~100,000 for OpenVPN + OpenSSL, ~400,000 for IPsec)
- No cipher agility — single, modern cryptographic construction. This eliminates downgrade attacks and negotiation complexity.
- Operates entirely in kernel space (Linux) for performance.
- Stealth by default — sends no response to unauthenticated packets (no port scanning fingerprint).

**Configuration:**
```ini
# /etc/wireguard/wg0.conf — Server

[Interface]
Address = 10.0.0.1/24
ListenPort = 51820
PrivateKey = <server_private_key>

# Firewall rules applied on tunnel up/down
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; \
         iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; \
           iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
# Client 1
PublicKey = <client1_public_key>
AllowedIPs = 10.0.0.2/32
```

```ini
# /etc/wireguard/wg0.conf — Client

[Interface]
Address = 10.0.0.2/24
PrivateKey = <client_private_key>
DNS = 1.1.1.1

[Peer]
PublicKey = <server_public_key>
Endpoint = vpn.example.com:51820
AllowedIPs = 0.0.0.0/0, ::/0  # Route all traffic through VPN
PersistentKeepalive = 25       # NAT traversal
```

**Hardening considerations:**
- Generate keys with `wg genkey` — never reuse across deployments.
- Restrict `AllowedIPs` to the minimum required routes (split tunneling where appropriate).
- Use `PresharedKey` for post-quantum resistance (additional symmetric key in handshake).
- Firewall the WireGuard port to known IP ranges if possible.
- Set `SaveConfig = false` to prevent runtime changes from persisting.

### 4.2 OpenVPN

OpenVPN operates in userspace over TLS, providing more configuration flexibility at the cost of complexity and performance.

**Current recommended settings (from angristan/openvpn-install):**
- **Data channel**: AES-128-GCM (default), AES-256-GCM, or CHACHA20-POLY1305
- **Certificates**: ECDSA with prime256v1 (default) or RSA 2048-4096
- **TLS minimum**: 1.2 (mandatory), with TLS 1.3 optional
- **TLS 1.3 ciphers**: TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256
- **Compression**: Disabled by default (VORACLE vulnerability — CVE-2018-7160)
- **Auth modes**: PKI (traditional CA) or peer fingerprint (OpenVPN 2.6+ — SHA256 fingerprint, no CA needed)

**Hardening beyond defaults:**
```
# server.conf additions
tls-version-min 1.2
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384
cipher AES-256-GCM
auth SHA512
dh none                    # Use ECDH instead of static DH
remote-cert-tls client     # Verify client certificate purpose
verify-x509-name client    # Verify certificate CN
tls-crypt /etc/openvpn/ta.key   # Encrypt control channel (hide VPN fingerprint)
user nobody
group nogroup
persist-key
persist-tun
```

### 4.3 WireGuard vs. OpenVPN Decision Matrix

| Factor | WireGuard | OpenVPN |
|--------|-----------|---------|
| **Codebase** | ~4,000 LOC (auditable) | ~100,000+ LOC |
| **Performance** | Kernel-space, ~1 Gbps+ | Userspace, ~200-500 Mbps |
| **Crypto agility** | None (fixed primitives) | Full (flexible but complex) |
| **Stealth** | Silent to unauth packets | TLS fingerprint detectable |
| **NAT traversal** | Built-in (UDP) | TCP fallback available |
| **Client support** | All major platforms | All major platforms |
| **Key management** | Static key pairs | PKI/CA infrastructure |
| **Audit surface** | Minimal | Large |
| **Use when** | Site-to-site, remote access, performance-critical | Legacy compliance, TCP-only networks, complex auth |

### 4.4 Algo VPN (Automated WireGuard/IPsec Deployment)

Trail of Bits' Algo automates VPN server deployment with security-first defaults:
- Deploys WireGuard + IPsec (IKEv2 with AES-GCM, SHA2, P-256)
- Rejects legacy protocols (L2TP, IKEv1, RSA key exchange)
- Minimal logging with automatic rotation
- Optional DNS-based ad-blocking
- Supports DigitalOcean, AWS, Azure, GCP, Vultr, Hetzner, and self-hosted Ubuntu 22.04
- Ansible-based — fully automated, reproducible

---

## 5. SSH Hardening Beyond Defaults

### 5.1 sshd_config Hardening

```bash
# /etc/ssh/sshd_config

# Protocol and authentication
Protocol 2
PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
KbdInteractiveAuthentication no
UsePAM yes

# Key types — Ed25519 preferred (smaller, faster, no NIST curve concerns)
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key  # Fallback for legacy clients

# Restrict key exchange, ciphers, MACs to strong algorithms only
KexAlgorithms curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

# Access restrictions
AllowGroups ssh-users
MaxAuthTries 3
MaxSessions 3
LoginGraceTime 30

# Disable forwarding unless needed
AllowTcpForwarding no
AllowAgentForwarding no
X11Forwarding no
PermitTunnel no
GatewayPorts no

# Session hardening
ClientAliveInterval 300
ClientAliveCountMax 2
TCPKeepAlive no

# Logging
LogLevel VERBOSE
SyslogFacility AUTH

# Chroot SFTP users (if applicable)
# Match Group sftp-only
#     ChrootDirectory /home/%u
#     ForceCommand internal-sftp
#     AllowTcpForwarding no
```

### 5.2 Key Management

```bash
# Generate Ed25519 key (client-side)
ssh-keygen -t ed25519 -a 100 -C "user@host-$(date +%Y%m%d)"

# Remove weak host keys (server-side)
rm /etc/ssh/ssh_host_dsa_key*
rm /etc/ssh/ssh_host_ecdsa_key*

# Regenerate RSA host key with sufficient length
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""

# Regenerate moduli — remove small primes
awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.safe
mv /etc/ssh/moduli.safe /etc/ssh/moduli
```

### 5.3 Two-Factor Authentication

```bash
# Install Google Authenticator PAM module
apt install libpam-google-authenticator

# Configure PAM (/etc/pam.d/sshd)
# Add at the top:
auth required pam_google_authenticator.so

# sshd_config additions
AuthenticationMethods publickey,keyboard-interactive
ChallengeResponseAuthentication yes  # Required for 2FA prompt
```

### 5.4 SSH Certificate Authority

For environments with many hosts and users, use an SSH CA instead of distributing authorized_keys:

```bash
# Generate CA key pair (keep private key offline)
ssh-keygen -t ed25519 -f /etc/ssh/ca_key -C "SSH CA"

# Sign a user key (valid 52 weeks, for specific principals)
ssh-keygen -s /etc/ssh/ca_key -I "user@corp" -n admin,deploy \
  -V +52w user_key.pub

# Sign a host key
ssh-keygen -s /etc/ssh/ca_key -I "web01.example.com" -h \
  -V +52w /etc/ssh/ssh_host_ed25519_key.pub

# sshd_config — trust the CA
TrustedUserCAKeys /etc/ssh/ca_key.pub

# Client config — trust host CA
# ~/.ssh/known_hosts:
# @cert-authority *.example.com <contents of ca_key.pub>
```

### 5.5 SSH Audit and Port Knocking

```bash
# Audit SSH configuration with ssh-audit
pip install ssh-audit
ssh-audit localhost

# Port knocking with knockd (obscurity, not security — defense-in-depth)
# /etc/knockd.conf
# [openSSH]
#     sequence    = 7000,8000,9000
#     seq_timeout = 5
#     command     = /usr/sbin/iptables -A INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
#     tcpflags    = syn
#
# [closeSSH]
#     sequence    = 9000,8000,7000
#     seq_timeout = 5
#     command     = /usr/sbin/iptables -D INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
#     tcpflags    = syn
```

---

## 6. IDS/IPS Placement and Architecture

### 6.1 fail2ban

Log-monitoring IPS that dynamically bans IPs based on repeated failures.

**Architecture:**
- Reads log files (auth.log, nginx/access.log, etc.)
- Applies **filters** (regex patterns) to identify malicious behavior
- Triggers **actions** (iptables/nftables/firewalld rules) when thresholds are exceeded
- Organized into **jails** (service-specific configurations)

**Configuration:**
```ini
# /etc/fail2ban/jail.local (never edit jail.conf directly)

[DEFAULT]
bantime = 3600          # 1 hour ban
findtime = 600          # 10 minute window
maxretry = 5            # 5 failures triggers ban
banaction = nftables    # Use nftables backend
ignoreip = 127.0.0.1/8 10.0.100.0/24  # Whitelist management subnet

[sshd]
enabled = true
port = 22
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 86400         # 24 hours for SSH brute force

[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
logpath = /var/log/nginx/error.log
maxretry = 5

[nginx-botsearch]
enabled = true
port = http,https
filter = nginx-botsearch
logpath = /var/log/nginx/access.log
maxretry = 2
bantime = 604800        # 1 week for scanners

[nginx-limit-req]
enabled = true
port = http,https
filter = nginx-limit-req
logpath = /var/log/nginx/error.log
maxretry = 10
```

**Custom filter example:**
```ini
# /etc/fail2ban/filter.d/nginx-4xx.conf
[Definition]
failregex = ^<HOST> .* "(GET|POST|HEAD|PUT|DELETE).*" (400|401|403|404|405) .*$
ignoreregex = .*\.(css|js|png|jpg|gif|ico|woff|svg)
```

### 6.2 CrowdSec

CrowdSec extends the fail2ban model with crowd-sourced threat intelligence and a modular "bouncer" architecture.

**Key differences from fail2ban:**
- **"Detect Here, Remedy There"** — detection is centralized, remediation is distributed across infrastructure layers.
- **Community Blocklist** — when one CrowdSec instance detects an attack, the attacker IP is shared with all participants. Proactive blocking before the attack reaches your infrastructure.
- **Hub ecosystem** — community-contributed scenarios (brute force, port scan, web scan), parsers (log format extractors), and collections (curated bundles).
- **Bouncer architecture** — separate remediation components that can act at different layers:
  - Firewall bouncer (iptables/nftables)
  - NGINX bouncer (application layer)
  - Cloudflare bouncer (edge/CDN)
  - Custom bouncers via API

**Deployment architecture:**
```
   ┌──────────────────────────────────────────────┐
   │                 CrowdSec Hub                  │
   │  (Scenarios, Parsers, Collections, Blocklists)│
   └──────────────────┬───────────────────────────┘
                      │ API
   ┌──────────────────▼───────────────────────────┐
   │           CrowdSec Security Engine            │
   │  ┌─────────┐  ┌──────────┐  ┌────────────┐  │
   │  │ Parsers │→ │Scenarios │→ │ Decisions  │  │
   │  │(log     │  │(detection│  │ (ban/     │  │
   │  │ ingest) │  │ logic)   │  │  captcha) │  │
   │  └─────────┘  └──────────┘  └─────┬──────┘  │
   └────────────────────────────────────┼─────────┘
                                        │ Local API
          ┌─────────────────────────────┼──────────┐
          │              │              │           │
   ┌──────▼─────┐ ┌─────▼──────┐ ┌────▼─────┐    │
   │ Firewall   │ │  NGINX     │ │Cloudflare│    │
   │ Bouncer    │ │  Bouncer   │ │ Bouncer  │    │
   │(nftables)  │ │(lua plugin)│ │(API)     │    │
   └────────────┘ └────────────┘ └──────────┘    │
```

### 6.3 IDS/IPS Placement Strategy

```
Internet
    │
    ▼
┌─────────┐
│  IPS    │ ← Inline IPS at network edge (Suricata/Snort in IPS mode)
│ (inline)│   Blocks known-bad traffic before it reaches DMZ
└────┬────┘
     │
┌────▼────┐
│  WAF    │ ← Application-layer filtering (ModSecurity, NAXSI)
│         │   SQL injection, XSS, OWASP Top 10
└────┬────┘
     │
┌────▼────┐
│  DMZ    │ ← Network IDS tap/span port (Suricata in IDS mode)
│ Servers │   Passive monitoring, alert generation, no blocking
└────┬────┘
     │
┌────▼──────┐
│ Internal  │ ← Host-based IDS (OSSEC/Wazuh agents)
│ Firewall  │   File integrity, log analysis, rootkit detection
└────┬──────┘
     │
┌────▼──────┐
│ Internal  │ ← Network IDS on internal segments
│ Servers   │   Detect lateral movement, C2 beaconing
└───────────┘

+ CrowdSec/fail2ban on every host (application-layer IPS)
+ Centralized SIEM aggregating all IDS alerts
```

**Placement principles:**
- IPS (blocking) at the perimeter — false positives are less disruptive at the edge.
- IDS (alerting) on internal segments — higher fidelity, lower risk of disrupting legitimate traffic.
- HIDS (host-based) on every server — file integrity, log analysis, local anomaly detection.
- WAF in front of every web application — not optional, even behind a network IPS.
- Centralized logging is non-negotiable — you cannot detect what you cannot see.

---

## 7. Server Hardening Checklist

### 7.1 Initial Setup

- [ ] **Minimal installation** — install only required packages. No GUI on servers.
- [ ] **Remove/disable unnecessary services** — `systemctl list-unit-files --state=enabled` and disable everything not needed.
- [ ] **Set hostname and timezone** — consistent naming convention, UTC timezone for log correlation.
- [ ] **Configure NTP** — use `systemd-timesyncd` or `chrony` pointing to trusted NTP servers. Accurate time is critical for log forensics.
- [ ] **Disk encryption** — LUKS for data at rest. Required for any system handling sensitive data.

### 7.2 User and Access Control

- [ ] **Disable root login** — `PermitRootLogin no` in sshd_config.
- [ ] **Create individual admin accounts** — no shared accounts. Every action must be attributable.
- [ ] **sudo with specific commands** — never `ALL=(ALL) ALL`. Grant only the commands each role needs.
- [ ] **Restrict su** — `dpkg-statoverride --update --add root adm 4750 /bin/su` or PAM `pam_wheel.so`.
- [ ] **Password policy** — `libpam-pwquality`: minlen=14, minclass=3, maxrepeat=3, reject username.
- [ ] **Account lockout** — `pam_faillock.so`: deny=5, unlock_time=900.
- [ ] **Login banner** — `/etc/issue` and `/etc/issue.net` with legal warning (establishes consent to monitoring).
- [ ] **Umask 027** — default file creation permissions prevent world-readable files.

### 7.3 SSH (see Section 5 for detailed configuration)

- [ ] Ed25519 keys only, passwords disabled
- [ ] Restrict to `ssh-users` group
- [ ] Rate limit with fail2ban (3 attempts, 24h ban)
- [ ] Two-factor authentication for privileged accounts
- [ ] Change default port (obscurity, but reduces log noise)
- [ ] Disable all forwarding unless specifically required

### 7.4 Firewall (see Section 2 for detailed rules)

- [ ] Default deny on INPUT and FORWARD
- [ ] Explicit allow only for required services
- [ ] Rate limiting on public-facing services
- [ ] Kernel network stack hardening (sysctl)
- [ ] Drop invalid packets and bogus TCP flags
- [ ] Log dropped packets for analysis

### 7.5 Kernel Hardening

```bash
# /etc/sysctl.d/99-security.conf

# ASLR (Address Space Layout Randomization) — maximum randomization
kernel.randomize_va_space = 2

# Restrict dmesg to root (information leakage prevention)
kernel.dmesg_restrict = 1

# Restrict kernel pointer exposure
kernel.kptr_restrict = 2

# Disable SysRq (magic SysRq key)
kernel.sysrq = 0

# Restrict ptrace (prevents process memory inspection by non-root)
kernel.yama.ptrace_scope = 2

# Restrict unprivileged user namespaces (container escape prevention)
kernel.unprivileged_userns_clone = 0

# Restrict eBPF to root
kernel.unprivileged_bpf_disabled = 1
net.core.bpf_jit_harden = 2

# Core dump restrictions
fs.suid_dumpable = 0

# Restrict loading kernel modules after boot
# kernel.modules_disabled = 1  # WARNING: prevents ANY module loading — use with caution

# Restrict access to kernel logs
kernel.printk = 3 3 3 3
```

### 7.6 File System Hardening

```bash
# /etc/fstab hardening — mount options

# /tmp — noexec, nosuid, nodev (prevent execution from temp directories)
tmpfs /tmp tmpfs defaults,noexec,nosuid,nodev,size=2G 0 0

# /var/tmp — same treatment
tmpfs /var/tmp tmpfs defaults,noexec,nosuid,nodev,size=1G 0 0

# /dev/shm — shared memory, no execution
tmpfs /dev/shm tmpfs defaults,noexec,nosuid,nodev 0 0

# /home — nosuid, nodev
/dev/mapper/vg-home /home ext4 defaults,nosuid,nodev 0 2

# /var/log — append-only (noexec, nosuid, nodev)
/dev/mapper/vg-log /var/log ext4 defaults,noexec,nosuid,nodev 0 2

# /boot — read-only after boot (remount rw for kernel updates)
# /dev/sda1 /boot ext4 defaults,nosuid,nodev,noexec,ro 0 2
```

```bash
# Set sticky bit on world-writable directories
chmod +t /tmp /var/tmp

# Find and fix world-writable files (audit only — review before changing)
find / -xdev -type f -perm -0002 -ls

# Find SUID/SGID binaries (audit regularly — each is a privilege escalation vector)
find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -ls

# Remove unnecessary SUID bits
chmod u-s /usr/bin/newgrp
chmod u-s /usr/bin/chfn
chmod u-s /usr/bin/chsh
```

### 7.7 Audit and Monitoring

```bash
# Install and configure auditd
apt install auditd audispd-plugins

# /etc/audit/rules.d/hardening.rules

# Monitor auth files
-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/sudoers -p wa -k sudo_changes
-w /etc/sudoers.d/ -p wa -k sudo_changes

# Monitor SSH config
-w /etc/ssh/sshd_config -p wa -k sshd_config

# Monitor cron
-w /etc/crontab -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /var/spool/cron/ -p wa -k cron

# Monitor kernel module loading
-w /sbin/insmod -p x -k kernel_modules
-w /sbin/rmmod -p x -k kernel_modules
-w /sbin/modprobe -p x -k kernel_modules

# Log all execve calls (high volume — enable for investigations)
# -a always,exit -F arch=b64 -S execve -k exec_log

# Monitor network configuration changes
-w /etc/hosts -p wa -k network_config
-w /etc/network/ -p wa -k network_config
-w /etc/sysctl.conf -p wa -k sysctl

# Immutable audit rules (cannot be changed without reboot)
-e 2
```

### 7.8 Intrusion Detection (Host-Based)

- [ ] **AIDE** or **Tripwire** — file integrity monitoring. Initialize baseline after hardening. Cron daily checks.
- [ ] **rkhunter** — rootkit detection. Run weekly: `rkhunter --check --skip-keypress`.
- [ ] **ClamAV** — malware scanning (not a substitute for proper hardening). `freshclam && clamscan -r /`.
- [ ] **Lynis** — security auditing: `lynis audit system`. Target score > 80.
- [ ] **OSSEC/Wazuh** — HIDS agent with centralized management. Log analysis, file integrity, rootkit detection, active response.

### 7.9 Automatic Updates

```bash
# Debian/Ubuntu — unattended-upgrades
apt install unattended-upgrades
dpkg-reconfigure -plow unattended-upgrades

# /etc/apt/apt.conf.d/50unattended-upgrades
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
};
Unattended-Upgrade::AutoFixInterruptedDpkg "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "false";  # Manual reboot for production
Unattended-Upgrade::Mail "admin@example.com";
```

### 7.10 Logging Architecture

- [ ] **Centralized logging** — rsyslog or syslog-ng forwarding to a dedicated log server.
- [ ] **Log server on isolated VLAN** — write-only from sources, no outbound connections.
- [ ] **Tamper protection** — log forwarding happens in real-time; local logs can be tampered with, remote copies cannot.
- [ ] **Retention** — minimum 90 days hot, 1 year cold storage. Regulatory requirements may dictate longer.
- [ ] **Log analysis** — ELK stack, Graylog, or Splunk for searchability and alerting.
- [ ] **Monitoring** — Prometheus + Grafana for infrastructure metrics; alerting on anomalies.

---

## 8. Infrastructure as Code Security

### 8.1 Secret Management

- Never store secrets in IaC files, environment variables in git, or tfstate.
- Use **HashiCorp Vault**, **AWS Secrets Manager**, or **SOPS** for encrypted secrets.
- Scan for secret leaks: **truffleHog**, **git-secrets**, **GitGuardian**, **gitleaks**.

### 8.2 Static Analysis

| Tool | Target | Purpose |
|------|--------|---------|
| **Checkov** | Terraform, CloudFormation, K8s | Misconfiguration detection |
| **tfsec** | Terraform | Security-focused Terraform scanning |
| **Trivy** | Container images, IaC, filesystems | Vulnerability and misconfiguration scanning |
| **Anchore/Grype** | Container images | CVE scanning |
| **KICS** | Terraform, Ansible, Docker, K8s | Multi-platform IaC scanning |
| **Semgrep** | General code + IaC | Pattern-based analysis |

### 8.3 State File Security

Terraform state files contain every secret value in plaintext.

- **Remote backend with encryption** — S3 + DynamoDB with server-side encryption and access logging.
- **State locking** — prevent concurrent modifications.
- **Access control** — IAM policies restricting state access to CI/CD pipeline service accounts.
- Never commit `.tfstate` to git. Add to `.gitignore`.

### 8.4 Immutable Infrastructure

- Provision new infrastructure for changes — do not mutate running systems.
- Use golden images (Packer) with hardening baked in.
- Blue/green or canary deployments for zero-downtime updates.
- Runtime threat detection: **Falco** for unexpected behavior in containers and hosts.

### 8.5 Policy as Code

```python
# Open Policy Agent (OPA) / Rego example
# Deny public S3 buckets
deny[msg] {
    input.resource.type == "aws_s3_bucket"
    input.resource.values.acl == "public-read"
    msg := sprintf("S3 bucket '%s' has public-read ACL", [input.resource.name])
}
```

---

## 9. Reference: Authoritative Hardening Standards

### By Platform

| Platform | Standard | Source |
|----------|----------|--------|
| Linux (general) | CIS Benchmark for Distribution Independent Linux | CIS |
| RHEL/CentOS | DISA STIG for RHEL | DISA |
| Ubuntu | CIS Benchmark for Ubuntu Linux | CIS |
| Windows | Microsoft Security Baselines | Microsoft |
| Docker | CIS Docker Benchmark | CIS |
| Kubernetes | NSA/CISA Kubernetes Hardening Guide | NSA |
| VMware ESXi | CIS VMware ESXi Benchmark | CIS |
| Network devices | NSA Network Device Hardening | NSA |

### By Domain

| Domain | Standard | Source |
|--------|----------|--------|
| TLS/SSL | NIST SP 800-52 Rev 2 | NIST |
| SSH | NIST IR 7966 | NIST |
| DNS | NIST SP 800-81-2 | NIST |
| Containers | NIST SP 800-190 | NIST |
| Hypervisors | NIST SP 800-125A/B | NIST |
| Active Directory | ANSSI AD Assessment Checklist (2022) | ANSSI |
| Web servers | OWASP TLS Cipher String Cheat Sheet | OWASP |

### Tools for Compliance Assessment

| Tool | Purpose |
|------|---------|
| **Lynis** | Linux security auditing (CIS-aligned) |
| **OpenSCAP** | Automated STIG/CIS compliance scanning |
| **InSpec** | Infrastructure compliance testing (Chef) |
| **Prowler** | AWS security assessment |
| **ScoutSuite** | Multi-cloud security auditing |
| **kube-bench** | CIS Kubernetes Benchmark |
| **docker-bench-security** | CIS Docker Benchmark |
| **ssh-audit** | SSH configuration auditing |
| **testssl.sh** | TLS configuration testing |
| **Mozilla Observatory** | Web security header testing |
