---
name: rt-shodan-recon
description: "Internet-wide asset discovery using Shodan, Censys, and FOFA. Use to find exposed services, open databases, network infrastructure, and vulnerable software versions without directly contacting target systems. Passive reconnaissance using internet scan databases. Requires API keys."
---

# rt-shodan-recon

## Overview

This skill performs passive reconnaissance against internet-wide scan databases (Shodan, Censys, FOFA, ZoomEye) to enumerate exposed services, discover shadow IT, identify vulnerable software, and map network infrastructure belonging to the target organization — all without sending a single packet to target systems.

**When to use this skill:**
- Blackbox engagements where stealth is required before active scanning
- Pre-engagement asset discovery to understand what the client exposes
- Identifying forgotten internet-facing services outside the client's own inventory
- Finding exposed databases, admin panels, VPNs, and industrial systems
- Discovering credential leaks tied to target infrastructure (combined with other OSINT)
- Building a comprehensive attack surface map prior to exploitation phases

**Primary data sources:**
| Platform | Coverage | Strength |
|----------|----------|----------|
| Shodan | IPv4 internet (crawls continuously) | Best CLI, widest adoption, richest filters |
| Censys | IPv4 + IPv6, TLS cert graph | Superior certificate and TLS data |
| FOFA | APAC coverage, Chinese infrastructure | Best for targets with Asian presence |
| ZoomEye | IPv4 + application fingerprinting | Good overlap, alternative data source |
| BinaryEdge | Real-time scans, risk scoring | Useful for recent exposure events |

**API keys required:** Shodan, Censys, FOFA, ZoomEye — store in `~/.config/rtexit/api_keys.env` (see Output section).

---

## Skill Levels

### BEGINNER

Understand the basics: search by organization name and ASN, download results, save to output directory.

```bash
# Install Shodan CLI
pip3 install shodan censys

# Configure Shodan with your API key
shodan init YOUR_SHODAN_API_KEY

# Search by organization name — returns all IPs Shodan sees for that org
shodan search --fields ip_str,port,org,product,version "org:\"Target Corporation\""

# Search by domain to find all hosts Shodan has indexed
shodan search --fields ip_str,port,hostnames,product "hostname:targetcorp.com"

# Look up a single IP for all banner data
shodan host 203.0.113.42

# Export results as JSON for autodoc ingestion
shodan search --fields ip_str,port,org,product,version,ssl.cert.subject.cn \
  "org:\"Target Corporation\"" \
  --limit 1000 \
  > _rtexit-output/docs/reconnaissance/shodan_org_raw.json

# Count results before pulling (saves API credits)
shodan count "org:\"Target Corporation\""
shodan count "net:203.0.113.0/24"
```

**Beginner checklist:**
- [ ] Shodan CLI installed and initialized with API key
- [ ] Org name search completed
- [ ] Single IP lookups performed for key hosts
- [ ] Raw results saved to reconnaissance output directory

---

### INTERMEDIATE

Combine multiple search operators, perform ASN and CIDR sweeps, pull certificate data, identify specific vulnerable services.

```bash
# --- ASN Enumeration ---
# Find the target's ASN first via BGP data
curl -s "https://api.bgpview.io/search?query_term=targetcorp.com" | python3 -m json.tool
# Or using whois
whois -h whois.radb.net -- '-i origin AS12345' | grep ^route

# Shodan: search by ASN
shodan search --fields ip_str,port,product,version,hostnames \
  "asn:AS12345" --limit 5000 \
  > _rtexit-output/docs/reconnaissance/shodan_asn_AS12345.json

# Shodan: CIDR range sweep
shodan search --fields ip_str,port,product,version \
  "net:203.0.113.0/24" \
  > _rtexit-output/docs/reconnaissance/shodan_cidr_203.0.113.json

# --- Certificate-Based Discovery (Censys) ---
# Find all certs issued to *.targetcorp.com — reveals subdomains
censys search "parsed.names: targetcorp.com" \
  --index certificates \
  --fields parsed.names,parsed.subject_dn,parsed.issuer_dn,parsed.validity \
  > _rtexit-output/docs/reconnaissance/censys_certs_targetcorp.json

# Censys hosts with TLS cert matching target org
censys search "services.tls.certificates.leaf_data.subject.organization: \"Target Corporation\"" \
  --index hosts \
  > _rtexit-output/docs/reconnaissance/censys_hosts_tls.json

# --- Find Exposed Services ---
# RDP exposed to internet
shodan search --fields ip_str,port,org "org:\"Target Corporation\" port:3389"

# VPN appliances (Pulse Secure, Fortinet, Cisco)
shodan search "org:\"Target Corporation\" product:\"Pulse Secure\""
shodan search "org:\"Target Corporation\" http.title:\"SSL VPN\""
shodan search "org:\"Target Corporation\" product:\"FortiGate\""

# Exposed databases
shodan search "org:\"Target Corporation\" product:MongoDB"
shodan search "org:\"Target Corporation\" product:\"Elasticsearch\""
shodan search "org:\"Target Corporation\" product:\"Redis\""
shodan search "org:\"Target Corporation\" port:5432 product:PostgreSQL"
shodan search "org:\"Target Corporation\" port:1433 product:\"Microsoft SQL Server\""

# Industrial control systems / SCADA
shodan search "org:\"Target Corporation\" tag:ics"
shodan search "org:\"Target Corporation\" port:102"  # Siemens S7
shodan search "org:\"Target Corporation\" port:502"  # Modbus

# --- Shodan Facets (aggregated stats without pulling all records) ---
shodan stats --facets port,product "org:\"Target Corporation\""

# --- Pull full host data for a CIDR range ---
shodan download --limit 10000 targetcorp_net net:203.0.113.0/24
shodan parse --fields ip_str,port,product,version targetcorp_net.json.gz \
  > _rtexit-output/docs/reconnaissance/shodan_parsed_net.csv
```

**Intermediate checklist:**
- [ ] ASN identified and all associated IP ranges documented
- [ ] Certificate data pulled from Censys — new subdomains identified
- [ ] Key exposed services (RDP, VPN, DB) inventoried
- [ ] Facets run to understand port/product distribution
- [ ] Results parsed to CSV for reporting

---

### ADVANCED

Shodan dorks, vulnerability filters, historical data, FOFA queries, Python automation, banner analysis.

```bash
# --- Shodan Dorks for High-Value Targets ---

# Admin panels
shodan search "org:\"Target Corporation\" http.title:\"Admin\""
shodan search "org:\"Target Corporation\" http.title:\"Dashboard\""
shodan search "org:\"Target Corporation\" http.title:\"phpMyAdmin\""
shodan search "org:\"Target Corporation\" http.title:\"Kibana\""
shodan search "org:\"Target Corporation\" http.title:\"Grafana\""

# Default credentials / setup pages
shodan search "org:\"Target Corporation\" http.title:\"Setup\""
shodan search "org:\"Target Corporation\" http.title:\"Installation\""
shodan search "org:\"Target Corporation\" http.title:\"Welcome to nginx\""

# Exposed Git repos
shodan search "org:\"Target Corporation\" http.title:\"Index of /.git\""

# Citrix / remote access
shodan search "org:\"Target Corporation\" http.title:\"Citrix Gateway\""
shodan search "org:\"Target Corporation\" http.title:\"NetScaler\""

# Microsoft Exchange / OWA
shodan search "org:\"Target Corporation\" http.title:\"Outlook Web App\""
shodan search "org:\"Target Corporation\" http.title:\"Microsoft Exchange\""
shodan search "org:\"Target Corporation\" product:\"Microsoft Exchange httpapi\""

# JIRA / Confluence (common data exfil targets)
shodan search "org:\"Target Corporation\" http.title:\"Jira\""
shodan search "org:\"Target Corporation\" http.title:\"Confluence\""

# Exposed S3-style storage proxies
shodan search "org:\"Target Corporation\" http.title:\"Minio\""

# Printers and IoT
shodan search "org:\"Target Corporation\" http.title:\"Printer\""
shodan search "org:\"Target Corporation\" port:9100"  # Raw printing

# --- CVE-based Vulnerability Searches ---
# Shodan vuln filter (requires Membership or above)
shodan search "org:\"Target Corporation\" vuln:CVE-2021-44228"    # Log4Shell
shodan search "org:\"Target Corporation\" vuln:CVE-2021-26855"    # ProxyLogon Exchange
shodan search "org:\"Target Corporation\" vuln:CVE-2019-19781"    # Citrix ADC
shodan search "org:\"Target Corporation\" vuln:CVE-2022-26134"    # Confluence RCE
shodan search "org:\"Target Corporation\" vuln:CVE-2023-23397"    # Outlook NTLM

# --- Historical Data ---
# Pull historical banners for an IP (shows what was running before)
shodan host --history 203.0.113.42

# --- FOFA Queries ---
# FOFA syntax differs from Shodan — use domain, title, cert operators
# FOFA base URL: https://fofa.info/
# CLI: pip3 install fofa-cli
fofa search 'domain="targetcorp.com"'
fofa search 'cert="targetcorp.com" && country="US"'
fofa search 'org="Target Corporation" && port="3389"'
fofa search 'title="Target Corporation" && app="Apache"'

# Export FOFA results to CSV
fofa search --fields "ip,port,title,country,city,protocol,server" \
  'cert="targetcorp.com"' \
  --size 1000 \
  --format csv \
  > _rtexit-output/docs/reconnaissance/fofa_cert_targetcorp.csv

# --- ZoomEye ---
# pip3 install zoomeye-sdk
zoomeye search "org:\"Target Corporation\""
zoomeye search "hostname:targetcorp.com"
zoomeye search "site:targetcorp.com +port:22"

# --- Censys Advanced ---
# Find hosts running specific software version
censys search "services.software.product: \"Apache\" AND services.software.version: \"2.4.49\"" \
  --index hosts

# Hosts with expired or self-signed certs (often forgotten/shadow IT)
censys search "services.tls.certificates.leaf_data.subject.organization: \"Target Corporation\" AND services.tls.certificates.leaf_data.issuer.organization: \"Target Corporation\"" \
  --index hosts

# --- BinaryEdge ---
# pip3 install pybinaryedge
binaryedge host 203.0.113.42
binaryedge search "org:\"Target Corporation\""
binaryedge search "hostname:targetcorp.com" --type services
```

**Advanced checklist:**
- [ ] All admin panel / login page exposure documented
- [ ] CVE-specific searches run against all known CVEs in current threat list
- [ ] Historical data reviewed for recently decommissioned services
- [ ] FOFA + ZoomEye run to cross-validate Shodan findings
- [ ] BinaryEdge risk scoring pulled for key IPs

---

### EXPERT

Full automation with Python SDK, bulk ASN/CIDR correlation, integration with RTExit autodoc engine, custom Shodan facet dashboards, continuous monitoring setup.

```bash
# --- Python Shodan SDK Automation ---
# Save as: scripts/shodan_bulk_recon.py
```

```python
#!/usr/bin/env python3
"""
rt-shodan-recon bulk automation script
Integrates with RTExit autodoc engine
Usage: python3 shodan_bulk_recon.py --org "Target Corporation" --asn AS12345 --cidr 203.0.113.0/24
"""

import shodan
import censys.search
import json
import csv
import argparse
import os
import sys
from datetime import datetime
from pathlib import Path

OUTPUT_BASE = Path("_rtexit-output/docs/reconnaissance")
SHODAN_API_KEY = os.environ.get("SHODAN_API_KEY")
CENSYS_API_ID  = os.environ.get("CENSYS_API_ID")
CENSYS_API_SECRET = os.environ.get("CENSYS_API_SECRET")

VULN_CVES = [
    "CVE-2021-44228",  # Log4Shell
    "CVE-2021-26855",  # ProxyLogon
    "CVE-2019-19781",  # Citrix ADC
    "CVE-2022-26134",  # Confluence RCE
    "CVE-2023-23397",  # Outlook NTLM
    "CVE-2022-22965",  # Spring4Shell
    "CVE-2021-21985",  # VMware vCenter
    "CVE-2020-5902",   # F5 BIG-IP
]

HIGH_VALUE_TITLES = [
    "phpMyAdmin", "Kibana", "Grafana", "Admin", "Dashboard",
    "Jira", "Confluence", "NetScaler", "Citrix Gateway",
    "Outlook Web App", "Minio", "GitLab", "Jenkins",
    "SonarQube", "Rancher", "Portainer",
]


def init_shodan():
    if not SHODAN_API_KEY:
        print("[!] SHODAN_API_KEY not set in environment", file=sys.stderr)
        sys.exit(1)
    return shodan.Shodan(SHODAN_API_KEY)


def shodan_search_paginated(api, query, max_results=1000):
    """Pull all pages of Shodan results for a query."""
    results = []
    try:
        count = api.count(query)["total"]
        print(f"[*] Shodan: '{query}' -> {count} results (pulling up to {max_results})")
        for result in api.search_cursor(query):
            results.append(result)
            if len(results) >= max_results:
                break
    except shodan.APIError as e:
        print(f"[!] Shodan API error: {e}", file=sys.stderr)
    return results


def run_org_search(api, org_name, output_dir):
    query = f'org:"{org_name}"'
    results = shodan_search_paginated(api, query)
    out_file = output_dir / "shodan_org_full.json"
    with open(out_file, "w") as f:
        json.dump(results, f, indent=2, default=str)
    print(f"[+] Org results saved: {out_file} ({len(results)} hosts)")
    return results


def run_vuln_search(api, org_name, output_dir):
    vuln_hits = {}
    for cve in VULN_CVES:
        query = f'org:"{org_name}" vuln:{cve}'
        results = shodan_search_paginated(api, query, max_results=500)
        if results:
            vuln_hits[cve] = results
            print(f"[!] CRITICAL: {len(results)} host(s) potentially vulnerable to {cve}")
    out_file = output_dir / "shodan_vuln_hits.json"
    with open(out_file, "w") as f:
        json.dump(vuln_hits, f, indent=2, default=str)
    print(f"[+] Vulnerability hits saved: {out_file}")
    return vuln_hits


def run_title_search(api, org_name, output_dir):
    title_hits = {}
    for title in HIGH_VALUE_TITLES:
        query = f'org:"{org_name}" http.title:"{title}"'
        results = shodan_search_paginated(api, query, max_results=200)
        if results:
            title_hits[title] = results
            print(f"[+] Found '{title}' on {len(results)} host(s)")
    out_file = output_dir / "shodan_admin_panels.json"
    with open(out_file, "w") as f:
        json.dump(title_hits, f, indent=2, default=str)
    return title_hits


def generate_summary_csv(org_results, vuln_hits, output_dir):
    """Generate operator-friendly CSV summary for reporting."""
    rows = []
    for host in org_results:
        ip = host.get("ip_str", "")
        for item in host.get("data", [host]):
            rows.append({
                "ip": ip,
                "port": item.get("port", ""),
                "product": item.get("product", ""),
                "version": item.get("version", ""),
                "org": item.get("org", ""),
                "hostnames": ", ".join(item.get("hostnames", [])),
                "country": item.get("location", {}).get("country_name", ""),
                "city": item.get("location", {}).get("city", ""),
                "cves": ", ".join([c for c, hits in vuln_hits.items()
                                   if any(h.get("ip_str") == ip for h in hits)]),
                "timestamp": item.get("timestamp", ""),
            })
    out_file = output_dir / "shodan_summary.csv"
    if rows:
        with open(out_file, "w", newline="") as f:
            writer = csv.DictWriter(f, fieldnames=rows[0].keys())
            writer.writeheader()
            writer.writerows(rows)
        print(f"[+] Summary CSV saved: {out_file} ({len(rows)} rows)")


def autodoc_log(output_dir, org_name, host_count, vuln_count):
    """Write a machine-readable log entry for the RTExit autodoc engine."""
    log_entry = {
        "skill": "rt-shodan-recon",
        "timestamp": datetime.utcnow().isoformat() + "Z",
        "target_org": org_name,
        "hosts_discovered": host_count,
        "vuln_cve_hits": vuln_count,
        "output_files": [
            str(output_dir / "shodan_org_full.json"),
            str(output_dir / "shodan_vuln_hits.json"),
            str(output_dir / "shodan_admin_panels.json"),
            str(output_dir / "shodan_summary.csv"),
            str(output_dir / "censys_certs.json"),
        ],
    }
    log_file = output_dir / "shodan_recon_autodoc.json"
    with open(log_file, "w") as f:
        json.dump(log_entry, f, indent=2)
    print(f"[+] Autodoc log written: {log_file}")


def main():
    parser = argparse.ArgumentParser(description="RTExit Shodan Bulk Recon")
    parser.add_argument("--org",  required=True, help='Organization name e.g. "Target Corporation"')
    parser.add_argument("--asn",  help="ASN number e.g. AS12345")
    parser.add_argument("--cidr", help="CIDR range e.g. 203.0.113.0/24")
    parser.add_argument("--out",  default=str(OUTPUT_BASE), help="Output directory")
    args = parser.parse_args()

    output_dir = Path(args.out)
    output_dir.mkdir(parents=True, exist_ok=True)

    api = init_shodan()

    print(f"\n[*] Starting Shodan recon for org: {args.org}")
    org_results  = run_org_search(api, args.org, output_dir)
    vuln_hits    = run_vuln_search(api, args.org, output_dir)
    run_title_search(api, args.org, output_dir)
    generate_summary_csv(org_results, vuln_hits, output_dir)

    total_vuln_hosts = sum(len(v) for v in vuln_hits.values())
    autodoc_log(output_dir, args.org, len(org_results), total_vuln_hosts)

    print(f"\n[*] Recon complete. {len(org_results)} hosts, {total_vuln_hosts} vuln hits.")


if __name__ == "__main__":
    main()
```

```bash
# Run the automation script
export SHODAN_API_KEY="your_key_here"
export CENSYS_API_ID="your_id_here"
export CENSYS_API_SECRET="your_secret_here"

python3 .agents/skills/rt-shodan-recon/scripts/shodan_bulk_recon.py \
  --org "Target Corporation" \
  --asn AS12345 \
  --cidr 203.0.113.0/24

# --- Continuous Monitoring (cron-based alert for new exposures) ---
# Install as cron to detect new services appearing on the internet
# Requires: pip3 install shodan diffoscope

# daily_shodan_monitor.sh
#!/bin/bash
DATE=$(date +%Y%m%d)
PREV_DATE=$(date -d "yesterday" +%Y%m%d)
ORG="Target Corporation"
OUT_DIR="_rtexit-output/docs/reconnaissance/monitoring"
mkdir -p "$OUT_DIR"

shodan search --fields ip_str,port,product,version \
  "org:\"$ORG\"" --limit 5000 \
  > "$OUT_DIR/shodan_${DATE}.json"

if [ -f "$OUT_DIR/shodan_${PREV_DATE}.json" ]; then
    diff "$OUT_DIR/shodan_${PREV_DATE}.json" "$OUT_DIR/shodan_${DATE}.json" \
      > "$OUT_DIR/delta_${DATE}.diff"
    echo "[*] Delta saved: $OUT_DIR/delta_${DATE}.diff"
fi

# --- Subdomain Discovery via Certificate Transparency + Shodan ---
# Pull all known certs for the target domain from crt.sh
curl -s "https://crt.sh/?q=%25.targetcorp.com&output=json" \
  | python3 -c "
import json,sys
data = json.load(sys.stdin)
names = set()
for cert in data:
    for name in cert.get('name_value','').split('\n'):
        name = name.strip().lstrip('*.')
        if name:
            names.add(name)
for n in sorted(names):
    print(n)
" > _rtexit-output/docs/reconnaissance/subdomains_crtsh.txt

# Feed discovered subdomains into Shodan
while IFS= read -r subdomain; do
    shodan search --fields ip_str,port,product \
      "hostname:${subdomain}" >> \
      _rtexit-output/docs/reconnaissance/shodan_subdomains.json
done < _rtexit-output/docs/reconnaissance/subdomains_crtsh.txt

# --- Integrate with RTExit autodoc engine ---
python3 _rtexit/scripts/autodoc_engine.py log-activity \
  --phase reconnaissance \
  --tool shodan \
  --description "Shodan internet-wide asset discovery completed" \
  --output-file _rtexit-output/docs/reconnaissance/shodan_recon_autodoc.json
```

**Expert checklist:**
- [ ] Bulk Python automation script executed and results aggregated
- [ ] CVE vulnerability scan run against all known critical CVEs
- [ ] Admin panel / high-value title discovery completed
- [ ] CT log subdomains cross-referenced with Shodan
- [ ] Continuous monitoring cron configured for duration of engagement
- [ ] Autodoc engine log entry written for audit trail
- [ ] Delta diff configured to detect new exposures

---

## Step-by-Step Engagement Workflow

Follow these steps in order at the start of every engagement where external reconnaissance is authorized.

### Step 1 — Environment Setup

```bash
# Create output directories
mkdir -p _rtexit-output/docs/reconnaissance/{shodan,censys,fofa,certs,monitoring}

# Install required tools
pip3 install shodan censys fofa-cli zoomeye-sdk pybinaryedge requests dnspython

# Configure API keys (store in env file, never commit to git)
cat >> ~/.config/rtexit/api_keys.env <<'EOF'
export SHODAN_API_KEY="YOUR_SHODAN_KEY"
export CENSYS_API_ID="YOUR_CENSYS_ID"
export CENSYS_API_SECRET="YOUR_CENSYS_SECRET"
export FOFA_EMAIL="your@email.com"
export FOFA_KEY="YOUR_FOFA_KEY"
export ZOOMEYE_API_KEY="YOUR_ZOOMEYE_KEY"
export BINARYEDGE_KEY="YOUR_BINARYEDGE_KEY"
EOF

source ~/.config/rtexit/api_keys.env

# Initialize Shodan CLI
shodan init "$SHODAN_API_KEY"

# Verify Shodan account plan and remaining query credits
shodan info
```

### Step 2 — Target Profile Collection

```bash
# Collect all known identifiers for the target
TARGET_ORG="Target Corporation"
TARGET_DOMAIN="targetcorp.com"
TARGET_ASN="AS12345"
TARGET_CIDR="203.0.113.0/24"

# Enumerate all IP ranges owned by the target via ARIN/RIPE/APNIC
whois -h whois.arin.net "org:TARGETCORP" | grep -E "^(NetRange|CIDR|OrgName)"

# BGP prefix lookup
curl -s "https://api.bgpview.io/asn/${TARGET_ASN}/prefixes" \
  | python3 -c "
import json,sys
d = json.load(sys.stdin)
for p in d.get('data',{}).get('ipv4_prefixes',[]):
    print(p['prefix'], p.get('description',''))
" > _rtexit-output/docs/reconnaissance/bgp_prefixes.txt

cat _rtexit-output/docs/reconnaissance/bgp_prefixes.txt
```

### Step 3 — Shodan Discovery

```bash
# Run all standard searches and save raw results
TARGET_ORG="Target Corporation"
OUTDIR="_rtexit-output/docs/reconnaissance/shodan"

# Primary org search
shodan search --fields ip_str,port,org,product,version,hostnames,ssl.cert.subject.cn \
  "org:\"${TARGET_ORG}\"" --limit 10000 \
  > "${OUTDIR}/01_org_search.json"

# By domain hostnames
shodan search --fields ip_str,port,product,version,hostnames \
  "hostname:${TARGET_DOMAIN}" --limit 5000 \
  > "${OUTDIR}/02_hostname_search.json"

# By SSL certificate CN
shodan search --fields ip_str,port,ssl.cert.subject.cn,ssl.cert.issuer.cn \
  "ssl.cert.subject.cn:\"${TARGET_DOMAIN}\"" --limit 5000 \
  > "${OUTDIR}/03_ssl_cert_search.json"

# Aggregated stats
shodan stats --facets port,product,country,org "org:\"${TARGET_ORG}\"" \
  > "${OUTDIR}/04_facets.txt"
```

### Step 4 — Censys Certificate Enumeration

```bash
OUTDIR="_rtexit-output/docs/reconnaissance/censys"

# Certificate transparency search
censys search "parsed.names: ${TARGET_DOMAIN}" \
  --index certificates \
  --fields parsed.names,parsed.subject_dn,parsed.validity.start,parsed.validity.end \
  --max-records 5000 \
  > "${OUTDIR}/01_certs_by_domain.json"

# Host enumeration via TLS cert org field
censys search "services.tls.certificates.leaf_data.subject.organization: \"${TARGET_ORG}\"" \
  --index hosts \
  --fields ip,services.port,services.service_name,services.software \
  > "${OUTDIR}/02_hosts_by_org_tls.json"

# Extract unique subdomains from cert data
python3 -c "
import json
with open('${OUTDIR}/01_certs_by_domain.json') as f:
    data = json.load(f)
names = set()
for cert in data.get('result',{}).get('hits',[]):
    for name in cert.get('parsed.names',[]):
        names.add(name.lstrip('*.'))
for n in sorted(names):
    print(n)
" > _rtexit-output/docs/reconnaissance/certs/unique_subdomains.txt

echo "[*] Unique subdomains discovered:"
wc -l < _rtexit-output/docs/reconnaissance/certs/unique_subdomains.txt
```

### Step 5 — FOFA Cross-Validation

```bash
OUTDIR="_rtexit-output/docs/reconnaissance/fofa"

# By certificate
fofa search --fields "ip,port,title,country,city,protocol,server,cert" \
  "cert=\"${TARGET_DOMAIN}\"" \
  --size 1000 --format json \
  > "${OUTDIR}/01_fofa_cert.json"

# By organization
fofa search --fields "ip,port,title,org,country" \
  "org=\"${TARGET_ORG}\"" \
  --size 1000 --format json \
  > "${OUTDIR}/02_fofa_org.json"

# Find Confluence/Jira (common data targets)
fofa search --fields "ip,port,title,country" \
  "cert=\"${TARGET_DOMAIN}\" && (title=\"Jira\" || title=\"Confluence\")" \
  --format json > "${OUTDIR}/03_fofa_jira_confluence.json"
```

### Step 6 — Vulnerability Assessment

```bash
# Run CVE-targeted Shodan queries
OUTDIR="_rtexit-output/docs/reconnaissance/shodan"
CVES=(
  "CVE-2021-44228" "CVE-2021-26855" "CVE-2019-19781"
  "CVE-2022-26134" "CVE-2023-23397" "CVE-2022-22965"
  "CVE-2021-21985" "CVE-2020-5902"  "CVE-2021-34473"
  "CVE-2023-20198" "CVE-2024-21762"
)

for CVE in "${CVES[@]}"; do
  COUNT=$(shodan count "org:\"${TARGET_ORG}\" vuln:${CVE}")
  if [ "$COUNT" -gt 0 ]; then
    echo "[CRITICAL] ${CVE}: ${COUNT} host(s) potentially vulnerable"
    shodan search --fields ip_str,port,product,version \
      "org:\"${TARGET_ORG}\" vuln:${CVE}" \
      > "${OUTDIR}/vuln_${CVE}.json"
  fi
done
```

### Step 7 — Aggregate and Generate Report

```bash
# Run the Python automation script for full aggregation
python3 .agents/skills/rt-shodan-recon/scripts/shodan_bulk_recon.py \
  --org "${TARGET_ORG}" \
  --asn "${TARGET_ASN}" \
  --out "_rtexit-output/docs/reconnaissance"

# Log activity to RTExit autodoc engine
python3 _rtexit/scripts/autodoc_engine.py log-activity \
  --phase reconnaissance \
  --tool "shodan,censys,fofa" \
  --description "Passive internet-wide asset discovery via scan databases" \
  --output-file _rtexit-output/docs/reconnaissance/shodan_recon_autodoc.json \
  --notes "No packets sent to target — fully passive"
```

### Step 8 — Feed Discovered Assets into Active Recon

```bash
# Extract unique IPs from all Shodan results for nmap (active — confirm in-scope first)
python3 -c "
import json, glob
ips = set()
for f in glob.glob('_rtexit-output/docs/reconnaissance/shodan/*.json'):
    try:
        with open(f) as fh:
            data = json.load(fh)
        if isinstance(data, list):
            for item in data:
                ip = item.get('ip_str') or item.get('ip')
                if ip: ips.add(ip)
    except: pass
for ip in sorted(ips):
    print(ip)
" > _rtexit-output/docs/reconnaissance/all_discovered_ips.txt

echo "[*] Total unique IPs discovered: $(wc -l < _rtexit-output/docs/reconnaissance/all_discovered_ips.txt)"

# Use discovered subdomains for active DNS resolution
cat _rtexit-output/docs/reconnaissance/certs/unique_subdomains.txt \
  | dnsx -silent -a -resp \
  > _rtexit-output/docs/reconnaissance/resolved_subdomains.txt
```

---

## All Commands Reference

| Command | Purpose |
|---------|---------|
| `shodan init KEY` | Initialize Shodan CLI with API key |
| `shodan info` | Show account plan, credits remaining |
| `shodan count "query"` | Count results without spending query credits |
| `shodan search "query"` | Search with default output |
| `shodan search --fields a,b "query"` | Search with selected fields |
| `shodan search --limit N "query"` | Limit result count |
| `shodan host IP` | Full banner data for one IP |
| `shodan host --history IP` | Historical banners for one IP |
| `shodan stats --facets port,product "query"` | Aggregated statistics |
| `shodan download FILE "query"` | Download results to gzipped JSON |
| `shodan parse --fields a,b FILE.json.gz` | Parse downloaded file |
| `shodan alert create NAME cidr` | Set up continuous monitoring alert |
| `censys search "query" --index hosts` | Search Censys hosts index |
| `censys search "query" --index certificates` | Search Censys certs index |
| `fofa search "query" --format json` | FOFA search with JSON output |
| `zoomeye search "query"` | ZoomEye search |
| `binaryedge host IP` | BinaryEdge host lookup |

**Shodan Filter Reference:**

| Filter | Example | Purpose |
|--------|---------|---------|
| `org:` | `org:"Target Corp"` | Organization name in WHOIS |
| `hostname:` | `hostname:targetcorp.com` | DNS hostname in banner |
| `net:` | `net:203.0.113.0/24` | CIDR range |
| `asn:` | `asn:AS12345` | Autonomous System Number |
| `port:` | `port:3389` | Service port |
| `product:` | `product:MongoDB` | Product name from banner |
| `version:` | `version:2.4.49` | Software version |
| `http.title:` | `http.title:"Admin"` | HTTP page title |
| `http.html:` | `http.html:"password"` | HTTP page body content |
| `ssl.cert.subject.cn:` | `ssl.cert.subject.cn:"*.targetcorp.com"` | TLS cert CN field |
| `vuln:` | `vuln:CVE-2021-44228` | Known CVE (Membership+) |
| `tag:` | `tag:ics` | Shodan category tag |
| `country:` | `country:US` | Country code |
| `city:` | `city:"New York"` | City |

---

## Tools Referenced

| Tool | URL | Install |
|------|-----|---------|
| Shodan CLI | https://github.com/achillean/shodan-python | `pip3 install shodan` |
| Censys Python SDK | https://github.com/censys/censys-python | `pip3 install censys` |
| FOFA CLI | https://github.com/LubyRuffy/fofa-tools | `pip3 install fofa-cli` |
| ZoomEye SDK | https://github.com/knownsec/ZoomEye-python | `pip3 install zoomeye-sdk` |
| BinaryEdge SDK | https://github.com/binaryedge/python-binaryedge | `pip3 install pybinaryedge` |
| dnsx | https://github.com/projectdiscovery/dnsx | `go install github.com/projectdiscovery/dnsx/cmd/dnsx@latest` |
| subfinder | https://github.com/projectdiscovery/subfinder | `go install github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest` |
| amass | https://github.com/owasp-amass/amass | `go install github.com/owasp-amass/amass/v4/...@master` |
| httpx | https://github.com/projectdiscovery/httpx | `go install github.com/projectdiscovery/httpx/cmd/httpx@latest` |
| crt.sh (web) | https://crt.sh | Web UI + API (no key required) |
| BGPView API | https://bgpview.io/api | REST API (no key required) |
| ARIN WHOIS | https://search.arin.net | Web + CLI WHOIS |

**SecLists references (used in downstream active recon after Shodan discovery):**
- `/usr/share/seclists/Discovery/DNS/subdomains-top1million-110000.txt` — subdomain brute force
- `/usr/share/seclists/Discovery/Web-Content/common.txt` — web path discovery
- `/usr/share/seclists/Passwords/Default-Credentials/default-passwords.csv` — default credential checks

---

## Output Instructions

All output files must be saved in the RTExit standard output tree:

```
_rtexit-output/docs/reconnaissance/
├── shodan/
│   ├── 01_org_search.json          # Full org search results
│   ├── 02_hostname_search.json     # Hostname-based results
│   ├── 03_ssl_cert_search.json     # SSL cert CN results
│   ├── 04_facets.txt               # Port/product distribution stats
│   ├── shodan_summary.csv          # Operator-friendly CSV for reporting
│   └── vuln_CVE-XXXX-XXXX.json    # Per-CVE vulnerability hits
├── censys/
│   ├── 01_certs_by_domain.json     # All certs issued to target domain
│   └── 02_hosts_by_org_tls.json    # Hosts with org name in TLS cert
├── fofa/
│   ├── 01_fofa_cert.json           # FOFA cert-based results
│   └── 02_fofa_org.json            # FOFA org-based results
├── certs/
│   └── unique_subdomains.txt       # De-duplicated subdomain list from CT logs
├── monitoring/
│   ├── shodan_YYYYMMDD.json        # Daily snapshots for delta comparison
│   └── delta_YYYYMMDD.diff         # New exposures since previous day
├── bgp_prefixes.txt                # All BGP-announced prefixes for target ASN
├── subdomains_crtsh.txt            # Subdomains from crt.sh CT logs
├── all_discovered_ips.txt          # Aggregated unique IPs for active recon
├── resolved_subdomains.txt         # DNS-resolved subdomains with IPs
└── shodan_recon_autodoc.json       # RTExit autodoc engine activity log
```

**Autodoc engine integration:**

```bash
# The autodoc log entry format for rt-shodan-recon:
{
  "skill": "rt-shodan-recon",
  "timestamp": "2026-05-31T10:00:00Z",
  "target_org": "Target Corporation",
  "hosts_discovered": 342,
  "vuln_cve_hits": 7,
  "output_files": [...]
}
```

**Naming conventions:**
- All files use snake_case
- CIDR-specific files include the network: `shodan_cidr_203.0.113.json`
- ASN-specific files include the ASN: `shodan_asn_AS12345.json`
- Vulnerability files include the CVE: `vuln_CVE-2021-44228.json`
- Monitoring snapshots include the date: `shodan_20260531.json`

---

## Operational Security Notes

- Shodan, Censys, FOFA queries do NOT contact target systems — all data comes from the scan database's own crawlers
- Your source IP is never seen by the target when querying these APIs
- Shodan query logs are stored by Shodan — avoid including operator-identifying strings in queries if OPSEC requires it
- API keys should be stored in `~/.config/rtexit/api_keys.env` — never commit to source control, never paste into reports
- Shodan Membership plan is minimum required for `vuln:` filter and full banner data downloads
- FOFA requires a paid account for bulk exports; free tier limited to 10 results
- Document all query timestamps in the autodoc log — required for legal chain of custody

---

## Resources

| Resource | URL |
|----------|-----|
| Shodan Search Reference | https://www.shodan.io/search/filters |
| Shodan Dorks Collection | https://github.com/jakejarvis/awesome-shodan-queries |
| Shodan Python Docs | https://shodan.readthedocs.io |
| Censys Search Docs | https://search.censys.io/search/help |
| Censys Python Docs | https://censys-python.readthedocs.io |
| FOFA Search Syntax | https://en.fofa.info/help |
| ZoomEye Dorks | https://github.com/daffainfo/AllAboutBugBounty/blob/master/Misc/ZoomEye%20Dorks.md |
| Shodan Dorks GitHub | https://github.com/lothos612/shodan |
| Awesome Shodan | https://github.com/jakejarvis/awesome-shodan-queries |
| Censys Universal Internet Dataset | https://censys.io/data |
| BGPView ASN Lookup | https://bgpview.io |
| crt.sh Certificate Transparency | https://crt.sh |
| ARIN IP Lookup | https://search.arin.net |
| RIPE NCC Database | https://apps.db.ripe.net |
| BinaryEdge Docs | https://docs.binaryedge.io |
| SecLists DNS Wordlists | https://github.com/danielmiessler/SecLists/tree/master/Discovery/DNS |
| OWASP Amass Docs | https://github.com/owasp-amass/amass/blob/master/doc/user_guide.md |
| ProjectDiscovery Tools | https://github.com/projectdiscovery |
