# DAST Reconnaissance Module

> **Module**: pentest-dast-recon
> **Version**: 1.0.0
> **Source**: DAST Scanner v5 - crawler/*.py + fingerprint/*.py
> **Capacites**: Crawler, OpenAPI, GraphQL, Fingerprinting, JS Analysis

---

## Vue d'Ensemble

Ce module fournit les capacites de reconnaissance avancee:
- Crawling intelligent (BFS avec scope enforcement)
- Parsing OpenAPI/Swagger
- Introspection GraphQL
- Analyse JavaScript client-side
- Detection de technologies

---

## 1. WEB CRAWLER INTELLIGENT

### 1.1 Configuration
```yaml
crawler:
  algorithm: "breadth_first"
  max_depth: 10
  max_pages: 1000
  timeout: 30  # secondes par page

  scope:
    include_patterns:
      - "https://target.com/*"
      - "https://api.target.com/*"

    exclude_patterns:
      - "*/logout*"
      - "*/signout*"
      - "*?action=delete*"
      - "*.pdf"
      - "*.zip"

  rate_limiting:
    requests_per_second: 10
    delay_between_requests: 100  # ms
```

### 1.2 Extraction de Liens
```yaml
link_extraction:
  html_sources:
    - tag: "a"
      attribute: "href"
    - tag: "form"
      attribute: "action"
    - tag: "script"
      attribute: "src"
    - tag: "link"
      attribute: "href"
    - tag: "img"
      attribute: "src"
    - tag: "iframe"
      attribute: "src"
    - tag: "area"
      attribute: "href"

  javascript_patterns:
    fetch_calls:
      - 'fetch\s*\(\s*["\']([^"\']+)["\']'
      - 'fetch\s*\(\s*`([^`]+)`'

    axios_calls:
      - 'axios\.(get|post|put|delete|patch)\s*\(\s*["\']([^"\']+)["\']'
      - 'axios\s*\(\s*\{[^}]*url:\s*["\']([^"\']+)["\']'

    xhr_calls:
      - 'XMLHttpRequest.*open\s*\([^,]+,\s*["\']([^"\']+)["\']'

    spa_routes:
      - 'path:\s*["\']([^"\']+)["\']'
      - 'route:\s*["\']([^"\']+)["\']'
      - 'to:\s*["\']([^"\']+)["\']'
```

### 1.3 Extraction de Formulaires
```yaml
form_extraction:
  collect:
    - action_url
    - method (GET/POST)
    - enctype
    - input_fields:
        - name
        - type
        - value
        - required
        - pattern
    - select_fields
    - textarea_fields
    - hidden_fields
    - csrf_tokens

  identify_types:
    login_form:
      indicators:
        - "password type input"
        - "login/signin in action"
    registration_form:
      indicators:
        - "email + password + confirm"
    search_form:
      indicators:
        - "search in name/id"
        - "GET method"
    upload_form:
      indicators:
        - "file type input"
        - "multipart/form-data"
```

---

## 2. OPENAPI/SWAGGER PARSER

### 2.1 Detection Endpoints
```yaml
openapi_discovery:
  common_paths:
    - "/openapi.json"
    - "/swagger.json"
    - "/api-docs"
    - "/api/docs"
    - "/docs"
    - "/swagger"
    - "/swagger-ui.html"
    - "/swagger-ui/"
    - "/api/swagger.json"
    - "/v1/api-docs"
    - "/v2/api-docs"
    - "/v3/api-docs"
```

### 2.2 Parsing Swagger 2.0
```yaml
swagger_2_parsing:
  extract:
    base_path: "$.basePath"
    host: "$.host"
    schemes: "$.schemes"

    paths: "$.paths"
    # Pour chaque path:
    #   - methods (get, post, put, delete, patch)
    #   - parameters (query, path, body, header)
    #   - consumes
    #   - produces
    #   - security

    definitions: "$.definitions"
    # Schemas pour body parameters

  generate_requests:
    # Pour chaque endpoint:
    1. Construire l'URL complete
    2. Ajouter les parametres obligatoires
    3. Generer des valeurs de test
    4. Ajouter les headers (Content-Type, auth)
```

### 2.3 Parsing OpenAPI 3.x
```yaml
openapi_3_parsing:
  extract:
    servers: "$.servers"
    # Liste des serveurs avec URL

    paths: "$.paths"
    # Pour chaque path:
    #   - operationId
    #   - parameters
    #   - requestBody (avec schema)
    #   - responses
    #   - security

    components:
      schemas: "$.components.schemas"
      securitySchemes: "$.components.securitySchemes"
      parameters: "$.components.parameters"

  resolve_refs:
    # Resoudre les $ref vers les schemas
    "$ref": "#/components/schemas/User"
    # -> Charger le schema User
```

### 2.4 Generation de Valeurs de Test
```yaml
test_value_generation:
  by_type:
    string: "test_string"
    integer: 1
    number: 1.5
    boolean: true
    array: ["item1", "item2"]
    object: {"key": "value"}

  by_format:
    email: "test@example.com"
    date: "2024-01-15"
    date-time: "2024-01-15T10:30:00Z"
    uuid: "550e8400-e29b-41d4-a716-446655440000"
    uri: "https://example.com"
    password: "TestPassword123!"

  by_name_pattern:
    "*id*": 1
    "*email*": "test@example.com"
    "*phone*": "+33612345678"
    "*url*": "https://example.com"
    "*date*": "2024-01-15"
    "*password*": "TestPassword123!"
    "*token*": "test_token_value"
```

---

## 3. GRAPHQL INTROSPECTION

### 3.1 Detection GraphQL
```yaml
graphql_discovery:
  common_endpoints:
    - "/graphql"
    - "/graphql/v1"
    - "/api/graphql"
    - "/query"
    - "/gql"

  detection_methods:
    content_type:
      - "application/graphql"

    response_patterns:
      - '"data":'
      - '"errors":'
      - "__schema"
```

### 3.2 Introspection Query
```graphql
query IntrospectionQuery {
  __schema {
    queryType { name }
    mutationType { name }
    subscriptionType { name }
    types {
      ...FullType
    }
    directives {
      name
      description
      locations
      args {
        ...InputValue
      }
    }
  }
}

fragment FullType on __Type {
  kind
  name
  description
  fields(includeDeprecated: true) {
    name
    description
    args {
      ...InputValue
    }
    type {
      ...TypeRef
    }
    isDeprecated
    deprecationReason
  }
  inputFields {
    ...InputValue
  }
  interfaces {
    ...TypeRef
  }
  enumValues(includeDeprecated: true) {
    name
    description
    isDeprecated
    deprecationReason
  }
  possibleTypes {
    ...TypeRef
  }
}

fragment InputValue on __InputValue {
  name
  description
  type {
    ...TypeRef
  }
  defaultValue
}

fragment TypeRef on __Type {
  kind
  name
  ofType {
    kind
    name
    ofType {
      kind
      name
      ofType {
        kind
        name
        ofType {
          kind
          name
          ofType {
            kind
            name
            ofType {
              kind
              name
            }
          }
        }
      }
    }
  }
}
```

### 3.3 Analyse du Schema
```yaml
schema_analysis:
  extract:
    queries:
      # Toutes les queries disponibles
      - name
      - arguments
      - return_type

    mutations:
      # Toutes les mutations disponibles
      - name
      - arguments
      - return_type

    types:
      # Tous les types custom
      - name
      - fields
      - relationships

  identify_sensitive:
    # Types/champs potentiellement sensibles
    patterns:
      - "*password*"
      - "*token*"
      - "*secret*"
      - "*key*"
      - "*admin*"
      - "*private*"
      - "*internal*"

  generate_queries:
    # Generer des queries de test
    for_each_query:
      - avec arguments valides
      - avec arguments invalides
      - avec arguments d'injection
```

### 3.4 Tests de Securite GraphQL
```yaml
graphql_security_tests:
  introspection_disabled:
    # Verifier si introspection est desactivee en prod
    test: "Envoyer introspection query"
    expected: "Erreur ou schema vide"

  query_depth_limit:
    # Tester les limites de profondeur
    payload: |
      { user { friends { friends { friends { name } } } } }
    expected: "Erreur de profondeur max"

  batch_attacks:
    # Alias attacks pour bypass rate limiting
    payload: |
      query {
        a: user(id: 1) { name }
        b: user(id: 2) { name }
        c: user(id: 3) { name }
        # ... 100 aliases
      }

  injection:
    # Injection dans les arguments
    payloads:
      - 'query { user(id: "1 OR 1=1") { name } }'
      - 'query { user(name: "<script>alert(1)</script>") { name } }'

  idor:
    # Acces non autorise
    test: "Acceder aux donnees d'autres users via ID"
```

---

## 4. JAVASCRIPT ANALYZER

### 4.1 Extraction depuis JS
```yaml
js_analysis:
  endpoints:
    patterns:
      - 'fetch\(["\']([^"\']+)["\']'
      - 'axios\.(get|post)\(["\']([^"\']+)["\']'
      - 'url:\s*["\']([^"\']+)["\']'
      - 'endpoint:\s*["\']([^"\']+)["\']'
      - '/api/[a-zA-Z0-9/_-]+'

  api_keys:
    patterns:
      - 'api[_-]?key["\']?\s*[:=]\s*["\']([^"\']+)["\']'
      - 'apiKey:\s*["\']([^"\']+)["\']'
      - 'authorization["\']?\s*[:=]\s*["\']Bearer ([^"\']+)["\']'

  secrets:
    patterns:
      - 'secret["\']?\s*[:=]\s*["\']([^"\']+)["\']'
      - 'password["\']?\s*[:=]\s*["\']([^"\']+)["\']'
      - 'token["\']?\s*[:=]\s*["\']([^"\']+)["\']'
      - 'private[_-]?key["\']?\s*[:=]\s*["\']([^"\']+)["\']'

  routes:
    patterns:
      - 'path:\s*["\']([^"\']+)["\']'
      - 'route:\s*["\']([^"\']+)["\']'
      - 'component:\s*["\']([^"\']+)["\']'
```

### 4.2 Source Maps
```yaml
source_maps:
  discovery:
    # Chercher les source maps
    - "*.js.map"
    - "sourceMappingURL="

  extraction:
    # Si source map trouvee:
    - Telecharger le .map
    - Extraire le code source original
    - Analyser pour secrets et endpoints

  security_risk: |
    Les source maps exposent le code source original.
    Risque MOYEN - Information disclosure.
```

### 4.3 Detection Vulnerabilites Client-Side
```yaml
client_side_vulns:
  dom_xss_sinks:
    dangerous:
      - "eval("
      - "setTimeout("
      - "setInterval("
      - "Function("

    html:
      - ".innerHTML"
      - ".outerHTML"
      - "document.write("

    navigation:
      - "location ="
      - "location.href ="

  insecure_storage:
    patterns:
      - "localStorage.setItem"
      - "sessionStorage.setItem"
    check: "Stockage de tokens/secrets?"

  postmessage:
    patterns:
      - "postMessage("
      - "addEventListener.*message"
    check: "Origin verification?"

  cors_bypass:
    patterns:
      - "Access-Control-Allow-Origin: *"
```

---

## 5. FINGERPRINTING AVANCE

### 5.1 Technologies Detection
```yaml
tech_fingerprinting:
  frontend:
    react:
      - "_reactRootContainer"
      - "data-reactroot"
      - "__REACT_DEVTOOLS"

    vue:
      - "__VUE__"
      - "data-v-[a-f0-9]"

    angular:
      - 'ng-version="'
      - "[ngClass]"

    jquery:
      - "jQuery"
      - "$.ajax"

  backend:
    express:
      - "X-Powered-By: Express"

    django:
      - "csrfmiddlewaretoken"
      - "sessionid cookie"

    laravel:
      - "laravel_session"
      - "XSRF-TOKEN"

    spring:
      - "JSESSIONID"
      - "/actuator"

    rails:
      - "_session_id"
      - "X-Runtime"

    nestjs:
      - "@nestjs"

  cms:
    wordpress:
      - "/wp-content/"
      - "/wp-admin/"

    drupal:
      - "Drupal.settings"
      - "X-Drupal-Cache"

  servers:
    nginx:
      - "Server: nginx"

    apache:
      - "Server: Apache"

    iis:
      - "Server: Microsoft-IIS"
```

### 5.2 Version Detection
```yaml
version_detection:
  methods:
    headers:
      - "Server: nginx/1.18.0"
      - "X-Powered-By: PHP/7.4.3"

    meta_tags:
      - '<meta name="generator" content="WordPress 5.8">'

    comments:
      - "<!-- jQuery v3.6.0 -->"

    file_hashes:
      # Comparer hash de fichiers statiques
      - "/static/js/main.js -> known version hash"

    error_pages:
      # Versions dans les pages d'erreur
      - "Apache/2.4.41 (Ubuntu)"
```

### 5.3 SSL/TLS Scanner
```yaml
ssl_scanner:
  checks:
    certificate:
      - validity_dates
      - issuer
      - subject
      - san_domains
      - key_size
      - signature_algorithm

    protocols:
      vulnerable:
        - SSLv2
        - SSLv3
        - TLSv1.0
        - TLSv1.1
      secure:
        - TLSv1.2
        - TLSv1.3

    ciphers:
      weak:
        - RC4
        - DES
        - 3DES
        - NULL
        - EXPORT
        - MD5

    vulnerabilities:
      - BEAST
      - POODLE
      - Heartbleed
      - FREAK
      - Logjam
      - DROWN
```

### 5.4 Port Scanner
```yaml
port_scanner:
  common_ports:
    web:
      - 80   # HTTP
      - 443  # HTTPS
      - 8080 # Alt HTTP
      - 8443 # Alt HTTPS

    database:
      - 3306 # MySQL
      - 5432 # PostgreSQL
      - 27017 # MongoDB
      - 6379 # Redis
      - 9200 # Elasticsearch

    services:
      - 21   # FTP
      - 22   # SSH
      - 25   # SMTP
      - 53   # DNS
      - 110  # POP3
      - 143  # IMAP
      - 389  # LDAP

    management:
      - 8000 # Django dev
      - 3000 # Node.js
      - 5000 # Flask
      - 9090 # Prometheus
      - 15672 # RabbitMQ

  scan_types:
    tcp_connect: "Full TCP handshake"
    syn_scan: "Half-open scan (requires root)"
```

---

## 6. DNS ENUMERATION

### 6.1 Subdomain Discovery
```yaml
subdomain_enum:
  methods:
    brute_force:
      wordlist:
        - "www"
        - "api"
        - "admin"
        - "dev"
        - "staging"
        - "test"
        - "beta"
        - "app"
        - "mail"
        - "ftp"
        - "cdn"
        - "static"
        - "assets"
        - "images"
        - "portal"
        - "dashboard"
        - "internal"
        - "vpn"

    dns_records:
      - A
      - AAAA
      - CNAME
      - MX
      - NS
      - TXT
      - SRV

    certificate_transparency:
      # Chercher dans les logs CT
      sources:
        - "crt.sh"
        - "censys.io"
        - "certspotter"

    search_engines:
      - "site:*.target.com"
```

### 6.2 Virtual Host Discovery
```yaml
vhost_discovery:
  method: |
    1. Obtenir l'IP du serveur
    2. Tester differents Host headers
    3. Comparer les reponses

  test_hosts:
    - "internal.target.com"
    - "admin.target.com"
    - "dev.target.com"
    - "localhost"

  detection:
    different_content: "Nouveau vhost trouve"
    same_content: "Meme site"
    error: "Vhost n'existe pas"
```

---

## 7. CONTRACT EXTRACTION

### 7.1 Backend Analysis
```yaml
backend_contract_extraction:
  frameworks:
    nestjs:
      patterns:
        controllers:
          - "@Controller\\(['\"]([^'\"]+)['\"]\\)"
          - "@Get\\(['\"]?([^'\"\\)]*)['\"]?\\)"
          - "@Post\\(['\"]?([^'\"\\)]*)['\"]?\\)"
          - "@Put\\(['\"]?([^'\"\\)]*)['\"]?\\)"
          - "@Delete\\(['\"]?([^'\"\\)]*)['\"]?\\)"
          - "@Patch\\(['\"]?([^'\"\\)]*)['\"]?\\)"

        params:
          - "@Param\\(['\"]([^'\"]+)['\"]\\)"
          - "@Query\\(['\"]([^'\"]+)['\"]\\)"
          - "@Body\\(\\)"

        guards:
          - "@UseGuards\\(([^)]+)\\)"

    express:
      patterns:
        - "app\\.(get|post|put|delete|patch)\\(['\"]([^'\"]+)['\"]"
        - "router\\.(get|post|put|delete|patch)\\(['\"]([^'\"]+)['\"]"

    django:
      patterns:
        - "path\\(['\"]([^'\"]+)['\"]"
        - "url\\(r['\"]([^'\"]+)['\"]"

    spring:
      patterns:
        - "@RequestMapping\\(['\"]([^'\"]+)['\"]\\)"
        - "@GetMapping\\(['\"]([^'\"]+)['\"]\\)"
        - "@PostMapping\\(['\"]([^'\"]+)['\"]\\)"

    laravel:
      patterns:
        - "Route::(get|post|put|delete|patch)\\(['\"]([^'\"]+)['\"]"
```

### 7.2 Cross-Correlation
```yaml
cross_correlation:
  sources:
    - frontend_js_calls
    - backend_routes
    - openapi_spec
    - runtime_requests

  output:
    # Pour chaque endpoint:
    endpoint:
      path: "/api/users/{id}"
      method: "GET"
      found_in:
        - frontend: "fetch('/api/users/' + id)"
        - backend: "@Get(':id')"
        - openapi: "paths./api/users/{id}.get"
      params:
        - name: "id"
          type: "path"
          required: true
      auth_required: true
      rate_limited: false
```

---

## 8. WORKFLOW RECONNAISSANCE

```
┌─────────────────────────────────────────────────────────────┐
│                  WORKFLOW RECONNAISSANCE                     │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  1. DNS ENUMERATION                                         │
│     └─ Subdomains discovery                                 │
│     └─ Virtual hosts detection                              │
│                                                              │
│  2. PORT SCANNING                                           │
│     └─ Common web ports                                     │
│     └─ Database ports                                       │
│     └─ Service ports                                        │
│                                                              │
│  3. TECHNOLOGY FINGERPRINTING                               │
│     └─ Frontend framework                                   │
│     └─ Backend framework                                    │
│     └─ Web server                                           │
│     └─ CMS detection                                        │
│                                                              │
│  4. SSL/TLS ANALYSIS                                        │
│     └─ Certificate info                                     │
│     └─ Protocol versions                                    │
│     └─ Cipher suites                                        │
│                                                              │
│  5. WEB CRAWLING                                            │
│     └─ HTML link extraction                                 │
│     └─ Form extraction                                      │
│     └─ JavaScript analysis                                  │
│                                                              │
│  6. API DISCOVERY                                           │
│     └─ OpenAPI/Swagger parsing                              │
│     └─ GraphQL introspection                                │
│     └─ Backend route extraction                             │
│                                                              │
│  7. CROSS-CORRELATION                                       │
│     └─ Frontend <-> Backend                                 │
│     └─ Docs <-> Runtime                                     │
│                                                              │
│  8. RAPPORT RECON                                           │
│     └─ Liste complete des endpoints                         │
│     └─ Technologies detectees                               │
│     └─ Surface d'attaque                                    │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

---

## 9. EXEMPLE DE RAPPORT RECON

```markdown
## Reconnaissance Report

### Target: https://target.com

### DNS/Infrastructure
| Domain | IP | Type |
|--------|-----|------|
| target.com | 1.2.3.4 | A |
| api.target.com | 1.2.3.5 | A |
| admin.target.com | 1.2.3.4 | CNAME |

### Open Ports
| Port | Service | Version |
|------|---------|---------|
| 443 | HTTPS | nginx/1.18.0 |
| 8080 | HTTP | Node.js |

### Technologies
| Layer | Technology | Version |
|-------|------------|---------|
| Frontend | React | 18.x |
| Backend | NestJS | 9.x |
| Database | PostgreSQL | 14.x |
| Server | Nginx | 1.18.0 |
| CDN | Cloudflare | - |

### SSL/TLS
| Attribute | Value |
|-----------|-------|
| Protocol | TLSv1.3 |
| Certificate | Valid until 2025-01-01 |
| Key Size | 2048 bits |
| Issues | None |

### Endpoints Discovered
| Method | Path | Source | Auth |
|--------|------|--------|------|
| GET | /api/users | OpenAPI | Yes |
| POST | /api/auth/login | Crawler | No |
| GET | /api/orders/{id} | JS Analysis | Yes |
| GraphQL | /graphql | Introspection | Yes |

### Forms Found
| URL | Method | Fields | Type |
|-----|--------|--------|------|
| /login | POST | email, password | Login |
| /register | POST | email, password, name | Registration |
| /search | GET | q | Search |

### Attack Surface Summary
- Total endpoints: 45
- Authenticated: 38
- Public: 7
- GraphQL queries: 15
- GraphQL mutations: 8
- Forms: 5
- File uploads: 1
```

---

*Module genere depuis DAST Scanner v5 - crawler/*.py + fingerprint/*.py*
*Crawler intelligent, OpenAPI, GraphQL, Fingerprinting, JS Analysis*
