<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: implementing-template-output-extraction
description: >-
  Implement data extraction in Nuclei templates using regex, JSON, XPath,
  KVal, and DSL extractors. Covers named extractors for template chaining,
  internal extractors for flow control, multi-part extraction, and
  structured output formatting for automated processing.
domain: cybersecurity
subdomain: nuclei-templating
tags:
  - nuclei
  - extraction
  - output
  - regex
  - json
  - templates
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: []
---

# Implementing Template Output Extraction

## Overview

Output extraction transforms raw scan results into structured, actionable data.
Nuclei extractors pull specific values from responses — IP addresses, version
strings, tokens, and configuration data — enabling automated triage, chained
template workflows, and integration with downstream security tools. Mastering
extractors is essential for building templates that produce machine-readable
findings.

Mode: `[MODE: BLUE]` — Structured output for automated security processing.

## Prerequisites

| Requirement | Details |
|---|---|
| Nuclei v3+ (`nuclei -version`) | Required |
| Understanding of regex, JSON path, and XPath query syntax | Required |
| Target application producing structured responses | Required |

## Key Concepts

### Regex Extraction with Capture Groups

```yaml
id: server-version-extraction

info:
  name: Server Version Header Extraction
  author: nerf
  severity: info
  description: |
    Extracts web server software and version from response headers.
    Version information enables targeted vulnerability scanning and
    EOL software identification.
  tags: extraction,version,recon,header
  metadata:
    max-request: 1

http:
  - method: GET
    path:
      - "{{BaseURL}}/"
    matchers:
      - type: status
        status:
          - 200
          - 301
          - 302
          - 403
          - 404
    extractors:
      - type: regex
        part: header
        name: server-version
        regex:
          - '(?i)server:\s*(.+)'
        group: 1
      - type: regex
        part: header
        name: powered-by
        regex:
          - '(?i)x-powered-by:\s*(.+)'
        group: 1
```

### JSON Path Extraction

```yaml
id: api-user-data-extraction

info:
  name: API User Data Structure Extraction
  author: nerf
  severity: info
  description: |
    Extracts user data field names from API responses to map the
    data schema and identify sensitive fields exposed by endpoints.
  tags: extraction,api,json,schema
  metadata:
    max-request: 1

http:
  - method: GET
    path:
      - "{{BaseURL}}/api/v1/users"
    matchers:
      - type: status
        status:
          - 200
    extractors:
      - type: json
        part: body
        name: user-fields
        json:
          - '.data[0] | keys[]'
      - type: json
        part: body
        name: user-count
        json:
          - '.data | length'
```

### KVal Header Extraction

```yaml
id: security-header-extraction

info:
  name: Security Response Header Extraction
  author: nerf
  severity: info
  description: |
    Extracts security-related HTTP response headers for compliance
    auditing and hardening analysis.
  tags: extraction,headers,security,audit
  metadata:
    max-request: 1

http:
  - method: GET
    path:
      - "{{BaseURL}}/"
    matchers:
      - type: status
        status:
          - 200
    extractors:
      - type: kval
        name: security-headers
        kval:
          - content-security-policy
          - strict-transport-security
          - x-frame-options
          - x-content-type-options
          - x-xss-protection
          - referrer-policy
          - permissions-policy
```

### DSL Expression Extraction

```yaml
id: response-analysis-extraction

info:
  name: Response Analysis Data Extraction
  author: nerf
  severity: info
  description: |
    Extracts computed values from responses using DSL expressions
    including response size, timing, and content analysis metrics.
  tags: extraction,dsl,analysis,metrics
  metadata:
    max-request: 1

http:
  - method: GET
    path:
      - "{{BaseURL}}/"
    matchers:
      - type: status
        status:
          - 200
    extractors:
      - type: dsl
        name: response-metrics
        dsl:
          - '"Size: " + to_string(content_length)'
          - '"Status: " + to_string(status_code)'
          - '"Type: " + content_type'
          - '"Server: " + server'
```

### Internal Extractors for Template Chaining

```yaml
id: chained-extraction-example

info:
  name: Chained Request with Internal Extraction
  author: nerf
  severity: medium
  description: |
    Demonstrates internal extractors that pass values between
    requests within a template. First request extracts a CSRF
    token, second request uses it in a form submission.
  tags: extraction,internal,chained,csrf
  metadata:
    max-request: 2

http:
  - method: GET
    path:
      - "{{BaseURL}}/login"
    extractors:
      - type: regex
        name: csrf_token
        part: body
        regex:
          - 'name="csrf_token"\s+value="([a-f0-9]+)"'
        group: 1
        internal: true

  - method: POST
    path:
      - "{{BaseURL}}/login"
    headers:
      Content-Type: application/x-www-form-urlencoded
    body: "username=test&password=test&csrf_token={{csrf_token}}"
    matchers:
      - type: word
        part: body
        words:
          - "dashboard"
          - "welcome"
        condition: or
    extractors:
      - type: regex
        name: session-id
        part: header
        regex:
          - 'session=([a-f0-9]+)'
        group: 1
```

### XPath Extraction from HTML

```yaml
id: html-metadata-extraction

info:
  name: HTML Metadata Extraction
  author: nerf
  severity: info
  description: |
    Extracts metadata from HTML pages using XPath selectors including
    title, meta descriptions, Open Graph tags, and generator info.
  tags: extraction,xpath,html,metadata
  metadata:
    max-request: 1

http:
  - method: GET
    path:
      - "{{BaseURL}}/"
    matchers:
      - type: status
        status:
          - 200
    extractors:
      - type: xpath
        name: page-title
        xpath:
          - '/html/head/title'
      - type: xpath
        name: generator
        xpath:
          - '//meta[@name="generator"]/@content'
```

## Workflow

### Step 1: Validate Template

```bash
nuclei -t extraction-template.yaml -validate
```

### Step 2: Run Extraction Scan

```bash
node scripts/agent.js --action scan --target https://example.com \
  --template extraction-template.yaml --output /tmp/extraction.json
```

### Step 3: Process Extracted Data

```bash
node scripts/agent.js --action extract --input /tmp/extraction.json \
  --output /tmp/extracted-values.json
```

## Verification

- [ ] Each extractor has a descriptive `name` field
- [ ] Regex extractors use `group: 1` for capture groups
- [ ] Internal extractors set `internal: true` for flow-only values
- [ ] JSON extractors use valid JQ-style path expressions
- [ ] KVal extractors target lowercase header names
- [ ] DSL extractors produce human-readable formatted strings
- [ ] Chained extractors properly pass values between requests
- [ ] XPath selectors validated against target HTML structure

## References

- [Nuclei Extractors](https://docs.projectdiscovery.io/templates/reference/extractors)
- [Nuclei Template Matchers](https://docs.projectdiscovery.io/templates/reference/matchers)
- [JQ Manual](https://jqlang.github.io/jq/manual/)
- [XPath Syntax](https://www.w3.org/TR/xpath-31/)
