<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
---
name: building-security-regression-tests
description: >-
  Create automated regression tests that verify previously fixed vulnerabilities
  remain patched. Map tests to CVEs and findings, integrate into CI gates, and
  prevent security fix rollbacks during refactoring.
domain: cybersecurity
subdomain: devsecops
tags:
  - regression-testing
  - security-testing
  - cve-tracking
  - vulnerability-management
  - ci-cd
  - pytest
  - shift-left
version: "1.0"
author: defconxt
license: AGPL-3.0
metadata:
  mitre-attack: []
---

# Building Security Regression Tests

## Overview

Security regression tests verify that previously identified and fixed vulnerabilities
do not reappear during code changes. Each test maps to a specific CVE, finding ID,
or security advisory, creating an ever-growing safety net. This skill covers writing
regression tests, organizing them by vulnerability class, and integrating into CI gates.

Mode: `[MODE: BLUE]` — Prevent vulnerability recurrence through automated regression testing.

## Prerequisites

| Requirement | Details |
|---|---|
| pytest with security test fixtures | Required |
| Vulnerability tracking system (findings mapped to test IDs) | Required |
| CI/CD pipeline with test stage | Required |
| Python 3.10+ or equivalent test framework | Required |

## Key Concepts

### Regression Test Structure

```python
# tests/security/regression/test_cve_2024_xxxx.py
"""Regression test for CVE-2024-XXXX: SQL injection in search endpoint.

Finding: FINDING-042
Fixed in: commit abc123 (PR #187)
Original severity: Critical (CVSS 9.8)
"""

import pytest
from app.api import app

SQL_INJECTION_PAYLOADS = [
    "' OR '1'='1' --",
    "'; DROP TABLE users; --",
    "1 UNION SELECT username, password FROM users",
    "admin'/*",
    "1; EXEC xp_cmdshell('whoami')",
]


class TestCVE2024XXXX:
    """Verify SQL injection fix in /api/search endpoint."""

    @pytest.fixture
    def client(self):
        return app.test_client()

    @pytest.mark.parametrize("payload", SQL_INJECTION_PAYLOADS)
    def test_search_rejects_sql_injection(self, client, payload: str) -> None:
        response = client.get(f"/api/search?q={payload}")
        assert response.status_code in (200, 400)
        data = response.get_json()
        assert "password" not in str(data).lower()
        assert "username" not in str(data).lower() or "results" in data

    def test_search_uses_parameterized_query(self) -> None:
        """Verify the fix: parameterized queries in search module."""
        import ast
        import inspect
        from app.search import search_handler

        source = inspect.getsource(search_handler)
        tree = ast.parse(source)
        for node in ast.walk(tree):
            if isinstance(node, ast.JoinedStr):
                for value in node.values:
                    if isinstance(value, ast.FormattedValue):
                        pytest.fail("f-string found in query — use parameterized queries")
```

### Regression Test Registry

```python
# tests/security/regression/conftest.py
"""Security regression test registry and fixtures."""

import pytest
from dataclasses import dataclass

@dataclass
class SecurityRegression:
    finding_id: str
    cve: str | None
    severity: str
    fixed_commit: str
    description: str
    test_module: str

REGRESSION_REGISTRY: list[SecurityRegression] = [
    SecurityRegression(
        finding_id="FINDING-042",
        cve="CVE-2024-XXXX",
        severity="critical",
        fixed_commit="abc123",
        description="SQL injection in search endpoint",
        test_module="test_cve_2024_xxxx",
    ),
    SecurityRegression(
        finding_id="FINDING-051",
        cve=None,
        severity="high",
        fixed_commit="def456",
        description="IDOR in user profile endpoint",
        test_module="test_finding_051_idor",
    ),
]

@pytest.fixture
def regression_registry():
    return REGRESSION_REGISTRY
```

### CI Integration

```yaml
name: Security Regression Tests
on:
  pull_request:
  push:
    branches: [main]

jobs:
  regression:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -r requirements-test.txt
      - name: Run security regression tests
        run: |
          pytest tests/security/regression/ -v \
            --tb=long \
            --junitxml=regression-results.xml \
            -x  # Stop on first failure — regression = critical
      - name: Publish results
        if: always()
        uses: mikepenz/action-junit-report@v4
        with:
          report_paths: regression-results.xml
          fail_on_failure: true
          check_name: Security Regression Tests
```

### IDOR Regression Test

```python
# tests/security/regression/test_finding_051_idor.py
"""Regression test for FINDING-051: IDOR in user profile endpoint.

Fixed in: commit def456 (PR #201)
Original severity: High
"""

import pytest
from app.auth import create_token

class TestFinding051IDOR:
    def test_user_cannot_access_other_profile(self, client, user_a, user_b) -> None:
        token = create_token(user_a)
        response = client.get(
            f"/api/users/{user_b.id}/profile",
            headers={"Authorization": f"Bearer {token}"},
        )
        assert response.status_code == 403

    def test_user_can_access_own_profile(self, client, user_a) -> None:
        token = create_token(user_a)
        response = client.get(
            f"/api/users/{user_a.id}/profile",
            headers={"Authorization": f"Bearer {token}"},
        )
        assert response.status_code == 200
```

## Workflow

### Step 1: Generate Regression Test from Finding

```bash
node scripts/agent.js --action generate --finding-id FINDING-042 --cve CVE-2024-XXXX --output ./tests/security/regression/
```

### Step 2: List Registered Regressions

```bash
node scripts/agent.js --action list --test-dir ./tests/security/regression/ --output /tmp/registry.json
```

### Step 3: Verify Coverage Against Findings

```bash
node scripts/agent.js --action coverage --test-dir ./tests/security/regression/ --output /tmp/regression-coverage.json
```

## Verification

- [ ] Every fixed vulnerability has a corresponding regression test
- [ ] Regression tests run on every PR with fail-fast (-x)
- [ ] Test registry maps findings to CVEs and commits
- [ ] Regression failure blocks merge immediately
- [ ] New findings automatically generate regression test templates

## References

- [OWASP Testing Guide — Regression](https://owasp.org/www-project-web-security-testing-guide/)
- [pytest Documentation](https://docs.pytest.org/)
- [CVE Database](https://cve.mitre.org/)
