# AI QA Pipeline

**User Story → Test Plan → Test Generation → Execution → Self-Healing → Dashboard**

One-command install into any web project. Works standalone or with any AI assistant (opencode, GitHub Copilot, Claude Code).

---

## One-Command Install

```bash
npx @ai-qa/workflow init --yes
```

That's it. This single command installs the full pipeline into your current project:

- Copies pipeline scripts (planner, generator, executor, healer, reporter)
- Installs Playwright + dashboard dependencies
- Adds npm scripts (`qa:*` commands)
- Creates directories (`user-story/`, `specs/`, `tests/`, `test-results/`)
- Registers your project in the dashboard

### Update an existing installation

```bash
npx @ai-qa/workflow update --yes
```

Updates pipeline files while **preserving** your configuration (`.qa-workflow.json`, `opencode.json`), user stories, test specs, and run results.

### What you get

```
your-project/
├── ai-qa-workflow.js          # CLI orchestrator (10 commands)
├── .qa-workflow.json          # Project config (auto-detected, editable)
├── scripts/                   # 6 automation modules
├── opencode.json              # MCP config (Playwright + GitHub)
├── .github/agents/            # AI agent definitions
├── prompts/                   # AI workflow instructions
├── qa-dashboard/              # Web UI (port 4000)
├── user-story/                # Your .md stories
├── specs/                     # Generated test plans
├── tests/                     # Generated Playwright specs
└── test-results/   
           # Run results, reports, screenshots
```

### Quick start after install

```bash
# 1. Initialize (auto-detects project config)
npm run qa:init

# 2. Write a user story in user-story/ or let the AI create one

# 3. Run full pipeline
npm run qa:run my-story.md

# 4. Start the dashboard
npm run dashboard
```

---

## How It Works

The framework operates in two distinct, powerful phases:

### Phase 1: Creation (Agent-Driven)
You interact with an AI Agent in your IDE (Cursor, OpenCode, Copilot, Antigravity). You simply provide a URL or a Markdown User Story. The agent automatically:
1. Loads its intelligence from `agents/router.md`
2. Uses Playwright MCP to open a hidden browser
3. Visually explores your application in real-time
4. Generates a flawless test plan (`specs/`) and executable code (`tests/`) using **real** semantic selectors. No more hallucinated CSS classes!

### Phase 2: Execution & Tracking (Dashboard-Driven)
Once the test scripts are generated by the Agent, you move to execution:
1. Open the Web Dashboard (`npm run dashboard`)
2. Click **Execute** or trigger it via your CI/CD pipeline
3. The framework handles Playwright execution, auto-healing (fixing minor flakiness), and report generation
4. View deep analytics, execution history, and Allure reports directly in the Dashboard UI.

---

## Zero-Config Design

The template works out of the box by **auto-detecting** your project configuration.

On `npm run qa:init`, it scans your project and generates `.qa-workflow.json`:

| Scan source | What it detects |
|-------------|----------------|
| `package.json` | Project name, description, dev server port |
| `README.md` | Project name, URL, description |
| `docker-compose.yml` | Exposed port, project name |
| `.env` | `APP_URL`, `APP_ENV` / `NODE_ENV` |
| `playwright.config.*` | Browser type (edge, webkit) |
| Directory structure | Framework (Angular, Next.js, Python) |

If a file is missing, the template falls back to sensible defaults. No manual setup required.

### The Config File (`.qa-workflow.json`)

```json
{
  "project": {
    "name": "my-app",
    "description": "My web application",
    "url": "http://localhost:5173",
    "environment": "Development"
  },
  "browser": {
    "type": "chromium",
    "cdpPort": 9222,
    "headed": false
  },
  "test": {
    "timeout": 120000,
    "retries": 0,
    "workers": 1
  },
  "auth": {
    "user": "",
    "credentials": {}
  }
}
```

Edit this file to override auto-detected values. The template merges your settings with defaults — you only need to set what differs.

### Application Context (`docs/application-context.md`)

This file serves as a **knowledge base for AI agents** during test planning and healing. On `init` it's created from config values. As you use the pipeline, AI agents populate it with:

- **Stable selectors** — CSS/XPath selectors discovered during exploration
- **Known flaky areas** — elements or flows that frequently break
- **Auth details** — users, tokens, environments
- **Tech stack notes** — framework specifics the AI should know

For best results, edit this file with your project's specifics before running the pipeline.

---

## Installation

### Requirements

| Component | Needed for | Install |
|-----------|-----------|---------|
| **Node.js 18+** | Running the pipeline | — |
| **Playwright** | Test execution | `npm install @playwright/test` |
| **Chromium** | Running tests | `npx playwright install chromium` |
| **Playwright MCP** | AI browser automation (explore, click, type) | `npm install -D @playwright/mcp` |
| **GitHub MCP** | AI creating PRs/issues from results | `npm install -D @modelcontextprotocol/server-github` + `GITHUB_TOKEN` env |

### Install into a project

```bash
# From npm (recommended — works anywhere)
npx @ai-qa/workflow init --yes

# From local template
node install.js ../my-project --yes
```

The installer copies all pipeline files, creates directories, adds npm scripts, installs dashboard dependencies, and registers your project in the dashboard.

### Update an installed project

```bash
# From npm
npx @ai-qa/workflow update --yes

# From local template
node install.js ../my-project --update --yes
```

Updates pipeline files while preserving: `.qa-workflow.json`, `opencode.json`, `user-story/`, `specs/`, `tests/`, `test-results/`, `qa-dashboard/data/`, `qa-dashboard/node_modules/`.

### Commands added to your project

```bash
npm run qa:init          # Initialize + auto-detect config
npm run qa:run <story>   # Full pipeline: plan → generate → execute → heal → report
npm run qa:plan <story>  # Generate test plan only
npm run qa:generate      # Generate test code from plan
npm run qa:execute       # Run Playwright tests
npm run qa:heal          # Self-heal failed tests
npm run qa:report        # Generate final report
npm run qa:status        # Check pipeline state
npm run qa:list          # List stories, plans, specs
npm run dashboard        # Launch web dashboard at :4000
```

---

## How to Get the Most Out of It

### 1. Provide good context upfront

Before running the pipeline, populate `docs/application-context.md`:

```markdown
## Project
- Name: My App
- URL: https://staging.myapp.com
- Environment: Staging

## Authentication
- Login flow: email + password via /api/auth/login
- Test user: test@example.com / TestPass123

## Stable Selectors
- Login button: button[data-testid="login-btn"]
- Error toast: .toast-error
```

The AI reads this file during test planning and healing. The more accurate this is, the better the generated tests.

### 2. Write structured user stories

Your `user-story/*.md` files should follow this format:

```markdown
# User Login

**Story ID**: US-001
**Title**: User Login
**Feature**: Authentication

## Description
As a registered user, I want to log in with my credentials so I can access the dashboard.

## Preconditions
- User exists with email test@example.com
- Browser is on the login page

## Acceptance Criteria
1. Enter valid email and password
2. Click "Sign In"
3. Redirected to dashboard
4. User name displayed in header
```

Well-structured stories produce better test plans. The planner extracts acceptance criteria, preconditions, and feature name automatically.

### 3. Review generated test plans

After `npm run qa:plan`, check `specs/*-test-plan.md`. The AI generates scenarios from acceptance criteria. Review and edit before generating test code:

- Add edge cases the AI missed
- Add negative test scenarios
- Fix any incorrect assumptions about the app

### 4. Use the run mode that fits your workflow

| Mode | Command | Best for |
|------|---------|----------|
| **Full auto** | `npm run qa:run <story>` | Quick smoke tests, CI pipelines |
| **Step by step** | plan → generate → execute → heal → report | Reviewing each stage, manual edits |
| **Manual test only** | `npm run qa:execute <test>` | When tests are already written |
| **Heal only** | `npm run qa:heal [runId]` | After manual test edits |

### 5. Understand self-healing

The healer makes up to **2 attempts** per failed test:

1. **Standard re-run** — flakes due to timing or network
2. **Longer timeout (60s)** — slow responses
3. **Stop** — marks as defect, classifies the failure

Failure classification:
| Type | Label | Auto-healed? |
|------|-------|-------------|
| Selector not found | `selector` | ✅ Usually |
| Timeout | `timing` | ✅ Usually |
| Target closed | `environment` | ❌ Bug |
| Network error | `environment` | ❌ Bug |
| Test syntax error | `test-syntax` | ❌ Manual fix |

### 6. Check the dashboard

```bash
npm run dashboard
# → http://localhost:4000
```

The dashboard shows multi-project management, execution history with pass/fail/healed stats, analytics charts (pass rate, duration, healing vs defects), and test data generation tools.

---

## Self-Healing Rules

| Attempt | Strategy | If still fails |
|---------|----------|----------------|
| 1 | Standard re-run | Classify failure |
| 2 | Longer timeout (60s) | Mark as defect |
| STOP | — | `test.fixme()` + classify |

---

## Project Structure

```
your-project/
├── ai-qa-workflow.js          # CLI orchestrator (9 commands)
├── .qa-workflow.json          # Project config (auto-detected, editable)
├── scripts/
│   ├── utils.js               # Config loading, auto-detection, shared helpers
│   ├── planner.js             # user-story → test-plan template
│   ├── generator.js           # test-plan → test spec skeleton
│   ├── executor.js            # runs Playwright tests
│   ├── healer.js              # 2 attempts max, classifies defects
│   └── reporter.js            # markdown + JSON reports
├── opencode.json              # MCP config (Playwright + GitHub)
├── .github/agents/            # AI agent definitions
├── prompts/                   # AI workflow instructions
├── qa-dashboard/              # Web UI (port 4000)
├── user-story/                # Your .md stories
├── specs/                     # Generated test plans
├── tests/                     # Generated Playwright specs
├── test-results/              # Run results, reports, screenshots
└── docs/                      # Application context (AI knowledge base)
```

---

## Allure Reports

The pipeline supports **Allure** for rich, interactive test reports.

### Setup

```bash
# 1. Install Allure dependencies
npm install -D allure-playwright

# 2. Add to Playwright config (playwright.config.ts or .js)
# reporter: [["list"], ["allure-playwright"]],

# 3. Run tests (generates allure-results/)
npm run qa:execute

# 4. Generate the HTML report
npm run qa:report:allure
```

### View in Dashboard

Once generated, the Allure report is available from the dashboard:

- **Runs list** → "Allure Report" button on each run
- **Run detail** → "Open Allure Report ↗" in the Report Actions card
- Direct URL: `/allure-report?project=<project-id>`

The dashboard auto-detects `allure-report/index.html` and shows the buttons only when the report exists.

---

## Token Efficiency (for AI)

1. Body text assertions over complex selectors
2. Screenshots off by default (only on failure)
3. Healer makes 1 fix attempt max per test
4. Real bugs classified immediately — no retries
5. AI edits 1-3 lines maximum per fix
