# @qa-gentic/stlc-agents

> AI-powered QA STLC automation — from Azure DevOps **or Jira Cloud** work item state change to self-healing Playwright TypeScript in a Helix-QA project.

Works with **GitHub Copilot** (VS Code Agent mode), **Claude Code**, **Cursor**, and **Windsurf**.  
Also runs fully headless via the **webhook bridge** — no human in the loop required.

![npm version](https://img.shields.io/badge/npm-v1.0.41-blue)
![PyPI version](https://img.shields.io/badge/pypi-v1.0.41-blue)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Node.js >=18](https://img.shields.io/badge/node-%3E%3D18-brightgreen)](https://nodejs.org)
[![Python >=3.10](https://img.shields.io/badge/python-%3E%3D3.10-blue)](https://python.org)

---

## What It Does

Five Python MCP servers cover the full QA Software Test Life Cycle. A sixth server (Playwright MCP) drives a real browser during code generation.

| Agent | Server name | Input | Output |
|---|---|---|---|
| Agent 1 | `qa-test-case-manager` | ADO PBI / Bug / Feature | Manual test cases created & linked (TestedBy-Forward), deduped on re-trigger |
| Agent 2 | `qa-gherkin-generator` | ADO Feature / PBI / Bug | `.feature` file validated and attached to work item |
| Agent 3 | `qa-playwright-generator` | Gherkin + optional AX-tree `context_map` | `locators.ts` + page objects + step defs (cached, retrieved via `get_generated_files`) |
| Agent 4 | `qa-helix-writer` | Generated `.ts` files + `helix_root` | Files written to Helix-QA directory layout, never overwrites |
| Agent 5 | `qa-jira-manager` | Jira Story / Bug / Task | Test cases created & linked in Jira, `.feature` attached to issue, deduped on re-trigger |

---

## End-to-End Flow

### Webhook-triggered (headless)

A work item state change in ADO or Jira fires a webhook POST to the bridge. The bridge normalises the payload and routes it through the pipeline automatically.

```
ADO Service Hook / Jira Webhook
        │  POST
        ▼
webhook_orchestrator/webhook_bridge.py  (FastAPI — qa-stlc-webhook)
        │
        ├─ parsers.py       raw payload → normalised event dict
        ├─ state_router.py  event → STAGE_TEST_CASES | STAGE_FULL_PIPELINE | SKIP
        │
        └─ ci_runner/pipeline.py
              ├─ run_test_cases()    → Agent 1 (ADO) or Agent 5 (Jira)
              └─ run_post_done()     → Agent 2 → Agent 3 → Agent 4
```

### State → stage mapping

| Platform | Work item state | Stage | Agents called |
|---|---|---|---|
| ADO | `Approved` / `Committed` | `STAGE_TEST_CASES` | Agent 1 only |
| ADO | `Done` | `STAGE_FULL_PIPELINE` | Agents 2 → 3 → 4 |
| Jira | `In Progress` / `Selected for Development` | `STAGE_TEST_CASES` | Agent 5 only |
| Jira | `Done` | `STAGE_FULL_PIPELINE` | Agents 2 → 3 → 4 |

Any other state is silently dropped. State names are configurable in `state_router.py`.

### STAGE_TEST_CASES — Agent 1 / Agent 5

```
fetch_work_item / fetch_jira_issue
        ↓
[LLM] generate_test_cases()        ← pipeline bridges fetch → LLM → create
        ↓
create_deduped_test_cases()        ← internally: get_linked → filter dupes → create_and_link
```

On re-trigger, titles already linked are skipped (case-insensitive, stop-word normalised). Only net-new test cases are created.

### STAGE_FULL_PIPELINE — Agents 2 → 3 → 4

First error stops the chain and surfaces in the result dict. Subsequent agents are not called.

**Agent 2 — Gherkin**

```
ADO Feature : fetch_feature_hierarchy  →  [LLM] generate_gherkin
ADO PBI/Bug : fetch_work_item_for_gherkin  →  [LLM] generate_gherkin
Jira        : fetch_jira_issue (via Agent 5)  →  [LLM] generate_gherkin
        ↓
validate_gherkin_content()             ← structural check before attach
        ↓  (if invalid → pipeline stops, returns validation errors)
ADO Feature : attach_gherkin_to_feature()
ADO PBI/Bug : attach_gherkin_to_work_item()
Jira        : attach_gherkin_to_issue()    ← uploads via Jira attachment API
```

The `generate_and_attach_gherkin` composite tool wraps validate + attach for CI/headless callers.

**Agent 3 — Playwright**

```
generate_playwright_code(gherkin_content, page_class_name)
        ↓  returns cache_key (files held in-memory)
get_generated_files(cache_key)
        ↓  returns { "path/to/file.ts": "content", ... }
```

`page_class_name` is derived from `event["title"]` (ADO) or `event["summary"]` (Jira) — camel-cased, max 4 words. Without a `context_map`, locators are Gherkin-inferred (stability=0). Pass `app_url` to embed a snapshot hint comment in `locators.ts`.

**Agent 4 — Helix writer**

```
inspect_helix_project(helix_root)
        ↓  framework_state: absent | partial | present
write_helix_files(helix_root, files, mode)
        mode = "scaffold_and_tests"  if absent or partial
        mode = "tests_only"          if present
```

Agent 4 handles all deduplication and conflict renaming internally. No file is ever overwritten.

---

## Install

```bash
# 1. Install the CLI + npm package globally
#    Prompted to choose integration: ado / jira / both
npm install -g @qa-gentic/stlc-agents

# 2. Bootstrap your project
qa-stlc init --vscode --integration ado    # GitHub Copilot / VS Code — ADO
qa-stlc init --vscode --integration jira   # GitHub Copilot / VS Code — Jira
qa-stlc init --vscode --integration both   # GitHub Copilot / VS Code — both
# or for Claude Code:
qa-stlc init --integration ado

# 3. Scaffold a new Playwright + Cucumber + TypeScript QA project
qa-stlc scaffold --name my-qa-project

# 4. Start the Playwright browser server (required for live-locator generation)
npx @playwright/mcp@latest --port 8931
```

### Migrating an existing Playwright project

If you have an existing Playwright (with or without Cucumber) project and want
to bring it onto the agent rails, use `stlc-migrate`:

```bash
# Preview without writing
stlc-migrate --source ./my-old-tests --helix ./helix-qa --dry-run

# Full migration — every agent skill, .mcp.json, healer scaffold, agent docs
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration both

# ADO-only or Jira-only
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration ado
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration jira

# Refresh scaffold/skills on an existing migrated tree.
# Also auto-removes stale .feature/.steps.ts/.page.ts/.locators.ts orphans
# from the role dirs (e.g. left over after switching --preserve-tree on/off).
stlc-migrate --source ./my-old-tests --helix ./helix-qa --conflict overwrite

# Preserve the source's subfolder grouping under features/steps/pages/locators
# (default flattens). Sub-dir names are kebab-cased.
stlc-migrate --source ./my-old-tests --helix ./helix-qa --preserve-tree

# Explicitly scaffold VRT (visual regression testing). Auto-enabled when the
# source already has vrt:* scripts or VRT_MODE in any .env file.
stlc-migrate --source ./my-old-tests --helix ./helix-qa --features=vrt
```

The tool emits a complete agent-ready tree:

| Layer | What lands |
|---|---|
| **Code** | Page objects + step files + locator files in `{selector, intent, stability}` format; every locator interaction routed through `LocatorHealer.*WithHealing`; non-BDD `.spec.*` files losslessly converted to one-scenario-per-test BDD |
| **Healing infra** | 8-strategy `LocatorHealer` (attribute / type-hint / role / label / text / AI Vision) with intent-shape-aware attribute matching (`:is(input, …)[id*="password"]` instead of bare `[id*="password"]`), `LocatorRepository` with schema-versioned heal-only persist + per-env file suffix, `HealingDashboard` (read/write at 7890 + read-only mirror at `HEALIX_REVIEW_PORT`, Confirm/Revert clear the entry from view, Approve-all bulk action), `TimingHealer`, `VisualIntentChecker` |
| **VRT (opt-in)** | `src/helper/vrt/{vrtCapture,vrtReport}.util.ts`, `src/test/steps/vrt.steps.ts`, `vrt:baseline` / `vrt:test` npm scripts, `pngjs`/`pixelmatch`/`fs-extra` runtime deps. Defaults are generic; project-specific masks via `VRT_GLOBAL_MASK_SELECTORS` env CSV. Enabled by `--features=vrt` or auto-detection. |
| **Agent assets** | `.mcp.json` (all 6 MCP servers + Playwright MCP), `.claude/skills/<name>/SKILL.md` per integration, `.github/copilot-instructions/<name>.md` flat copies, `AGENT-BEHAVIOR.md` + `ORCHESTRATION_RULES.md` at root + `.claude/` + `.github/` |
| **Configs** | `cucumber.js` with `dotenv/config` as first `requireModule`, `package.json` with `dotenv` + `@types/*` companions, `tsconfig.json` aliases, `.env.example` documenting every healing knob |
| **Report** | `MIGRATION-REPORT.md` — counts, TODOs, removed orphans, VRT manual-wiring snippet (when source ships its own hooks) |

See `skills/migrate-framework/SKILL.md` for the full pipeline + environment variable reference.

### Healing dashboard

Once a migrated project has run any test that triggers healing, inspect the
results via the dashboard:

```bash
ENV=qa4 npm run healix:dashboard     # full UI at http://localhost:7890
ENV=qa4 npm run healix:review        # read-only mirror at http://localhost:7891
```

The dashboard shows each healed locator with its original selector, the
specific selector that actually matched on the live page (introspected from
the matched element — e.g. `#login-username`, not the generic search union),
and **Confirm / Revert** buttons. **Confirm rewrites the original locator
`.ts` file's `selector:` field** so the next run uses the verified selector
as the primary probe — no healing chain required until the page changes
again.

### Cost Tracking Activation

**npm install** (`npm install -g @qa-gentic/stlc-agents`): Cost tracking is activated automatically after the Python servers are installed. No manual step required.

**pip install** (`pip install qa-gentic-stlc-agents`): Cost tracking is **not** activated automatically. You must run one of the following after pip install:

```bash
qa-stlc-apply-cost
# or
python -m stlc_agents.shared.install_hook
```
This patches all MCP servers to log tokens and cost for every tool call.

`qa-stlc init` does five things:

1. `pip install qa-gentic-stlc-agents` — installs all five Python MCP servers + rules files
2. Copies skill files to `.github/copilot-instructions/` (and `.claude/` if not `--vscode`)
3. Copies custom agent files to `.github/agents/`
4. Copies `ORCHESTRATION_RULES.md` to project root for reference during multi-step tasks
5. Writes `.vscode/mcp.json` (or `.mcp.json`) with all six servers configured

---

## Requirements

| Requirement | Value |
|---|---|
| Node.js | >= 18 |
| Python | >= 3.10, < 3.14 |
| ADO PAT | Required for Agents 1–4 (ADO pipeline) |
| Jira OAuth 2.0 (3LO) | Required for Agent 5 (Jira pipeline) |
| Playwright MCP | `npx @playwright/mcp@latest --port 8931` |

**ADO `.env` vars:**

```env
ADO_ORGANIZATION_URL=https://dev.azure.com/your-org
ADO_PROJECT_NAME=YourProject
ADO_PAT=your-personal-access-token
APP_BASE_URL=https://your-app.example.com

# LLM — pick one provider:
AI_HEALING_PROVIDER=anthropic
AI_HEALING_API_KEY=sk-ant-...
# or: OPENAI_API_KEY / AZURE_OPENAI_API_KEY / GITHUB_TOKEN (Copilot)
```

**Jira additional `.env` vars:**

```env
JIRA_CLIENT_ID=your-atlassian-oauth-client-id
JIRA_CLIENT_SECRET=your-atlassian-oauth-client-secret
JIRA_CLOUD_ID=your-atlassian-cloud-id
```

**Webhook bridge additional `.env` vars:**

```env
WEBHOOK_SECRET=your-shared-secret
HELIX_PROJECT_ROOT=/path/to/helix-qa   # where Agent 4 writes files
PLAYWRIGHT_MCP_URL=http://localhost:8931/mcp   # leave blank to skip Agent 3
```

---

## CLI Commands

| Command | Description |
|---|---|
| `qa-stlc init [--vscode] [--integration ado\|jira\|both]` | Full bootstrap: pip install + skills + agent files + MCP config |
| `qa-stlc scaffold [--name n] [--dir path]` | Copy full Playwright + Cucumber + TypeScript boilerplate to a new project |
| `qa-stlc skills [--target claude\|vscode\|cursor\|windsurf]` | Copy skill files to the correct AI coding agent directory |
| `qa-stlc mcp-config [--vscode] [--print]` | Write `.vscode/mcp.json` or `.mcp.json` with all servers configured |
| `qa-stlc verify` | Check that all MCP servers are reachable and auth is cached |
| `qa-stlc-webhook [--host] [--port] [--reload]` | Start the webhook bridge (FastAPI — requires `pip install qa-gentic-stlc-agents[webhook]`) |

---

## Orchestration Rules

When `qa-stlc init` is run, `ORCHESTRATION_RULES.md` is installed to your project root and placed in both npm (`node_modules/@qa-gentic/stlc-agents/`) and pip (`site-packages/stlc_agents/`) installations.

Refer to this file in multi-step QA workflows to ensure:
- **Step breakdown:** every task is decomposed into named steps with explicit inputs/outputs
- **Pre-flight gates:** output validation runs *before* generation, not after
- **No skipped steps:** intermediate data is structured and handed off explicitly, never inferred
- **Countable verification:** code generation steps state expected item counts before implementation
- **Zero stubs:** no partial outputs with TODOs or empty bodies passed downstream

Integration per AI coding agent:

- **GitHub Copilot (VS Code):** Add to `.github/copilot-instructions.md` or reference in your workspace prompt
- **Claude Code:** Reference in `.claude/instructions.md` or as a project document
- **Cursor:** Add to `.cursor/rules/orchestration.mdc` (scope: always)
- **Windsurf:** Reference in `.windsurf/rules.md` or global rules panel

See section 8 of `ORCHESTRATION_RULES.md` for agent-specific integration notes.

---

## Tool Reference

### Agent 1 — `qa-test-case-manager` _(Azure DevOps)_

| Tool | Description |
|---|---|
| `fetch_work_item` | Fetch a PBI, Bug, or Feature with acceptance criteria, coverage hints, and existing TC count. Returns `epic_not_supported` for Epics. |
| `get_linked_test_cases` | List all test cases linked via TestedBy-Forward (used by dedup). |
| `create_and_link_test_cases` | Create structured manual test cases and link to work item. Feature confirmation gate fires unless `confirmed=true`. |
| `create_deduped_test_cases` | **Headless/webhook tool.** Internally calls `get_linked_test_cases`, filters duplicates (normalised title match), then calls `create_and_link_test_cases` on net-new only. Safe to call on every re-trigger. |

### Agent 2 — `qa-gherkin-generator` _(Azure DevOps)_

| Tool | Description |
|---|---|
| `fetch_feature_hierarchy` | Fetch a Feature + all child PBIs/Bugs + coverage hints. |
| `fetch_work_item_for_gherkin` | Fetch a single PBI or Bug with parent Feature context. |
| `validate_gherkin_content` | Structural check: @smoke/@regression tags, scenario count (5–10 feature / 3–9 work_item), every scenario has `When`, no duplicate titles. |
| `attach_gherkin_to_feature` | Validate + upload + link `.feature` to a Feature work item. |
| `attach_gherkin_to_work_item` | Validate + upload + link `.feature` to a PBI or Bug. |
| `generate_and_attach_gherkin` | **Headless/webhook composite.** Accepts pre-generated `gherkin_content`, validates, attaches. Returns `status: validation_failed` with errors if invalid — pipeline must re-generate. |

### Agent 3 — `qa-playwright-generator` _(ADO + Jira)_

| Tool | Description |
|---|---|
| `generate_playwright_code` | Generate `locators.ts`, `*Page.ts`, `*.steps.ts`, `cucumber-profile.js`. Returns `cache_key`. Optional `context_map` for AX-tree-verified locators; optional `app_url` embeds snapshot hint. |
| `get_generated_files` | Retrieve full file content by `cache_key` from Agent 3's in-memory cache. |
| `scaffold_locator_repository` | Generate the five Helix-QA healing infrastructure files (`LocatorHealer`, `TimingHealer`, `VisualIntentChecker`, `LocatorRepository`, `HealingDashboard`). Call once per project. |
| `validate_gherkin_steps` | Check for duplicate step strings and missing `When` steps. |
| `attach_code_to_work_item` | Attach generated TypeScript delta files to an ADO work item. |

### Agent 4 — `qa-helix-writer` _(ADO + Jira)_

| Tool | Description |
|---|---|
| `inspect_helix_project` | Returns `framework_state`: `present` / `partial` / `absent`. Drives `mode` selection in pipeline. |
| `write_helix_files` | Write files to Helix-QA layout. `mode=scaffold_and_tests` for new/partial projects; `mode=tests_only` for existing. Never overwrites; deduplicates and conflict-renames. |
| `list_helix_tree` | Full directory listing of a Helix-QA project. |
| `read_helix_file` | Read an existing file for overlap detection. |

### Agent 5 — `qa-jira-manager` _(Jira Cloud)_

| Tool | Description |
|---|---|
| `fetch_jira_issue` | Fetch a Story, Bug, or Task with acceptance criteria and coverage hints. Returns `epic_use_hierarchy` for Epics. |
| `get_linked_test_cases` | List all issues linked via `is tested by` / `Test` link type (used by dedup). |
| `create_and_link_test_cases` | Create test case issues in Jira (type `Test`, falls back to `Task`) and link. Steps stored as ADF table — no Xray required. Epic confirmation gate fires unless `confirmed=true`. |
| `create_deduped_test_cases` | **Headless/webhook tool.** Internally calls `get_linked_test_cases`, filters duplicates (normalised summary match), then calls `create_and_link_test_cases` on net-new only. Safe on every re-trigger. |
| `attach_gherkin_to_issue` | Upload a `.feature` file as an attachment to a Jira issue via the Jira attachment API. File is named `{issue_key}_{summary_kebab}_regression.feature`. |

---

## Three-Layer Self-Healing Architecture

| Layer | Class | What It Heals |
|---|---|---|
| 1 — Locator | `LocatorHealer` | cached-healed → primary → **attribute (intent-shape-aware)** → type-hint → role → label → text → AI Vision |
| 2 — Timing | `TimingHealer` | network-trace drift → auto-adjusted timeouts → HealingDashboard |
| 3 — Visual | `VisualIntentChecker` | element screenshot diff at assertions → HealingDashboard |

Healed selectors persist in `LocatorRepository`. All AI suggestions require human approval at `http://localhost:7890` before entering version control. The attribute strategy constrains its substring matches by intent shape (`"password input"` → `:is(input, textarea, select, [contenteditable])`) so it can't pick up unrelated elements such as `<a id="forgot-password">` when the intent calls for an input.

---

## Webhook / Auto-Trigger Setup

### Local dev

```bash
# 1. Install webhook extras
pip install "qa-gentic-stlc-agents[webhook]"

# 2. Start the bridge
make serve

# 3. Expose via ngrok
ngrok http 8080

# 4. Register hooks
make register-ado-hooks  BRIDGE_URL=https://xxxx.ngrok.io
make register-jira-hooks BRIDGE_URL=https://xxxx.ngrok.io
```

### Azure Functions deploy

```bash
make deploy-azure FUNC_APP=stlc-webhook-bridge
make register-ado-hooks  BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api
make register-jira-hooks BRIDGE_URL=https://stlc-webhook-bridge.azurewebsites.net/api
```

### Docker

```bash
make docker-build-webhook
make docker-run-webhook
```

### LLM provider selection

| Provider | Env var(s) | Default model |
|---|---|---|
| `anthropic` | `AI_HEALING_API_KEY` or `ANTHROPIC_API_KEY` | `claude-haiku-4-5-20251001` |
| `copilot` | `GITHUB_TOKEN` | `gpt-4o` |
| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
| `azure-openai` | `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_ENDPOINT` | `gpt-4o-mini` |
| `ollama` | `OLLAMA_HOST` (optional) | `llama3.2` |

Override model at any time: `LLM_MODEL=claude-sonnet-4-6`

### Token cost (headless, Haiku default)

| Stage | Tokens / work item | Approx cost |
|---|---|---|
| Test case generation (state → Approved) | ~8,000–15,000 | $0.01–$0.03 |
| Gherkin generation (state → Done) | ~10,000–18,000 | $0.01–$0.04 |
| Playwright TS generation (state → Done) | ~20,000–35,000 | $0.03–$0.07 |

---

## ADO vs Jira — Side by Side

| Step | Azure DevOps | Jira Cloud |
|---|---|---|
| Trigger | Service Hook: `workitem.updated` | Webhook: `jira:issue_updated` |
| Fetch issue | `fetch_work_item` | `fetch_jira_issue` |
| Check duplicates | `get_linked_test_cases` | `get_linked_test_cases` |
| Create test cases (interactive) | `create_and_link_test_cases` | `create_and_link_test_cases` |
| Create test cases (headless) | `create_deduped_test_cases` | `create_deduped_test_cases` |
| Gherkin attach | `attach_gherkin_to_feature` / `attach_gherkin_to_work_item` | `attach_gherkin_to_issue` |
| Headless Gherkin composite | `generate_and_attach_gherkin` | `generate_and_attach_gherkin` + `attach_gherkin_to_issue` |
| Playwright generation | `generate_playwright_code` (shared) | `generate_playwright_code` (shared) |
| Write to disk | `write_helix_files` (shared) | `write_helix_files` (shared) |
| Link relation | `TestedBy-Forward` | `is tested by` / `Test` |
| Auth | MSAL silent + browser (`~/.msal-cache/`) | OAuth 2.0 3LO + browser (`~/.jira-cache/`) |

---

## Run Tests

```bash
# Full suite
ENABLE_SELF_HEALING=true \
HEALING_DASHBOARD_PORT=7890 \
APP_BASE_URL=<your-app-base-url> \
APP_EMAIL=<email> \
APP_PASSWORD=<password> \
cucumber-js --config=config/cucumber.js -p <feature_profile>

# Smoke only
cucumber-js --config=config/cucumber.js -p <feature_profile> --tags "@smoke"
```

---

## Documentation

- [docs/ARCHITECTURE-ADO.md](docs/ARCHITECTURE-ADO.md) — Full technical architecture, ADO pipeline
- [docs/ARCHITECTURE-JIRA.md](docs/ARCHITECTURE-JIRA.md) — Full technical architecture, Jira pipeline
- [docs/QUICKSTART-ADO.md](docs/QUICKSTART-ADO.md) — Quick start for Azure DevOps
- [docs/QUICKSTART-JIRA.md](docs/QUICKSTART-JIRA.md) — Quick start for Jira Cloud
- [docs/WALKTHROUGH-ADO.md](docs/WALKTHROUGH-ADO.md) — End-to-end walkthrough, ADO pipeline
- [docs/WALKTHROUGH-JIRA.md](docs/WALKTHROUGH-JIRA.md) — End-to-end walkthrough, Jira pipeline
- [docs/migration-guide.md](docs/migration-guide.md) — Full `stlc-migrate` migration guide (preserve-tree, VRT, healer wiring)
- [docs/qa-stlc-commands.html](docs/qa-stlc-commands.html) — Command reference for the CLI and setup flows

---

## License

MIT