# STLC Agents — Architecture: Azure DevOps Pipeline

> Technical reference for the Azure DevOps pipeline.
> For the Jira Cloud pipeline see [ARCHITECTURE-JIRA.md](ARCHITECTURE-JIRA.md).

---

## System Overview

STLC Agents is a Python monorepo of four Model Context Protocol (MCP) servers that together automate the full Software Test Life Cycle on top of Azure DevOps. Each server is an independent Python process. A coding agent (Claude Code, GitHub Copilot, Cursor, Windsurf) installs all four with a single `npm install -g @qa-gentic/stlc-agents`, reads the skill files once, and can then drive the entire STLC pipeline — from work item to running self-healing Playwright tests — in a single prompt.

A fifth server, the external **Playwright MCP** (`http://localhost:8931/mcp`), drives a real browser during code generation to produce zero-hallucination selectors from the live accessibility tree.

---

## Data Flow

```
ADO Work Item (PBI / Bug / Feature)
         │
         │  Agent 1: qa-test-case-manager
         ├─ fetch_work_item → coverage hints, complexity
         └─ create_and_link_test_cases → TC work items in ADO (TestedBy-Forward)
                   │
                   ▼
         ADO Test Cases linked to work item
                   │
                   │  Agent 2: qa-gherkin-generator
                   ├─ fetch_feature_hierarchy → full child hierarchy
                   ├─ validate_gherkin_content → structural check
                   └─ attach_gherkin_to_feature → .feature file in ADO
                             │
                             ▼
                    Gherkin .feature file
                             │
                             │  Playwright MCP (external)
                             ├─ browser_navigate → live app page
                             └─ browser_snapshot → AX tree → context_map
                                       │
                                       │  Agent 3: qa-playwright-generator
                                       ├─ generate_playwright_code → 4 TS files per feature
                                       ├─ scaffold_locator_repository → 5 infra files
                                       └─ attach_code_to_work_item → files in ADO
                                                 │
                                                 │  Agent 4: qa-helix-writer
                                                 └─ write_helix_files → Helix-QA disk layout
                                                           │
                                                           ▼
                                              cucumber-js test run
                                                           │
                                              HealingDashboard :7890
                                              (human review of AI suggestions)
```

---

## Agent 1 — `qa-test-case-manager`

**Purpose:** Convert ADO work item acceptance criteria into structured manual test cases, linked back via `TestedBy-Forward` relation.

### Tools

| Tool | ADO API Call | Returns |
|---|---|---|
| `fetch_work_item` | `GET /wit/workitems/{id}?$expand=relations` | Title, AC, story points, coverage hints, existing TC count, parent feature |
| `get_linked_test_cases` | `GET /wit/workitems/{id}/relations` | Titles and IDs of TCs already linked (deduplication gate) |
| `create_and_link_test_cases` | `POST /wit/workitems/$Test Case` + `PATCH` | Created TC IDs and ADO URLs |

### Coverage Hint Extraction

`_extract_coverage_hints()` scans acceptance criteria text for keywords and returns which of 11 categories need test cases:

```
toast_notifications  file_size_boundary  post_action_state  platform_specific
computed_values      data_persistence    accessibility       image_manipulation
file_formats         cancel_flows        validation_errors
```

### Complexity Classification

| Story Points | Complexity | TC Range |
|---|---|---|
| 1–3 | Simple | 3–6 |
| 4–8 | Medium | 6–12 |
| 9+ | Complex | 12–18 |

### Work Item Type Routing

| Work Item Type | Behaviour |
|---|---|
| Epic | Returns `epic_not_supported` — manual TCs are never created for Epics |
| Feature | Returns `confirmation_required: true` — user must confirm before creation |
| PBI / Bug | Normal deduplication + creation flow |

---

## Agent 2 — `qa-gherkin-generator`

**Purpose:** Convert an ADO work item hierarchy into a validated Gherkin `.feature` file and attach it.

### Tools

| Tool | Description |
|---|---|
| `fetch_feature_hierarchy` | Fetch a Feature with all child PBIs/Bugs and linked test case steps |
| `fetch_work_item_for_gherkin` | Fetch a PBI or Bug with parent Feature context |
| `attach_gherkin_to_feature` | Validate and attach a `.feature` file to a Feature work item |
| `attach_gherkin_to_work_item` | Validate and attach a `.feature` file to a PBI or Bug |
| `validate_gherkin_content` | Structural validation — returns `valid: bool` + `errors` + `warnings` |

### Work Item Type Routing (strict — no inference)

```
Epic ID    → ⛔ Not supported — warn user; ask for a Feature or PBI/Bug ID
Feature ID → fetch_feature_hierarchy
PBI/Bug ID → fetch_work_item_for_gherkin
Unknown    → call fetch tool; read work_item_type from response
```

### Skill-Enforced Gherkin Rules

- 5–10 scenarios per feature file
- `@smoke` on the primary happy path; `@regression` on all others
- `Scenario Outline` for data-driven boundary tests
- `Background` only when 2+ scenarios share identical preconditions
- Every scenario must include explicit navigation steps
- File naming: `{id}_{title-in-kebab-case}_regression.feature`

---

## Agent 3 — `qa-playwright-generator`

**Purpose:** Convert a validated Gherkin file into production-ready, three-layer self-healing Playwright TypeScript, with selectors derived from the live browser accessibility tree.

### Tools

| Tool | Description |
|---|---|
| `generate_playwright_code` | Gherkin + `context_map` → 4 TypeScript files per feature. Hard-blocks if `context_map` is absent — prevents hallucinated locators. |
| `scaffold_locator_repository` | Generate 5 shared healing infrastructure files (once per project) |
| `validate_gherkin_steps` | Check for duplicate step strings and missing `When` steps |
| `attach_code_to_work_item` | Attach delta TypeScript files to an ADO work item |

### Generated Files Per Feature

| File | Contents |
|---|---|
| `src/locators/{kebab}.locators.ts` | AX-tree-verified selectors as `{ selector, intent, stability }` entries |
| `src/pages/{kebab}.page.ts` | Page object with healer fields + ctor registrations under both namespaced (`<page>.<key>`) and bare-key conventions |
| `src/test/steps/{kebab}.steps.ts` | Cucumber step definitions via PageFixture DI |
| `src/test/features/{kebab}.feature` | Generated Gherkin (when Agent 3 emits it) |
| `config/cucumber.js` | Cucumber profile entry (kept compatible with the `dotenv/config` + `ts-node/register` + `tsconfig-paths/register` `requireModule` chain) |

### Shared Healing Infrastructure (once per project)

| File | Purpose |
|---|---|
| `src/utils/locators/LocatorHealer.ts` | 8-strategy locator resolution chain (tunable via `HEAL_STRATEGY_ORDER`); the **attribute** strategy is intent-shape-aware (`_intentShapeFilter` prefixes a CSS `:is(input, …)` filter when the intent says "input"/"field"/"password"/etc.) so substring matches can't cross-pollinate element types. After match, introspects the element and persists a **specific** single-attribute selector (`#id`, `[data-testid=…]`, …), not the search union |
| `src/utils/locators/LocatorRepository.ts` | Heal-store JSON with schema v1 envelope, heal-only persist filter, debounced writes, singleton exit-flush, per-env file suffix via `ENV` / `HELIX_ENV`, `HEAL_HISTORY_CAP` |
| `src/utils/locators/TimingHealer.ts` | Network trace drift detection and auto-healing |
| `src/utils/locators/VisualIntentChecker.ts` | Element screenshot pixel diff against approved baseline |
| `src/utils/locators/HealingDashboard.ts` | Two-server HTTP UI — read/write at `HEALING_DASHBOARD_PORT` (default 7890), read-only mirror at `HEALIX_REVIEW_PORT` (opt-in). Auto-discovers all sibling heal-store files in the directory. Confirm on a healed entry writes the healed selector back to the source `*.locators.ts` and clears the entry from the dashboard (Revert clears it identically — both actions end the entry's lifecycle in one click). When ≥ 2 healed entries are pending, an **Approve all** button at the top right of the page bulk-applies every healed selector to its locator file in a single sweep. |
| `src/utils/locators/dashboard-server.ts` | Standalone launcher behind `npm run healix:dashboard` / `healix:review` |
| `src/utils/locators/{ElementContextHelper, HealApplicator, LocatorRules, LocatorStrategy, healix-ci-apply, review-server}.ts` | Filled from BOILERPLATE — pure additions that integrate with the enhanced scaffolds above |

### Zero-Hallucination Guarantee

Agent 3 requires a `context_map` derived from Playwright MCP `browser_snapshot` calls. It hard-blocks generation if `context_map` is absent. Every selector in the context map is validated against the live DOM before being written to `locators.ts` — only selectors matching exactly one element are used.

---

## Agent 4 — `qa-helix-writer`

**Purpose:** Write already-generated files to disk at the correct Helix-QA directory layout. No ADO dependency; no LLM calls.

### Helix-QA Directory Mapping

| Generated File | Helix-QA Path |
|---|---|
| `locators.ts` | `src/locators/<kebab>.locators.ts` |
| `<Page>.page.ts` | `src/pages/<Page>.page.ts` |
| `<feature>.steps.ts` | `src/test/steps/<feature>.steps.ts` |
| `<feature>.feature` | `src/test/features/<feature>.feature` |
| Cucumber profile key | `src/config/cucumber.config.ts` (appended, not replaced) |
| `utils/locators/*.ts` | `src/utils/locators/<file>` |

### Mandatory Tool Call Order

1. `list_helix_tree` — always first, to see what exists
2. `read_helix_file` — for specific file deduplication
3. `write_helix_files` — pass the `files` dict from Agent 3 directly

---

## Three-Layer Self-Healing Architecture

### Layer 1 — Locator Healing (`LocatorHealer.ts`)

When a primary selector fails, `LocatorHealer.resolve()` walks an ordered
chain of strategies — order is controlled by `HEAL_STRATEGY_ORDER` (CSV env
var, default order below):

| Strategy | Method | Default timeout | Notes |
|---|---|---|---|
| 1 | `LocatorRepository.getHealed()` — cached selector from prior run | 3s attach | Persisted via singleton exit-flush; survives process restart |
| 2 | `page.locator(primarySelector)` — selector from `locators.ts` | 5s | Original definition |
| 3 | **Attribute partial-match** — single combined CSS selector across `name`, `placeholder`, `id`, `aria-label`, `data-testid`, `data-test`, `autocomplete`, prefixed with the intent shape (`:is(input, textarea, select, [contenteditable])` for input-like intents; `:is(button, [role="button"], …)` for buttons; `:is(a, [role="link"])` for links) | 3s | Most reliable for form inputs without an accessible name. Shape prefix prevents an `id*="password"` match from landing on `<a id="forgot-password">` when the intent is `"password input"`. |
| 4 | **Type-hint heuristics** — keyword-driven `input[type=password]`, `[type=email]`, `button[type=submit]`, etc. | 3s | Per `LOCATOR_TIMEOUT` |
| 5 | `page.getByRole(role, { name: regex })` for `button`/`link`/`textbox`/`combobox`/`checkbox`/`radio` | 3s × 6 roles | |
| 6 | `page.getByLabel(intent, { exact: false })` | 3s | |
| 7 | `page.getByText(keyword, { exact: false })` | 3s | First word > 3 chars |
| 8 | **AI Vision** — `ariaSnapshot` → Anthropic / Copilot / Claude Code | ~10s | Off via `ENABLE_AI_HEALING=false`; requires API key |

After any strategy matches, **`_resolveSpecificSelector(loc)` introspects
the live element** and persists the most stable single-attribute selector
(`data-testid` > `id` > `name` > `placeholder` > `aria-label` >
`autocomplete` > `input[type]`) — not the broad selector the strategy
itself used to search. So heals look like the qa-playwright-generator's
own generation-time output: concrete, stable, single-attribute.

`LOCATOR_HEAL_ATTEMPTS=N` bounds the total number of probes before
falling through to AI Vision (default 0 = unlimited).

### Layer 2 — Timing Healing (`TimingHealer.ts`)

Measures actual network duration per named action, compares to stored baseline. When drift exceeds 20%, it auto-adjusts with 50% headroom for the current run and queues a suggestion to HealingDashboard.

### Layer 3 — Visual Healing (`VisualIntentChecker.ts`)

At locators marked `visualIntent: true`, captures a scoped element screenshot and pixel-diffs against an approved baseline. A 0.5% threshold catches colour changes, truncated labels, or layout shifts that the locator layer would miss.

### Selector Stability Ranking

| Selector Strategy | Score | Rationale |
|---|---|---|
| `data-testid` | 100 | Explicit test contract — never changes incidentally |
| `aria-role + aria-label` | 90 | Semantic; survives CSS and DOM restructuring |
| Non-generated `id` | 80 | Stable when `id` is not auto-generated |
| `aria-label` alone | 70 | Semantic but may be internationalised |
| `placeholder` | 60 | Input-only; survives minor refactors |
| Role + text (XPath) | 50 | Fallback; validated at runtime |

### HealingDashboard — Heal Review + Source Write-Back

Two HTTP servers:

| Port (env var) | Default | Role |
|---|---|---|
| `HEALING_DASHBOARD_PORT` | 7890 | Full UI — **Healed locators** table (Confirm / Revert) + **Pending AI suggestions** table (Approve / Reject). POST endpoints commit changes. |
| `HEALIX_REVIEW_PORT` | 0 (opt-in) | Read-only mirror — same GET endpoints (`/api/healed`, `/api/pending`, `/api/analytics`, `/api/heal-log`) but rejects POST with HTTP 403. For QA leads / external dashboards. |

Two heal flows converge on the dashboard:

| Source | Default behaviour | Dashboard action |
|---|---|---|
| Automatic strategies (attribute / type-hint / role / label / text) | Commit immediately — test proceeds, heal persisted to `self-heals/healed-locators[.<env>].json` | **Confirm** marks the heal `appliedToSourceAt` AND rewrites the source `*.locators.ts` file's `selector:` field so future runs use it as the primary probe. **Revert** clears the heal — file is purged from the heal-store. |
| AI Vision (strategy 8) | Queued as `pendingSuggestion` — operator must approve before commit | **Approve** commits + writes back. **Reject** discards. |

The heal store is **heal-only** — pure-registered baseline entries don't get written, so the JSON only contains entries a human needs to review. `HEAL_STORE_PATH` controls the location; `ENV` / `HELIX_ENV` auto-suffix it (`healed-locators.qa4.json`) so multi-env runs don't trample each other. Set `HEALING_DASHBOARD_PORT=0` to disable the UI in fully automated pipelines — heals are still written to disk for async review. A tab-separated audit log (`HEAL_LOG_PATH`, default `./self-heals/heal.log`) is appended per heal.

---

## Authentication

Agents 1–4 share `shared/auth.py` for ADO authentication via MSAL. The module uses the same cache as `microsoft/azure-devops-mcp`, so signing into any Microsoft tool (VS Code, `az login`) also authenticates these agents.

**Token acquisition sequence:**

```
1. Load ~/.msal-cache/msal-cache.json
2. msal.acquire_token_silent()  →  token found and valid → Bearer token (zero UI)
3. msal.acquire_token_interactive()  →  browser opens once → cached for ~90 days
CI/CD: AZURE_CLIENT_ID + AZURE_CLIENT_SECRET + AZURE_TENANT_ID  →  service principal
```

**Constants (must match `microsoft/azure-devops-mcp` exactly):**

| Constant | Value |
|---|---|
| `_ADO_SCOPE` | `499b84ac-1321-427f-aa17-267ca6975798/.default` |
| `_DEFAULT_CLIENT_ID` | `04b07795-8ddb-461a-bbee-02f9e1bf7b46` (Azure CLI) |
| `_CACHE_FILE` | `~/.msal-cache/msal-cache.json` |

---

## MCP Configuration

**VS Code / GitHub Copilot (`.vscode/mcp.json`):**

```json
{
  "servers": {
    "qa-test-case-manager":    { "command": "/path/.venv/bin/qa-test-case-manager" },
    "qa-gherkin-generator":    { "command": "/path/.venv/bin/qa-gherkin-generator" },
    "qa-playwright-generator": { "command": "/path/.venv/bin/qa-playwright-generator" },
    "qa-helix-writer":         { "command": "/path/.venv/bin/qa-helix-writer" },
    "qa-migration":            { "command": "/path/.venv/bin/stlc-migrate" },
    "playwright": { "type": "http", "url": "http://localhost:8931/mcp" }
  }
}
```

**Claude Code (`.mcp.json`):**

```json
{
  "mcpServers": {
    "qa-test-case-manager":    { "command": "/path/.venv/bin/qa-test-case-manager" },
    "qa-gherkin-generator":    { "command": "/path/.venv/bin/qa-gherkin-generator" },
    "qa-playwright-generator": { "command": "/path/.venv/bin/qa-playwright-generator" },
    "qa-helix-writer":         { "command": "/path/.venv/bin/qa-helix-writer" },
    "qa-migration":            { "command": "/path/.venv/bin/stlc-migrate" },
    "playwright": { "command": "npx", "args": ["@playwright/mcp@latest", "--isolated"] }
  }
}
```

Use `qa-stlc mcp-config [--vscode]` to generate this manually, or let
`stlc-migrate` emit `.mcp.json` automatically as part of a migration (with
absolute binary paths discovered from the active venv).

---

## Migration (Existing Playwright project → Helix-QA)

Use `stlc-migrate` to bring an existing project onto the agent rails. See
`skills/migrate-framework/SKILL.md` for the full pipeline reference. In
short:

```bash
stlc-migrate --source ./my-old-tests --helix ./helix-qa --integration ado \
  [--preserve-tree] [--features=vrt] [--conflict overwrite|skip|interactive]
```

The tool writes the entire Helix-QA tree, including the healer scaffolds
listed above, `.claude/skills/`, `.github/copilot-instructions/`,
`AGENT-BEHAVIOR.md`, `ORCHESTRATION_RULES.md`, and `.mcp.json` with
absolute paths to every agent in the active venv. Healed selectors land in
`self-heals/healed-locators[.<env>].json` per the new healing architecture.

Optional flags layer behaviour on top:

- `--preserve-tree` mirrors the source's subfolder grouping under
  `src/test/features/`, `src/test/steps/`, `src/pages/`, `src/locators/`
  (e.g. `src/test/features/amplify/e-commerce/checkout.feature`). Sub-dir
  names are kebab-cased.
- `--features=vrt` drops the VRT runtime (`src/helper/vrt/*`, step defs,
  `vrt:baseline` / `vrt:test` npm scripts, `pngjs`/`pixelmatch`/`fs-extra`
  deps) and appends the VRT block to `.env.example`. Auto-enabled when the
  source already uses VRT.
- `--conflict overwrite` refreshes existing files **and** sweeps
  `src/test/features|steps` and `src/pages|locators` for stale role-suffix
  files this migration didn't write. Useful when toggling layout flags
  (flat ↔ preserve-tree) so the target self-corrects in one pass.

---

## Design Decisions

**Why four separate MCP servers rather than one?** Each agent has a distinct concern and a distinct skill file. A team can use just Agent 1 without knowing anything about Playwright. Failures in one server do not affect the others. All four share `shared/auth.py` to avoid duplication.

**Why Python for the MCP servers?** The MCP Python SDK maps cleanly to stdio transport. `requests` covers all ADO REST needs. Code generation is pure string manipulation — no TypeScript compiler is needed server-side.

**Why not auto-commit healed selectors?** AI suggestions are probabilistic. A healed selector that works for one run may not be semantically correct. HealingDashboard makes every AI-generated change visible, attributable, and reversible before it enters version control.

**Why validate selectors at generation time?** AI-generated selectors frequently match zero or multiple elements. Validating against the live browser during the `context_map` capture phase eliminates this class of error entirely — only selectors matching exactly one element reach `locators.ts`.