# C8 Cross-Component End-to-End Test Suite

## Overview

The C8 Cross-Component End-to-End Test Suite represents an extensive collection of end-to-end tests meticulously tailored for Camunda C8, encompassing the entire product stack. This specialized suite is dedicated to test scenarios that traverse multiple C8 components, harnessing the power of Playwright with TypeScript. It adheres to the Page Object Model design pattern, ensuring effortless test creation and seamless maintenance. The focus of the test suite is to align with the C8 persona profiles and their user flows as well as the most important cross-component user flows for the product stack determined by each product team. The persona profiles that will be focused on can be observed in this [link](https://confluence.camunda.com/display/HAN/Personas)

## Supported Testing Platforms

### C8 SaaS, C8 Self-Managed (SM) with Helm, and C8 Run

This test suite supports testing across three primary platforms:

1. **C8 SaaS**
2. **C8 Self-Managed with Helm Chart**
3. **C8 Run**

## Supported Cluster Versions for SaaS

For the Camunda SaaS offering, the test suite supports following cluster versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X
- 8.10.X

**By default, the test suite is configured to run tests on the latest snapshot versions.** If you wish to run tests on a different generation, you can adjust the configurations in the `.github/c8_versions.json` file.

## Supported Versions for SM with Helm

For the Camunda SM Helm offering, the test suite supports following versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X

The test suite primarily uses Elasticsearch as the default search engine. However, it also supports OpenSearch, with dedicated workflows explicitly named to indicate OpenSearch as the selected engine.

## Supported Versions for C8Run

For the Camunda C8Run offering, the test suite supports following versions:

- 8.6.X
- 8.7.X
- 8.8.X
- 8.9.X
- 8.10.X

### GitHub Actions for Testing

GitHub Actions can be used to facilitate testing across various cluster versions and components:

#### SaaS

For SaaS, a test run can be manually triggered for specific minor versions and their associated cluster generation. This can be done through the [SaaS On-Demand GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_saas_manual.yml). These tests run against the INT environment.

The workflow expects the following arguments:

| Argument                | Description                                        | Examples                                    |
| ----------------------- | -------------------------------------------------- | ------------------------------------------- |
| C8 Minor Version        | Minor version on format `<major>.<minor>`          | `8.9`, `8.8`                                |
| Cluster Generation Name | The full name of the cluster generation (not UUID) | `Camunda 8.8+gen16`, `Camunda 8.9.0-alpha5` |

> [!NOTE]
> The cluster generation name used in the workflow run must be part of the Stable, Internal Dev, QA, or Alpha channel for it to be testable. A generation can be added to a channel in the [Camunda Accounts dashboard](https://accounts.ultrawombat.com/consoleadmin/channels)

#### SaaS Upgrade Path Testing

To validate an upgrade between two SaaS cluster versions, use the [SaaS Upgrade Path Testing GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/saas-upgrade-path-testing.yml).

The workflow provisions a fresh organization, runs pre-upgrade smoke tests, performs the UI-driven cluster upgrade, then validates the upgraded cluster. It supports the following upgrade paths:

- 8.7 → 8.8
- 8.8 → 8.9

The workflow expects the following arguments:

| Argument          | Description                                              | Examples                       |
| ----------------- | -------------------------------------------------------- | ------------------------------ |
| Source Version    | Minor version to upgrade **from**                        | `8.8`, `8.9`                   |
| Source Generation | Full name of the source cluster generation               | `Camunda 8.8+gen16`            |
| Target Version    | Minor version to upgrade **to**                          | `8.9`, `8.10`                  |
| Target Generation | Full name of the target cluster generation               | `Release Test QA-8.9.0-alpha5` |
| Tasklist Version  | Tasklist UI mode (`v1` or `v2`); forced to `v1` for <8.8 | `v1`, `v2`                     |

#### SM with Helm

For SM using Helm, a test run can manually trigger by specifying any component version as an input. TThis can be done through the following workflows:

- [SM On-Demand GitHub Action for versions ≤ 8.7.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual.yml)
- [SM On-Demand GitHub Action for versions ≥ 8.8.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_install_8_8_plus.yml)

#### C8Run

For C8Run, a test run can be manually triggered for specific minor versions, for each supported Operating System. This can be done through the following workflows:

- [C8Run On-Demand with Linux GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_linux.yml)
- [C8Run On-Demand with macOS GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_mac.yml)
- [C8Run On-Demand with Windows GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_windows.yml)

#### Flakiness Fix Agent

Found a flaky test? The [Flakiness Fix Agent](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_flakiness_fix.yml) is a single workflow that works across all three platforms (SaaS, SM, C8Run). It runs the target test 100× against the real integration environment, sends any failures to Claude for analysis, applies fixes, and iterates until it reaches 100% pass rate — then opens a PR for you to review.

> Provide the test file path (e.g. `tests/8.10/smoke-tests.spec.ts`) and the platform is auto-detected from the path. See the [Flakiness Fix Agent section in AGENTS.md](./AGENTS.md#flakiness-fix-agent) for full usage details and how to prepare SM credentials.

## Prerequisites

1. Before running the test suite, make sure you have the following prerequisites installed:

- Node.js (with npm, >= v14)

## Getting Started

### Vault Token

To run the tests, you need your personal Vault token. You should normally have access to this token. If not, you’ll need to reach out to IT or Infra teams.

1. Clone the repository:

   ```bash
   git clone https://github.com/camunda/c8-cross-component-e2e-tests.git
   cd c8-cross-component-e2e-tests
   ```

2. Install dependencies and auto-configure Vault CLI for Mac, Linux, and Windows:

   ```bash
   npm install
   npx playwright install
   ```

   - This will automatically download the correct Vault CLI for your system (Mac, Windows, Linux) into the `bin/` directory.
   - No manual Vault CLI install or admin rights required.

3. Create your `.env` file automatically:

   ```bash
   npm run setup
   ```

   - When prompted, paste your Vault token from [vault.int.camunda.com/ui/vault/access](https://vault.int.camunda.com/ui/vault/access).
   - The script fetches all necessary credentials and secrets, applies in-place transformations to the required email usernames, and generates your `.env` file in the project root.

4. Set the following variables manually in `.env`:

   After running `npm run setup`, open your `.env` file and fill these:

   - `TASKLIST_VERSION` (Set to `v1` or `v2` for 8.8+)
   - `PLAYWRIGHT_BASE_URL` (Base_url for Helm chart testing, or remove it completely)
   - `CLUSTER_VERSION` (Cluster version on INT to test, or leave blank)
   - `MINOR_VERSION` (Minor version, e.g. `SM-8.8`, corresponds to the test folders in this repo)

   The setup script will leave these blank and clearly mark them for manual input.

   If you do not have immediate Vault access or are otherwise blocked, you can copy the required `.env` values from Keeper:

   - In Keeper, go to **Shared Engineering All → C8 Cross Component E2E Test Suite Credentials**
   - Copy all provided environment variable values
   - Paste them into your local `.env` file

**Setting up your own organization (optional)**

Request the creation of an organisation identical to `Automation QA`  
**Org ID:** `3626cf46-8f97-4711-81b3-ecfc341348c2`  
**Environment:** `INT`

**Requirements:**

- You must be set as the **Owner**
- Name the organisation: `Automation QA - Your_Name`

**Inviting test users**

Within the `Automation QA - Your_Name` organisation, invite the following users:

- Alias of `C8_USERNAME`
- Alias of `C8_USERNAME_TEST`
- Alias of `C8_USERNAME_TEST_2`
- Alias of `C8_USERNAME_TEST_3`
- Alias of `C8_USERNAME_TEST_4` (v8.8+)
- Alias of `C8_USERNAME_TEST_5` (v8.8+)
- Alias of `C8_USERNAME_TEST_6` (v8.8+)
- Alias of `C8_USERNAME_TEST_7` (v8.8+)
- Alias of `C8_USERNAME_TEST_8` (v8.8+)
- Alias of `C8_USERNAME_TEST_9` (v8.8+)

**Passwords & invitation acceptance**

1. Retrieve the Gmail passwords from Keeper:  
   Shared → Engineering All → **C8 Cross Component E2E Test Suite Credentials**
2. Use these emails to sign up and accept the invitations.
3. Align the passwords with:
   - `C8_PASSWORD`
   - `C8_PASSWORD_TEST`

## Running Smoke Tests in CI

### SaaS On-Demand Generation Creation Smoke

To trigger the SaaS smoke tests in CI, use the workflow:

[SaaS On-Demand Generation Creation Smoke](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_saas_manual_generation_creation_smoke.yml)

- You’ll be prompted to input the version you want to test for each component.
- The **minor version** is simply the minor version your PR is targeting.
  - _Example:_ If your PR targets `8.9`, then the minor version is `8.9`.

### Helm Chart Smoke Tests (Self-Managed)

For Helm chart tests (if you want to run only the smoke test), use the workflow:

[SM On-Demand Smoke 8.8+](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_smoke_8_8_plus.yml)

- Provide the component images you want to test against as workflow inputs.

## Running Tests Locally

1. Run Tests in Headless Mode

To run the test suite in headless mode (without a graphical user interface), use the following command:

```bash
npm run test:local
```

To run one specific test file in headless mode (without a graphical user interface), use the following command:

```bash
npx playwright test --project=chromium tests/your-test-folder/your-test-spec
```

2. Run Tests with UI (Interactive Mode)

To run the test suite with the UI (interactive mode), append --ui at the end of the npx playwright test command (note the test 'Create Cluster' must be run first):

```bash
npx playwright test --ui
```

### Additional Notes for Local Testing

#### C8 SaaS

For a full test run, no additional configuration is required. However, if you are running a single test, ensure you have a healthy and active cluster.

If you want to run tests against the production environment, add the following to your local .env file:

```bash
IS_PROD=true
```

#### C8 Self-Managed (SM) with Helm

To test locally on the SM Helm environment, you need an active SM instance. You can create one through any of the following GitHub Actions workflows:

- [Basic Helm Chart for versions ≤ 8.7.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_no_tests.yml)
- [Basic Helm Chart for versions ≥ 8.8.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_install_8_8_plus.yml)
- [Helm Chart With RBA Enabled for versions ≤ 8.7.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_rba_no_tests.yml)
- [Helm Chart With RBA Enabled for versions ≥ 8.8.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_rba_8_8_plus.yml)
- [Helm Chart With Multi Tenancy Enabled for versions ≤ 8.7.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_mt_no_tests.yml)
- [Helm Chart With Multi Tenancy Enabled for versions ≥ 8.8.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_mt_8_8_plus.yml)
- [Basic Helm Chart with OpenSearch for versions ≤ 8.7.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_no_tests_opensearch.yml)
- [Basic Helm Chart with OpenSearch for versions ≥ 8.8.X](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_opensearch_8_8_plus.yml)

Once the SM instance is created, copy the generated test URL and input it into your `.env` file under the `PLAYWRIGHT_BASE_URL` variable.

Note: For workflows ≥ 8.8.x, if you only need to retrieve the test URL without running tests, simply uncheck the Enable tests checkbox.

If you want to run tests with Resource-Based Access (RBA) enabled, add the following to your .env file, otherwise SM tests default to RBA disabled:

```bash
IS_RBA=true
```

If you want to run tests with Multi-Tenancy (MT) enabled, add the following to your .env file, otherwise SM tests default to MT disabled:

```bash
IS_MT=true
```

#### C8 Run

To test locally against C8 Run, you must set up a local instance of C8 Run. Please follow the instructions provided in the [C8 Run README.md](https://github.com/camunda/camunda/blob/main/c8run/README.md). By default, the BASE_URL is localhost:8080, however if this is configured as something differently, tests can be run against this by inputting the new BASE_URL into your `.env` file under the `PLAYWRIGHT_BASE_URL` variable.

## Running MCP Tests (c8Run Only)

The MCP (Model Context Protocol) test suite requires the MCP test server to be running locally. These tests are designed for **c8Run environments only** (version 8.9+), as they require localhost access to the MCP server.

The MCP test server is available as a Docker image from the [camunda/mcp](https://github.com/camunda/mcp) repository.

### Running MCP Tests Locally

To run MCP tests locally with c8Run, you need to start both the MCP test server and c8Run:

1. Start the MCP server:

   **macOS/Windows:**

   ```bash
   docker run -d \
     --name mcp-test-server \
     -p 12001:12001 \
     registry.camunda.cloud/mcp/mcp-test-server:latest
   ```

   **Linux:**

   ```bash
   docker run -d \
     --name mcp-test-server \
     --network host \
     registry.camunda.cloud/mcp/mcp-test-server:latest
   ```

2. Start c8Run locally (follow [c8Run setup instructions](https://github.com/camunda/camunda/blob/main/c8run/README.md))

3. Run the MCP tests (the `MCP_SERVER_AVAILABLE` env var is required to enable them):

   ```bash
   MCP_SERVER_AVAILABLE=true npx playwright test --project=chromium-v2 tests/c8Run-8.9/mcp-user-flows.spec.ts
   ```

4. Stop the MCP server when done:

   ```bash
   docker stop mcp-test-server && docker rm mcp-test-server
   ```

### MCP Server Configuration

The MCP test server runs on port `12001` by default and provides the following endpoints:

- **MCP Endpoint**: `http://localhost:12001/mcp`
- **Health Check**: `http://localhost:12001/actuator/health`

You can override the MCP server URL by setting the `MCP_SERVER_URL` environment variable in your `.env` file:

```bash
MCP_SERVER_URL=http://localhost:12001
```

### Available MCP Operations

The comprehensive MCP test suite validates all 7 operations of the MCP Remote Client connector:

#### Tools Operations

- **tools/list**: Lists all available MCP tools (e.g., `greet`, `add`, `echo`)
- **tools/call**: Executes a specific tool with arguments (tests `greet` tool with `name: "John"`)

#### Resources Operations

- **resources/list**: Lists available MCP resources (tests `file://jokes-guide.md`)
- **resources/templates/list**: Lists resource templates (validates empty array response)
- **resources/read**: Reads content from a specific resource URI (`file://jokes-guide.md`)

#### Prompts Operations

- **prompts/list**: Lists available MCP prompts (tests `get-greeting` prompt)
- **prompts/get**: Retrieves a specific prompt with arguments (tests `get-greeting` with `name: "John"`)

Each operation stores its result in a dedicated process variable:

- `listToolsResult`, `callToolResult`, `listResourcesResult`, `listResourcesTemplateResult`, `readResourceResult`, `listPromptsResult`, `getPromptResult`

The test suite validates the JSON structure and content of each operation's response to ensure complete connector functionality.

### Available MCP Tools

The MCP test server provides the following tools for testing:

- **String Tools**: `echo`, `uppercase`, `lowercase`, `base64Encode`, `base64Decode`
- **Greeting Tools**: `greet` (supports EN, DE, ES, IT)
- **Calculator Tools**: `add`

For more details about the MCP server and its capabilities, see the [MCP repository](https://github.com/camunda/mcp).

### CI/CD Integration

In CI/CD pipelines, the MCP server runs **only on Linux (`ubuntu-latest`)** and only under the following conditions:

- c8Run version **>= 8.9**
- Tasklist **v2** mode (not v1)
- **Detached mode** only (not Docker Compose mode)

The `MCP_SERVER_AVAILABLE` environment variable is set to `true` when the server starts successfully. The test suite checks this variable and skips all MCP tests if it is not set, so runs on macOS, Windows, or unsupported versions will skip gracefully rather than fail.

**Why macOS and Windows are excluded from CI:**

- **macOS runners** (`macos-latest`, `macos-14`): GitHub Actions macOS runners do not have Docker pre-installed, so `docker run` fails with `docker: command not found`.
- **Windows runners**: The MCP test server image (`registry.camunda.cloud/mcp/mcp-test-server:latest`) has no `windows/amd64` manifest, so `docker run` fails with "no matching manifest for windows/amd64".

MCP tests can still be run locally on macOS and Windows using the instructions above.

## Automated Nightly Test Runs

The test suite is configured to run automatically every night. The results of these nightly test runs are published on TestRail. If any test fails during these automated runs, the Designated Responsible Individual (DRI) of the project is notified via email for immediate attention and resolution. On a nightly basis, the test suite is configured to run the following tests:

Failures are classified into categories (Infrastructure, Flaky, Product, Mixed) to aid triage.
See [docs/ci-failure-categories.md](docs/ci-failure-categories.md) for the full model and criteria.

- **C8 SaaS**: Tests are run against a SNAPSHOT generation cluster.
- **C8 SM with Helm**: Tests are run against the alpha branch of the [camunda-platform-helm repository](https://github.com/camunda/camunda-platform-helm), using SNAPSHOT images for each component. This is peformed with both Elasticsearch and OpenSearch.
- **C8 Run**: Tests are run against the main branch of the [camunda/camunda repository](https://github.com/camunda/camunda).

## Test Suite Structure

The test suite is organized using the Page Object Model design pattern, where each test file contains test scenarios related to a specific component or feature of the Camunda C8 product stack. Additionally, it leverages separate page objects for each web page or component being tested.

You can find the page objects in the pages directory, and the test files in the tests directory.

For example:

Page files:

LoginPage.ts - Page object for the login functionality.

Test files:

login.spec.ts - Tests related to the login functionality using the LoginPage object.

Feel free to add more test files or page objects to expand the test coverage based on your specific requirements.

## GitHub Actions Workflow Naming Convention

To maintain consistency and clarity in workflow names displayed in the GitHub Actions UI, we use the following naming pattern:

### **Naming Pattern:**

`<Distribution> <Trigger> <Scope (if needed)>`

- **Distribution:**

  - `SaaS` → Cloud-based environment
  - `SM` → Self-Managed using the Helm Chart
  - `C8Run` → C8Run environment

- **Trigger:**

  - `Nightly` → Runs automatically every night
  - `PR` → Runs when a pull request is opened/updated
  - `On-Demand` → Runs when triggered via **workflow dispatch**

- **Scope** (optional):
  - `Linux`, `Connectors`, `OpenSearch`, `No Tests` etc.

### **Examples**

| Workflow Name                      | Meaning                                                                                              |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `SaaS Nightly 8.8`                 | Runs SaaS tests nightly against the latest 8.8 SNAPSHOT                                              |
| `SM Nightly Chrome 8.7 OpenSearch` | Runs SM tests nightly against the latest 8.7 SNAPSHOT in Chrome with OpenSearch                      |
| `SaaS/SM/C8Run PR`                 | Runs SaaS or Self-Manged or C8Run tests on PR updates                                                |
| `SM On-Demand MT No Tests`         | Creates a Self-Managed environment with Multi-Tenancy Enabled via workflow dispatch with no test run |

---

By following this convention, we ensure **clarity, consistency, and alignment with GitHub terminology**.

### SM Possible Directory Names

| Directory Name                | Meaning                                                                                              |
| ----------------------------- | ---------------------------------------------------------------------------------------------------- |
| `camunda-platform-8.9-latest` | Runs SM tests against the latest 8.9 SNAPSHOT                                                        |
| `camunda-platform-8.8-latest` | Runs SM tests against the latest 8.8 SNAPSHOT                                                        |
| `camunda-platform-8.7-latest` | Runs SM tests against the latest 8.7 SNAPSHOT                                                        |
| `camunda-platform-8.6-latest` | Runs SM tests against the latest 8.6 SNAPSHOT                                                        |
| `camunda-platform-8.9`        | Runs SM tests against the 8.9 directory. This is where the `release-candidate-8.9` branch is located |
| `camunda-platform-8.8`        | Runs SM tests against the 8.8 directory. This is where the `release-candidate-8.8` branch is located |
| `camunda-platform-8.7`        | Runs SM tests against the 8.7 directory. This is where the `release-candidate-8.7` branch is located |
| `camunda-platform-8.6`        | Runs SM tests against the 8.6 directory. This is where the `release-candidate-8.6` branch is located |
| `all-latest`                  | Runs SM tests against all latest versions                                                            |

### c8Run Supported Versions, Modes and Projects

| Version | Docker Allowed? | OS When Docker     | OS When Non-Docker     | Projects (Non-Docker)               | Projects (Docker) |
| ------- | --------------- | ------------------ | ---------------------- | ----------------------------------- | ----------------- |
| 8.6     | No              | —                  | ubuntu, macOS          | —                                   | —                 |
| 8.7     | Yes             | ubuntu-latest only | ubuntu, macOS, windows | chromium (ES)                       | chromium (ES)     |
| 8.8     | Yes             | ubuntu-latest only | ubuntu, macOS, windows | chromium-v1 (ES) + chromium-v2 (ES) | chromium-v2 (ES)  |
| 8.9     | No              | —                  | —                      | chromium-v2 (RDBMS)                 | —                 |
| 8.10    | No              | —                  | —                      | chromium-v2 (RDBMS)                 | —                 |

## Viewing Test Results

### Locally:

The test results are presented as a comprehensive list in the terminal when all tests pass. Additionally, within the
project's test-results folder, you can find the test results accompanied by images. Moreover, if any test encounters
issues or instability, an HTML report automatically opens, facilitating in-depth analysis.

### On the CI:

Within the CI environment, test results are accessible through TestRail, located in the C8 Project folder under Test
Runs. These reports not only include detailed test results but also feature screenshots. In case of a test failure, you
will find the stacktrace. Furthermore, the CI environment offers extensive tools for in-depth analysis of test results
and reports.

### Recording video:

To record video, you need to set the following config in the `use` block in the `playwright.config.ts` file:

```typescript
use: {
  video: 'retain-on-failure';
}
```

Don't keep this enabled, or you'll blow up the disk space quickly. Files are recorded in the `test-results/videos`
folder locally and to TestRail in the CI.

## Contributing

Contributions to the C8 Cross-Component End-to-End Test Suite are welcome! If you find any issues or want to add new test cases, please submit a pull request or open an issue on GitHub.

General guidelines:

- Anyone can contribute.
- The reviewer of the pull request must be someone from the Test Automation Team.
- Test cases must adhere to the Page Object Model design pattern.
- Test cases must involve multiple C8 components.

### Multi-version PRs

Tests and page objects are organized by platform and minor version (e.g. `tests/8.9/`, `tests/8.10/`, `tests/SM-8.9/`).

| Change type | Recommendation |
|---|---|
| Small mechanical fix (e.g. selector rename, one-liner) applied consistently across versions | Acceptable in a single PR — note it in the PR description |
| New test, new helper, or any change that requires review of logic per-version | **Split into one PR per version** |
| Refactoring or structural change | **Split into one PR per version** |

Reviewers are expected to understand the context of each version they review. Large multi-version PRs make meaningful review impractical and hide regressions. When in doubt, split.

**How to split:**

1. Create a branch per version: `feat/my-change-8-9`, `feat/my-change-8-10`, etc.
2. Apply your change to the first branch and open a PR.
3. Cherry-pick or re-apply to subsequent branches and open follow-up PRs.
4. Link the sibling PRs in each description for traceability.

### Test runs before review

**Do not request a review — and do not assign reviewers — until a successful test run is linked in your PR.**

This applies to all changes that touch test specs, page objects, fixtures, or utilities.

#### Automatic test runs on PR

The [SaaS/SM/C8Run PR](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright.yml) workflow runs automatically on every pull request. It detects which `tests/` and `pages/` sub-directories have changed and runs the tests for those versions/platforms.

**This works reliably only when a PR touches a single version and a single platform.** Examples that work automatically:

- Only `tests/8.10/` and/or `pages/8.10/` are changed → 8.10 SaaS tests run.
- Only `tests/SM-8.9/` and/or `pages/SM-8.9/` are changed → SM-8.9 tests run.

If no `tests/` or `pages/` folders are changed (e.g. infrastructure or tooling changes), the test run is skipped automatically — state this explicitly in the PR description.

#### When you must trigger tests manually

The automatic PR workflow **does not reliably cover** changes that span multiple versions or multiple platforms. In those cases you must trigger the relevant on-demand workflows manually — or, preferably, split the PR (see [Multi-version PRs](#multi-version-prs) above).

| Scenario | Action required |
|---|---|
| Only one version and one platform changed | Wait for the automatic PR workflow |
| Multiple SaaS versions changed (e.g. `8.9` + `8.8`) | Trigger [SaaS On-Demand](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_saas_manual.yml) separately for each version, **or split the PR** |
| Multiple SM versions changed (e.g. `SM-8.9` + `SM-8.8`) | Trigger [SM On-Demand ≥8.8](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual_install_8_8_plus.yml) or [SM On-Demand ≤8.7](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_sm_manual.yml) separately for each version, **or split the PR** |
| SaaS + SM versions changed together (e.g. `8.9` + `SM-8.8`) | Trigger both the SaaS and SM on-demand workflows, **or split the PR** |
| Connector tests | [SaaS Connectors](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_saas_connectors.yml) or the relevant SM connector workflow |
| C8 Run tests (`tests/c8Run-8.x/`) | [C8Run On-Demand Linux](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_linux.yml) |

#### Steps

1. Wait for the automatic PR workflow to complete **or** trigger the relevant on-demand workflow(s) manually.
2. Confirm each run passes (or document known flakiness with a re-run showing it as non-deterministic).
3. Paste every workflow run URL in the **Definition of Done** section of the PR before assigning reviewers.

Reviewers are expected to verify that a passing run for every changed version/platform is linked before approving.

### PR checklist summary

Before marking a PR as ready for review:

- [ ] Change scope is appropriate — large multi-version changes are split into separate PRs
- [ ] A relevant test workflow has been run and **passed**
- [ ] The workflow run URL is pasted in the PR description
- [ ] If new tests were added: add the `testrail-test-case` label after the test run passes to auto-enrich TestRail cases (see [TestRail Integration](#testrail-integration)); update Confluence if applicable

### TestRail Integration

Test case documentation in TestRail is **automatically synced** from spec files. When a PR adds or modifies test cases, the sync workflow uses AI (GitHub Copilot) to generate human-readable step descriptions directly from the Playwright test code.

#### How to trigger the sync

**Option A — Label after a passing test run (recommended)**

1. Run the relevant test workflow and confirm it passes.
2. Add the `testrail-test-case` label to the PR.
3. The [Sync TestRail Cases](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/sync-testrail-cases.yml) workflow fires automatically, detects the changed spec files, and enriches the matching TestRail cases with AI-generated steps.

> Re-triggering is easy: remove the label and add it again.

**Option B — Manual dispatch**

Go to [Actions → Sync TestRail Cases → Run workflow](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/sync-testrail-cases.yml) and provide the version and spec file path manually. Useful for one-off enrichment without opening a PR.

#### What the sync does

- Finds the matching TestRail case by automation ID (set by the JUnit reporter) or normalised title.
- Skips cases that already have steps on the correct template.
- Re-enriches cases that are on the wrong TestRail template (fixes template + steps in one pass).
- Calls the GitHub Copilot API to produce Action + Expected Result step rows from the raw Playwright code.

#### Access

If you do not have access to TestRail, contact the Test Automation Team to request access.

Thank you for using the C8 Cross-Component End-to-End Test Suite for Camunda C8 full product stack e2e testing. Happy testing! If you have any questions or need assistance, feel free to reach out to the maintainers for support.
