# Checksum.ai Runtime

### Quick Start

1. Install the package using `npm install -D checksumai` or `yarn add checksumai -D`.
2. Navigate to the directory where you want to initialize the Checksum tests folder and run `npx checksumai init`.
3. In the newly created `checksum` folder:
   1. Edit `checksum.config.ts` and add the necessary configuration, including your API key. Define `BASE_URL` in `checksum/.env` because the starter config reads `process.env.BASE_URL!`.
   2. Update `login.ts` with your Playwright login flow. See the Login Function section below for the current params-object API.
   3. Review the starter examples in `checksum/tests/examples/example.checksum.spec.ts` and `checksum/tests/examples/example.checksum.md`.
   4. Re-running `npx checksumai init` later backfills any missing starter files without overwriting files you already customized.
4. Run `npx playwright install --with-deps` to install Playwright dependencies.
5. Run `npx checksumai test` to execute the starter `.checksum.spec.ts` example and verify your login setup.
6. If you haven't already, visit [app.checksum.ai](https://app.checksum.ai) to complete the configuration and generate a test. Then, wait for the pull request (PR) to be created and approve it.

### Starter Files

- `checksum/login.ts` shows the runtime login callback signature and where `environment`, `user`, and `config` come from.
- `checksum/tests/examples/example.checksum.spec.ts` is a minimal TypeScript starter using `init()`, `defineChecksumTest(...)`, and `login(page)`.
- `checksum/tests/examples/example.checksum.md` is a minimal markdown flow starter. Replace the placeholder `appId` and `checksumTestId` values before using it.
- Repo Mirror reads both `.checksum.spec.ts` and `.checksum.md` files, so both starter formats are generated during `checksumai init`.

### Login Function

1. This function is executed at the start of each test.
2. We recommend using a consistent seeded user for every test. For example, before each test, call a webhook that creates a user, seeds it with data, and returns the username and password. This approach ensures test reliability and allows parallel test execution. Configure this webhook [in your project](https://app.checksum.ai/#/settings/wizard) for consistent test generation.
3. After logging in, assert that the login was successful. Playwright waits for assertions to be correct, ensuring that the page is ready for interaction before proceeding.
4. To reuse authentication states between tests, refer to the Playwright guide on [authentication](https://playwright.dev/docs/auth). At the start of the login function, check if the user is already authenticated and return if so.
5. Use the `ChecksumLoginFunction` type to correctly implement the login function:
   ```
     import { ChecksumLoginFunction } from "@checksum-ai/runtime";

     const login: ChecksumLoginFunction = async (
       page,
       { environment, user, config }
     ) => {
       // environment contains the selected environment from checksum.config.ts.
       // user contains the selected credentials for that environment.
       // config is the loaded checksum.config.ts object.
       const loginUrl = environment.loginURL ?? "/login";

       await page.goto(loginUrl);
       await page.getByPlaceholder("Email").fill(user.username ?? "");
       await page.getByPlaceholder("Password").fill(user.password ?? "");
       await page.getByRole("button", { name: "Log in" }).click();
       await page.waitForURL(new URL("/", environment.baseURL).toString());
     };

     export default login;
   ```

When you define multiple environments or roles, helpers like `login(page, { environment, role })` and `getEnvironment({ name, userRole })` decide which `environment` and `user` reach this function.

## Checksum AI Magic

The tests generated by Checksum are based on Playwright. When executed using the Checksum runtime CLI with an API key, Checksum enhances Playwright's functionality, improving test reliability and automatically maintains tests. Each enhancement is enabled/disabled using the `checksum.config.ts` file.

#### Smart Selectors
When generating tests, Checksum stores extensive metadata for each action (stored in the `checksum/test-data` folder). If a traditional selector fails, this metadata is used to locate the target element. For example, if a test identifies an element by its ID but the ID changes, Checksum utilizes other data points (e.g., element class, text, parents) to locate the element. Use the `checksumSelector("<id>")` method to link an action to its metadata. Do not alter the IDs.


#### Checksum AI
If Smart Selectors also fail, Checksum's custom-trained model can regenerate the failed section of the test. In such cases, the model might add, remove, or alter actions to achieve the same objectives, without changing the assertions. The assumption is that as long as the assertions pass, the model has successfully fixed the test. Use the `checksumAI` method to wrap each step in order to achieve this behavior.

Example using both `checksumSelector` and `checksumAI`:
```js
 await checksumAI("Save the form and continue", () =>
      page
        .checksumSelector("gIexv")
        .getByRole("button", { name: "Save" })
        .click()
    );
```

You can modify the description as needed for our model. Additionally, you can include steps with only ChecksumAI descriptions, prompting our model to generate the Playwright code. For example, `await page.checksumAI("Click on 'New Task' button")` without the actual action will have our model generate the necessary Playwright code. You can even author entire tests in this manner.


### Mock Data

When generating tests, Checksum records all network responses, allowing tests to run in the same context. These recordings are stored as HAR files, stored in the  `checksum/test-data/har` folder. Using this method is particularly useful for debugging newly generated tests, especially if your testing database/context differs from the one used for test generation. Note that if your network responses' format changes, the mocked data may not work as expected. 

## Checksum Config
Alongside standard test run configurations found in `playwright.config.ts`, use the `checksum.config.ts` file to configure your test runs.

```js
{
  /**
   * Checksum Run mode.
   * Normal is currently the only supported mode.
   */
  runMode: RunMode.Normal,

  /**
   * Insert here your Checksum API key.
   * You can find it in https://app.checksum.ai/#/settings/
   */
  apiKey: "<API key>",

     /**
   * Define your test run environments and test users within each environment.
   * The environments must be aligned with those set here:
   * https://app.checksum.ai/#/settings/
   */
  environments: [{
      name: 'staging';
      users: [
         {
            role: "host",
            username: "<host username>",
            password: "<host password>",
         },
         {
            role: "guest",
            username: "<guest username>",
            password: "<guest password>",
         },
      ],
      baseURL: 'staging.mydomain.com';
      loginURL: 'staging.mydomain.com/login';
      default: true;
  }]

  options: {
    /**
     * Whether to use Checksum Smart Selector in order to
     * recover from failing to locate an element for an action
     */
    useChecksumSelectors: true,
    /**
     * Whether to use Checksum AI in order to 
     * recover from a failed action or assertion
     * @actions - enable recovery from checksumAI steps
     * @assertions - enable recovery from assertion steps
     * @visualComparison - enable visual comparison recovery when using toHaveScreenshot assertions 
     */
    useChecksumAI: { actions: true, assertions: false, visualComparison: false },
    /**
     * Whether to use recorded network responses when running your tests
     */
    useMockData: false,
    /**
     * Whether to upload the test reports to Checksum cloud 
     * allowing you to view them at https://app.checksum.ai/#/test-runs.
     * Note: Only relevant if Playwright reporter config is set to HTML
     * Note: Reports will be saved locally either way (according to Playwright configs)
     * and can be viewed using the CLI command show-reports.
     */
    hostReports: !!process.env.CI,
  }
}
```
## Playwright Configuration

A Playwright configuration file  `playwright.config.ts` should be available to the Playwright tests runner in order to provide project configuration.
For your convenience we extended the configuration to allow the addition of the `playwright-extra-plugins` packages.
Available plugins can be found at https://github.com/berstend/puppeteer-extra/tree/master/packages/playwright-extra

To add a plugin:

- Install it using your package manager (yarn, npm, pnpm)
- Import the plugin in your `playwright.config.ts` file
- In the `projects` definition under `use`, add it to the playwrightExtra array
- Make sure to initialize the plugin before or during the addition to the array

Example `playwright.config.ts` with the `puppeteer-extra-plugin-stealth` plugin:

```js
import { PuppeteerExtraPlugin } from "puppeteer-extra-plugin";
import StealthPlugin from "puppeteer-extra-plugin-stealth"; // <--- Added import line

export default defineConfig<{ playwrightExtra?: PuppeteerExtraPlugin[] }>({
  // ....
  projects: [
    {
      name: "chromium",
      testMatch: /checksum.spec/,
      use: {
        ...devices["Desktop Chrome"],
        playwrightExtra: [StealthPlugin()], // <--- Initialized and added to the playwrightExtra array
      },
    },
  ],
});
```

**See more detailed instructions inside the `checksum-root/playwright.config.ts` file**

## Checksum Helpers API
Helpers are deconstructed from the result of the initial call to the imported @checksum-ai/runtime `init` method, as following:
```js
import { test as base } from "@playwright/test";
import { init, IChecksumPage } from "@checksum-ai/runtime";
const { test, expect, defineChecksumTest, login, checksumAI } = init(base);
```

```js
function init(base: ChecksumTestType<PlaywrightTestArgs>): {
   /**
    * The Playwright test instance enhanced by Checksum
    */ 
  test: ChecksumTestType<ChecksumPlaywrightTestArgs>;
  /**
   * The login method which calls the user defined login function
   */
  login: (
    page: Page,
    { role, environment }?: { role?: string; environment?: string }
  ) => Promise<void>;
  /**
   * The test title definition along with the test id
   * used by Checksum to identify the test
   */
  defineChecksumTest: (title: string, testId: string) => string;
     /**
    * The Playwright expect instance enhanced by Checksum
    */ 
  expect: ChecksumExpect;
  /**
   * Wrapping each test action with this method allows
   * Checksum to recover from failure using our test recover AI agent
   */
  checksumAI: (description: string, testFunction: Function) => Promise<any>;
  /**
   * Return environment data as defined in the checksum configuration 
   * given the environment name and userRole, along with a login function
   * for this environment and user combination
   */
  getEnvironment: ({
    name,
    userRole,
  }: {
    name?: string;
    userRole?: string;
  }) => {
    environment: ChecksumConfigEnvironment;
    user: EnvironmentUser;
    login: (
      page: Page,
      { role }?: { role?: string }
    ) => Promise<void>;
  };
};
```
## Checksum Page API

```js
interface ChecksumPage extends Page {
   /**
    * Affiliates the test step with selection data calculated during test generation and used as part of the recovery mechanism in case Playwright fails to locate the element
    */
  checksumSelector: (id: string) => ChecksumPage;

  /**
   * Wraps a test step allowing ChecksumAI to recover from failures or even generate
   * the required Playwright code when no callback is provided.
   * Can be chained with `.withDialog` to automatically listen to browser dialogs
   * or with `.skipAutoRecovery` to prevent recovery attempts.
   */
  checksumAI: (thought: string, body?: () => any) => Promise<any> | ChecksumPage;

   /**
    * For tests requiring file uploads, this method resolves to the configured assets folder storing all relevant files
    */ 
  resolveAssetsFolder: (assets: string[]) => string[];

  /**
   * For multi-tab test flows, this method gets the ChecksumPage in order of appearance, when 0 is the initially opened tab.
   */
  getPage(index: number): Promise<ChecksumPage>;

  /**
   * Creates a compound selection by grouping multiple anchors and optionally
   * resolving a target element relative to their common parent.
   */
  compoundSelection: {
    (
      anchors: (base: Locator) => Array<Locator | string>,
      target?: (base: Locator) => Locator | string
    ): ChecksumLocator;
    (selection: {
      anchors: (base: Locator) => Array<Locator | string>;
      target?: (base: Locator) => Locator | string;
    }): ChecksumLocator;
  };

  /**
   * When required to login mid-test, use reauthenticate with the user's role. Note: It is not possible to change environments during a single test run.
   */
  reauthenticate: (role: string) => Promise<void>;

   /**
   * When required to handle actions that involve browser Native Dialogs that are part of the WebAPI,
   * use waitForDialog to listen for any dialog events, and handle them accordingly inside the handler.
   * Note: waitForDialog can only work within a checksumAI.withDialog call which initiates the dialog listener. 
   * 
   * await checksumAI.withDialog("delete item and confirm", () => {
   *  await page.locator('.delete').click();
   *  await page.waitForDialog().then((confirm) => {
   *      confirm.accept();
   *  });
   * });
   */
  waitForDialog: (timeout?: number) => Promise<Dialog>;
}

```

## ChecksumLocator API

ChecksumLocator extends the existing Playwright Locator, adding the following functionality:
* canvasClick
* compoundSelection

```js
interface ChecksumLocator extends Locator {
  /*
   * Click on certain text within the canvas element the locator chain points to.
   * When more than one element with the same text exists, uses the rectSizeIndex
   * to resolve the target element by its bounding box size, 0 being the largest.
   * Example:
   * await page.locator('canvas').canvasClick('graph-point-1', 1); // clicking on the 2nd largest
   */
  canvasClick: (canvasText: string, rectSizeIndex?: number) => Promise<void>;

  /**
   * Will create a compound selection that selects elements by grouping multiple locators as anchors
   * and finding the target elements, if specified, from their common root parent.
   * If no target is provided, the compound selection will return a locator to the common parents that were calculated from the anchors.
   *
   * **Usage example**
   *
   * ```js
   * await page.compoundSelection(
   *  (base) => [base.getByText("<selector to first anchor>""), page.locator("selector to second anchor"), "<text content of third anchor>"],
   *  (base) => base.locator("<relative selector to target element>")
   * ]).first().click();
   * ```
   *
   * @param anchors Method that returns array of locators and/or text context, to group and calculate the common parent from.
   *                The method receives the base locator as an argument, which is the relative locator or page that the compound selection is called on.
   *                The method should return an array of locators or strings that point at the anchor elements.
   * @param target [optional] Method that returns the relative locator or string content that will point at the target element from the common parent
   *               that was calculated from the anchors.
   *               If no target is provided, the compound selection will return a locator pointing at the common parents.
   * @returns Locator to the common parent(s) or the target element(s) if specified.
   */
  compoundSelection(
    anchors: (base: Locator) => Array<Locator | string>,
    target?: (base: Locator) => Locator | string
  ): ChecksumLocator;

  /**
   * Will create a compound selection that selects elements by grouping multiple locators as anchors
   * and finding the target elements, if specified, from their common root parent.
   * If no target is provided, the compound selection will return a locator to the common parents that were calculated from the anchors.
   *
   * **Usage example**
   *
   * ```js
   * await page.compoundSelection({
   *    anchors: (base) => [base.getByText("<selector to first anchor>""), page.locator("selector to second anchor"), "<text content of third anchor>"],
   *    target?: (base) => base.locator("<relative selector to target element>")
   * }).first().click();
   * ```
   * @param selection
   * @returns Locator to the common parent(s) or the target element(s) if specified.
   */
  compoundSelection(selection: {
    /**
     * Method that returns array of locators and/or text context, to group and calculate the common parent from.
     * The method receives the base locator as an argument, which is the relative locator or page that the compound selection is called on.
     * The method should return an array of locators or strings that point at the anchor elements.
     *
     * @param base Base locator that the compound selection is called on.
     */
    anchors: (base: Locator) => Array<Locator | string>;
    /**
     * Method that returns the relative locator or string content that will point at the target element from the common parent
     * that was calculated from the anchors.
     * If the target is null, the compound selection will return a locator pointing at the common parents.
     *
     * @param base Base locator that the compound selection is called on.
     */
    target?: (base: Locator) => Locator | string;
  }): ChecksumLocator;
}
```


## CLI Commands

1. `init` - Initialize the Checksum directory and configurations.
2. `test` - Run Checksum tests. Accepts all [Playwright command line flags](https://playwright.dev/docs/test-cli). To override `checksum.config.ts`, pass full or partial JSON as a string using `--cksm-config`. Use `--cksm-title="<message>"` to set a custom commit message for the run.
3. `show-report` - Locally shows the latest test run HTML report. Only applicable following completion of a test run configured to output an HTML report.
4. `dotenv` - Manage environment variables. Use `--download` to download the `.env` file for your project. An `--api-key` argument can be supplied if not present in `checksum.config.ts`.
    Examples:
    - `npx checksumai dotenv --download`
    - `npx checksumai dotenv --download --api-key=<apiKey>`

## Troubleshooting

**Q: I'm seeing various exceptions when I run "npx checksumai test", even before the test starts.**

A: If you had a pre-installed version of Playwright, it might not be compatible with Checksum. Remove both Playwright and Checksum libraries, delete the relevant folder from `node_modules`, and run `npm install -D checksumai`.
