# @elench/testkit

CLI that reads `runner.manifest.json` plus `testkit.config.json` from a product repo, starts the required local services, provisions local Postgres isolation, and runs manifest-defined suites across `k6` and Playwright.

## Prerequisites

```bash
sudo apt-get install -y jq
```

`@elench/testkit` ships its own `k6` binary and uses it for both HTTP and DAL suites by default.
If you need to force a different binary, set `TESTKIT_K6_BIN` to an absolute or relative path.

Database isolation uses Docker-managed local Postgres containers. `testkit` creates and reuses template databases automatically, then clones per-worker databases from those templates for fast reruns. The default image is `pgvector/pgvector:pg16`.

## Usage

```bash
cd my-product

# Run every testkit-managed suite
npx @elench/testkit

# Filter by type
npx @elench/testkit int
npx @elench/testkit dal
npx @elench/testkit e2e

# Filter by framework
npx @elench/testkit --framework playwright
npx @elench/testkit --framework k6

# Parallelize with isolated worker stacks
npx @elench/testkit --jobs 3

# Run a deterministic shard
npx @elench/testkit --shard 1/3
npx @elench/testkit --jobs 2 --shard 2/3

# Specific service / suite
npx @elench/testkit frontend e2e -s auth
npx @elench/testkit api int -s health

# Lifecycle
npx @elench/testkit status
npx @elench/testkit destroy
```

## K6 Authoring

Consumer k6 tests can import the shared authoring API directly from the package:

```js
import { defineHttpSuite } from "@elench/testkit";

const suite = defineHttpSuite(({ rawReq }) => {
  const res = rawReq("GET", "/health");
});

export const options = suite.options;
export const setup = suite.setup;
export default suite.exec;
```

For auth or schema-specific behavior, keep a small consumer-owned adapter next
to the tests and pass it into the generic suite factory:

```js
import { defineHttpSuite } from "@elench/testkit";
import { clerkSessionAuth } from "../helpers/testkit-auth.js";

const suite = defineHttpSuite({ auth: clerkSessionAuth }, ({ req, setupData }) => {
  req("GET", "/api/auth/me", setupData);
});
```

`testkit` bundles these imports before invoking k6, so tests do not need
generated `_testkit` files or direct package-manager path imports.

If you need to override the packaged `k6` binary for local environment reasons:

```bash
TESTKIT_K6_BIN=/path/to/k6 npx @elench/testkit int
```

Legacy compatibility:

- `testkit runtime install`
- `testkit runtime status`
- `testkit runtime update`

still exist, but direct package imports are now the preferred model.

From outside the product repo, use `--dir` explicitly:

```bash
npx @elench/testkit --dir my-product int
npx @elench/testkit --dir my-product api int -s health
```

## How it works

1. **Discovery** — reads `runner.manifest.json` for services, suites, files, and frameworks
2. **Config** — reads `testkit.config.json` for local runtime, migration, dependency, and database settings
   Per-service `.env` files declared in config are loaded when present.
3. **Database** — provisions Docker-managed local Postgres when a service declares one
4. **Seed** — runs optional product seed commands against the provisioned database
5. **Runtime** — starts required local services, waits for readiness, and injects test env
6. **Execution** — schedules file-level execution tasks across a global worker pool, reuses warm dependency graphs when possible, bundles `k6` files before execution so package imports resolve cleanly, and batches Playwright files per worker
7. **Cleanup** — stops local processes and preserves `.testkit/` state for inspection or later destroy

Each run ends with a compact summary that shows per-service pass/fail status, suite totals, failed suites, and total duration.

`testkit` also writes `.testkit/results/latest.json` for every run. When `testkit.config.json` includes a top-level `telemetry` block, it can best-effort upload that artifact to a configured HTTP endpoint with bearer auth.

## File roles

- `runner.manifest.json`: canonical test inventory
- `testkit.config.json`: local execution and provisioning config

`testkit.config.json` can also declare:

- `telemetry` for optional generic HTTP result upload
- `envFile` / `envFiles` for service-specific environment loading
- `databaseFrom` for dependency services that must share another service's `DATABASE_URL`
- `database.provider` for local Postgres settings
- `database.template.inputs` to define the local template cache invalidation inputs
- `migrate.backends` / `seed.backends` for optional local-only command overrides

## Parallel execution

`@elench/testkit` can run test work in parallel with `--jobs <n>`.

`--jobs` is global for the whole run, not per service.

Each worker gets its own:

- cloned local Postgres database
- `.testkit` state subtree
- local service ports

Workers prefer to stay on the same dependency graph so service stacks can be reused across assigned work, including dependent services such as `frontend -> api`.

Use `--shard <i/n>` to split the selected suite set deterministically before worker scheduling.

`testkit` also writes `.testkit/timings.json` and uses those file timings on later runs for longest-first balancing.

## Suite metadata

`runner.manifest.json` remains the source of truth for suites. Optional per-suite `testkit` metadata can tune execution:

```json
{
  "name": "health",
  "files": ["tests/example.js"],
  "testkit": {
    "maxFileConcurrency": 2,
    "weight": 3
  }
}
```

- `maxFileConcurrency`: k6-only opt-in for batching multiple files from the same suite onto one worker
- `weight`: optional fallback scheduling weight when no file timing history exists yet

## Schema

See [testkit-config-schema.md](testkit-config-schema.md).

## Development Tests

```bash
npm test
npm run test:unit
npm run test:integration
npm run test:system
```

`test:system` runs real end-to-end fixtures from `test/fixtures/system/`. It requires:

- Docker with a running daemon
- Playwright Chromium browser assets available to `@playwright/test`
