# @elench/testkit

`@elench/testkit` discovers `*.testkit.ts` files, infers suite ownership from the
filesystem, starts local services, provisions Docker-managed local Postgres
databases, and runs HTTP, DAL, and Playwright suites.

The package is now driven by `testkit.setup.ts`, not `testkit.config.json`.

## Usage

```bash
cd my-product

# Run every testkit-managed suite
npx @elench/testkit

# Filter by type
npx @elench/testkit --type int
npx @elench/testkit --type dal
npx @elench/testkit --type e2e
npx @elench/testkit --type int,e2e,dal
npx @elench/testkit --type pw

# Parallel file execution
npx @elench/testkit --workers 8

# One file-level wall clock budget for every suite file
npx @elench/testkit --file-timeout-seconds 60

# Run a deterministic shard
npx @elench/testkit --shard 1/3

# Specific service / suite
npx @elench/testkit --service frontend --type pw -s navigation
npx @elench/testkit --service api --type int -s health
npx @elench/testkit --type int,e2e,dal -s dal:queries

# Exact file
npx @elench/testkit --type int --file __testkit__/health/health.int.testkit.ts

# Temporarily ignore repo-declared skip rules
npx @elench/testkit --ignore-skip-rules --file __testkit__/billing/billing.int.testkit.ts

# Deterministic git-trackable status snapshot
npx @elench/testkit --type int --write-status

# Lifecycle
npx @elench/testkit status
npx @elench/testkit destroy

# Known-failures tooling
npx @elench/testkit known-failures validate --issue-mode error
npx @elench/testkit known-failures render --output KNOWN_FAILURES.md

# Capture a template DB schema snapshot
npx @elench/testkit db snapshot capture --service api --output scripts/testkit/schema-baseline.sql
```

## Setup

Create `testkit.setup.ts` at repo root:

```ts
import {
  commandStep,
  defineTestkitSetup,
  localDatabase,
  moduleStep,
  nextService,
  service,
  sqlFileStep,
  tsxService,
} from "@elench/testkit/setup";

export default defineTestkitSetup({
  execution: {
    workers: 8,
    fileTimeoutSeconds: 60,
  },
  reporting: {
    knownFailuresFile: "testkit.known-failures.json",
    issueValidation: {
      provider: "github",
      mode: "warn",
      cacheTtlSeconds: 900,
    },
  },
  services: {
    api: service({
      ...tsxService({
        cwd: ".",
        entry: "src/index.ts",
        port: 3004,
        readyPath: "/health",
      }),
      envFiles: [".env.testkit"],
      database: localDatabase({
        template: {
          inputs: ["db/schema.sql", "scripts/seed.ts"],
          migrate: [sqlFileStep("db/schema.sql")],
          seed: [commandStep("npm run db:seed")],
          verify: [moduleStep("src/testkit/verify-seed.ts#verifySeed")],
        },
      }),
      runtime: {
        instances: 1,
        maxConcurrentTasks: 4,
      },
      requirements: {
        files: [
          {
            path: "src/api/routes/__testkit__/crawls-worker-concurrency.int.testkit.ts",
            locks: ["global-worker-loop"],
          },
        ],
      },
    }),
    frontend: service({
      ...nextService({
        cwd: "frontend",
        port: 3000,
        env: {
          NEXT_PUBLIC_API_URL: "{baseUrl:api}",
        },
      }),
      dependsOn: ["api"],
      envFiles: ["frontend/.env.testkit"],
      runtime: {
        instances: 1,
      },
    }),
    billing: service({
      skip: {
        files: [
          {
            path: "__testkit__/invoices/invoices.int.testkit.ts",
            reason: "Billing is still stubbed locally",
          },
        ],
        suites: [
          {
            selector: "pw:lifecycle",
            reason: "End-to-end billing lifecycle is not implemented yet",
          },
        ],
      },
    }),
  },
});
```

`testkit.setup.ts` is optional for simple repos, but it is the primary escape hatch
for:

- worker count and per-file runtime budget
- per-file wall clock timeout budget
- multi-service graphs
- local runtime instance counts
- per-runtime concurrent task caps
- local DB binding configuration
- template database migrate / seed / verify stages
- template schema snapshot capture
- explicit per-file or per-suite locks
- named HTTP suite profiles
- known-failure annotation merge for enriched status/run artifacts
- optional GitHub-backed known-failure issue validation
- repo-declared suite/file skip policies with explicit reasons
- telemetry upload configuration

If `reporting.knownFailuresFile` is configured, `testkit` enriches
`.testkit/results/latest.json` and `testkit.status.json` with:

- per-file `failureDetails`
- per-file `triage` metadata (issue, classification, description)
- top-level `triageSummary` counts for known vs untriaged failures

Known-failure entry authoring uses this contract:

- `title`
  - exact issue-tracker title for the linked issue
  - shared issues may reuse the same title across multiple local entries
- `description`
  - product-local bug slice or route-family summary
  - this is where file-specific nuance belongs
- `whyFailing`
  - underlying technical cause of the failure

If `reporting.issueValidation` is also configured, `testkit` validates known-failure
issue references against GitHub and adds top-level `knownFailuresIssueValidation`
data to the run/status artifacts. The most important stale-triage signal is:

- a known-failure test still fails, but the linked GitHub issue is closed

In `mode: "error"`, exact GitHub metadata drift is also treated as a validation
failure:

- title does not match the linked issue title
- local open/closed state does not match the linked issue state

Reproduction warnings are execution-aware:

- `failed` means the known failure reproduced
- `passed` means the matched test executed and did not reproduce
- `skipped` and `not_run` do not count as reproduction evidence

Known-failure operations are also available as first-class CLI commands:

- `testkit known-failures validate`
- `testkit known-failures render`

## Authoring

HTTP suites:

```ts
import { defineHttpSuite } from "@elench/testkit";

const suite = defineHttpSuite(({ rawReq }) => {
  rawReq("GET", "/health");
});

export default suite;
```

`testkit` suite files should default-export the suite object returned by
`defineHttpSuite(...)` or `defineDalSuite(...)`.

Named HTTP profiles live in `testkit.setup.ts` and can be referenced by name:

```ts
import { defineHttpSuite } from "@elench/testkit";

const suite = defineHttpSuite({ profile: "default-auth" }, ({ req, setupData }) => {
  req("GET", "/api/auth/session", setupData);
});
```

DAL suites:

```ts
import { defineDalSuite } from "@elench/testkit";

const suite = defineDalSuite(({ db }) => {
  db.query("select 1");
});

export default suite;
```

Low-level runtime primitives remain available:

```ts
import { check, group, http, waitFor } from "@elench/testkit/runtime";
```

`waitFor()` consumes the file budget configured by `execution.fileTimeoutSeconds`.
Consumers should not set local timeout values in test files.

```ts
import { waitFor } from "@elench/testkit/runtime";

const response = waitFor(
  () => req("GET", "/api/v1/jobs/123", setupData),
  (res) => JSON.parse(res.body).data?.status === "completed",
  { description: "job 123 to complete" }
);
```

## Discovery

`testkit` discovers suites from `__testkit__/` directories.

Example layouts:

- `src/api/routes/__testkit__/auth/me.int.testkit.ts`
- `src/db/__testkit__/sessions/count-type.dal.testkit.ts`
- `frontend/__testkit__/navigation/navigation.pw.testkit.ts`
- `src/internal/handler/__testkit__/repos/crud.int.testkit.ts`

`testkit` uses these suffixes automatically:

- `*.int.testkit.ts`
- `*.e2e.testkit.ts`
- `*.dal.testkit.ts`
- `*.load.testkit.ts`
- `*.pw.testkit.ts`

Ownership is inferred from:

- the deepest matching service root from `services.<name>.local.cwd`
- optional `services.<name>.discovery.roots` overrides for shared-root edge cases

Suite names are inferred from the colocated path:

- `auth/__testkit__/*.int.testkit.ts` => `auth`
- `routes/__testkit__/auth/*.int.testkit.ts` => `auth`

## Local Databases

`@elench/testkit` provisions Docker-managed local Postgres automatically for
services that define `database: localDatabase(...)`.

- template databases are cached
- runtime databases are cloned from templates when binding is `per-runtime`
- shared databases are reused when binding is `shared`
- template fingerprints are derived automatically from env files, migrate/seed
  config, and repo contents

Manual `template.inputs` overrides are still available for edge cases.

## Development Tests

```bash
npm test
npm run test:unit
npm run test:integration
npm run test:system
```
