# Feature Spec: Validation Script Hardening (AOP)

> **Purpose of this document**: Define implementation-ready improvements to the three existing CI validators (`validate:mcp-contracts`, `validate:docker-mcp`, `validate:architecture`) based on gaps identified through static analysis. Each improvement closes a concrete drift or correctness risk.

**Version:** 1.0  
**Date:** 2026-03-03  
**Status:** Draft  
**Roadmap Mapping:** Infrastructure / Quality

---

## 0. Standards and Dependencies

### 0.1 Required Standards

All implementation MUST preserve:

- All three validators continue to pass on a clean codebase with zero output on success
- CI command signatures are unchanged: `npm run validate:mcp-contracts`, `npm run validate:docker-mcp`, `npm run validate:architecture`
- Exit code 1 on any new failure; exit code 0 only when all checks pass
- Nx + Vitest coverage thresholds maintained (≥ 90% per file)

### 0.2 Upstream Inputs

Implementing agents MUST read:

- `agentic/orchestrator/tools/catalog.json` — source of truth for tool metadata
- `apps/control-plane/src/application/tools/tool-metadata.ts` — `MUTATING_TOOLS` runtime list
- `apps/control-plane/src/core/constants.ts` — `TOOLS` constants
- `apps/control-plane/scripts/validate-mcp-contracts.ts` — current MCP validator
- `apps/control-plane/scripts/validate-architecture-rules.mjs` — current architecture validator
- `apps/control-plane/scripts/validate-docker-mcp-contract.mjs` — current Docker validator
- `apps/control-plane/vitest.config.ts` — coverage exclusions

### 0.3 Feature Scope

This spec implements improvements in three areas:

- **V1** MCP contract cross-validation hardening
- **V2** Docker MCP contract completeness
- **V3** Architecture rule completeness and correctness

---

## 1. Objectives

### 1.1 Must-Have Outcomes

- The `mutating` flag in `catalog.json` must be verifiably consistent with `TOOL_BEHAVIOR_METADATA` in code — any skew detected at CI time, not at runtime.
- All tool input/output schema files referenced in the catalog must be syntactically valid JSON Schema (parseable and compilable by AJV).
- Schema files present on disk but unreferenced by any catalog entry are reported as orphans.
- The `supported_roles` field on every catalog tool is validated against the known role set.
- `application/adapters/` files and `application/multi-project-loader.ts` are subject to defined architecture rules rather than silently skipped.
- `core` is prohibited from importing `application-services`, `application-tools`, `mcp`, or `providers` layer files.
- The Docker Dockerfile Node version pin is validated against a minimum major version.
- The entrypoint passthrough (`exec "$@"`) is verified to be present.

### 1.2 Non-Goals

- No rewrite of validators from scratch — changes are additive to existing checks.
- No new npm packages beyond `ajv` (already a project dependency).
- No runtime enforcement changes in kernel or tool runtime (only CI validators change).
- No coverage gating on the validator scripts themselves (they are scripts, not source).

---

## 2. Architecture Decisions

### 2.1 Validators Remain Scripts

All three validators are plain `.ts` / `.mjs` scripts run via `tsx` or `node`. They do not import from `src/` source — they read files from disk. This remains the pattern.

### 2.2 Catalog Remains the Source of Truth

`catalog.json` is authoritative. The code-side `MUTATING_TOOLS` in `tool-metadata.ts` must match it. When there is a discrepancy, the validator fails and prints the tool name. The developer must reconcile `tool-metadata.ts` to match the catalog.

### 2.3 Architecture Layer Map is Canonical

The layer classification in `validate-architecture-rules.mjs` is the contract. Adding a new source directory without also adding it to the layer map must fail loudly, not silently skip.

---

## 3. Implementation: V1 — MCP Contract Hardening

### 3.1 Cross-Check `mutating` vs `TOOL_BEHAVIOR_METADATA`

**File:** `apps/control-plane/scripts/validate-mcp-contracts.ts`

**Check:** For every tool entry in `catalog.json` with `mutating: true`, verify that the tool's `handler_id` appears as a key in `TOOL_BEHAVIOR_METADATA` (exported from `tool-metadata.ts`) with `mutating: true`.

**Implementation approach:**

The validator is a TypeScript file run via `tsx`. It can import `TOOL_BEHAVIOR_METADATA` directly from the source using a relative path:

```typescript
import { TOOL_BEHAVIOR_METADATA } from '../src/application/tools/tool-metadata.js';
```

Then for each tool in the catalog:

```typescript
const mutatingInCatalog = tool.mutating === true;
const mutatingInCode = TOOL_BEHAVIOR_METADATA[tool.handler_id]?.mutating === true;
if (mutatingInCatalog !== mutatingInCode) {
  errors.push(
    `Tool "${tool.handler_id}" mutating flag mismatch: catalog=${mutatingInCatalog}, code=${mutatingInCode}`,
  );
}
```

**Failure message example:**

```
✗ Tool "cost.record" mutating flag mismatch: catalog=true, code=false
  Fix: add TOOLS.COST_RECORD to MUTATING_TOOLS in tool-metadata.ts
```

**Current known failure to fix first:** `cost.record` — `TOOLS.COST_RECORD` must be added to `MUTATING_TOOLS` in `tool-metadata.ts` before the check is added, otherwise CI will immediately fail. Fix the code, then add the check.

### 3.2 Validate Input/Output Schemas Are Valid JSON Schema

**File:** `apps/control-plane/scripts/validate-mcp-contracts.ts`

**Check:** For each `input_schema_ref` and `output_schema_ref` in the catalog, load the JSON file and compile it with AJV. A schema that fails to compile (bad `$schema`, invalid keyword, etc.) must be reported.

**Implementation approach:**

```typescript
import Ajv from 'ajv';
const ajv = new Ajv({ strict: false });

for (const tool of catalog.tools) {
  for (const schemaRef of [tool.input_schema_ref, tool.output_schema_ref]) {
    const schemaPath = path.resolve(toolsDir, schemaRef);
    const schema = JSON.parse(fs.readFileSync(schemaPath, 'utf8'));
    try {
      ajv.compile(schema);
    } catch (e) {
      errors.push(`Schema "${schemaRef}" failed AJV compilation: ${e.message}`);
    }
  }
}
```

`ajv` is already a dependency (`"ajv": "^8.x"` appears in package.json via existing usage in the kernel). No new dependency needed.

### 3.3 Detect Orphan Schema Files

**File:** `apps/control-plane/scripts/validate-mcp-contracts.ts`

**Check:** Collect all `.json` files under `agentic/orchestrator/tools/schemas/input/` and `schemas/output/`. Compare against the set of schema refs mentioned in the catalog. Files on disk that are not referenced by any tool are reported as orphans.

**Known current orphans (informational, not blocking — these may be shared base schemas):**

- `schemas/input/mutating.schema.json`
- `schemas/input/read.schema.json`
- `schemas/output/standard_success.schema.json`

These three files should be either:

1. Added as `$ref` base schemas used by at least one tool, or
2. Explicitly listed in a `known_shared_schemas` array in the validator (not subject to orphan check).

**Implementation approach:**

```typescript
const referencedSchemas = new Set<string>();
for (const tool of catalog.tools) {
  referencedSchemas.add(tool.input_schema_ref);
  referencedSchemas.add(tool.output_schema_ref);
}

const allSchemaFiles = [
  ...glob.sync('schemas/input/*.json', { cwd: toolsDir }),
  ...glob.sync('schemas/output/*.json', { cwd: toolsDir }),
];

const knownShared = new Set([
  'schemas/input/mutating.schema.json',
  'schemas/input/read.schema.json',
  'schemas/output/standard_success.schema.json',
]);

for (const file of allSchemaFiles) {
  if (!referencedSchemas.has(file) && !knownShared.has(file)) {
    warnings.push(`Orphan schema file not referenced by any catalog tool: ${file}`);
  }
}
```

Orphans emit **warnings** (printed but do not fail CI) unless the file is outside the `knownShared` exemption list.

### 3.4 Validate `supported_roles` Against Known Role Set

**File:** `apps/control-plane/scripts/validate-mcp-contracts.ts`

**Check:** Every value in every tool's `supported_roles` array must be one of: `orchestrator`, `planner`, `builder`, `qa`, `system`.

```typescript
const KNOWN_ROLES = new Set(['orchestrator', 'planner', 'builder', 'qa', 'system']);

for (const tool of catalog.tools) {
  for (const role of tool.supported_roles) {
    if (!KNOWN_ROLES.has(role)) {
      errors.push(`Tool "${tool.name}" has unknown role "${role}" in supported_roles`);
    }
  }
}
```

---

## 4. Implementation: V2 — Docker MCP Contract Completeness

### 4.1 Validate Node Major Version Pin

**File:** `apps/control-plane/scripts/validate-docker-mcp-contract.mjs`

**Check:** The `FROM` line in the Dockerfile must pin to a specific Node major version (`node:22` or higher). `node:latest`, `node:lts`, or any unversioned form must fail.

```javascript
const MIN_NODE_MAJOR = 22;

const fromMatch = dockerfile.match(/^FROM\s+node:(\d+)/m);
assert(fromMatch, 'Dockerfile must use a pinned Node version (e.g. node:22-bookworm-slim)');
const nodeMajor = parseInt(fromMatch[1], 10);
assert(
  nodeMajor >= MIN_NODE_MAJOR,
  `Dockerfile Node version ${nodeMajor} is below minimum required ${MIN_NODE_MAJOR}`,
);
```

### 4.2 Verify Entrypoint `exec "$@"` Passthrough

**File:** `apps/control-plane/scripts/validate-docker-mcp-contract.mjs`

**Check:** The entrypoint shell script (referenced in `ENTRYPOINT` of the Dockerfile) must contain `exec "$@"` as the final command, ensuring the container actually runs the requested process after startup checks.

```javascript
const entrypointPath = path.resolve(repoRoot, 'docker', 'entrypoint.sh');
const entrypoint = fs.readFileSync(entrypointPath, 'utf8');
assert(/\bexec\s+"\$@"/.test(entrypoint), 'Entrypoint script must contain exec "$@" passthrough');
```

---

## 5. Implementation: V3 — Architecture Rule Completeness

### 5.1 Classify `application/adapters/` and `application/multi-project-loader.ts`

**File:** `apps/control-plane/scripts/validate-architecture-rules.mjs`

**Problem:** Files in `application/` that are not under `application/services/` or `application/tools/` are currently classified as `'unknown'` and skipped. These files can import from any layer without detection.

**Fix:** Add an `'application'` layer classification and rules. Files that match `application/` but not the more specific `application/services/` or `application/tools/` paths resolve to `'application'`.

**Layer classification update:**

```javascript
function classifyFile(relativePath) {
  if (relativePath.startsWith('application/services/')) return 'application-services';
  if (relativePath.startsWith('application/tools/')) return 'application-tools';
  if (relativePath.startsWith('application/')) return 'application'; // NEW
  if (relativePath.startsWith('cli/')) return 'cli';
  if (relativePath.startsWith('core/')) return 'core';
  if (relativePath.startsWith('interfaces/')) return 'interfaces';
  if (relativePath.startsWith('mcp/')) return 'mcp';
  if (relativePath.startsWith('providers/')) return 'providers';
  if (relativePath.startsWith('supervisor/')) return 'supervisor';
  return 'unknown';
}
```

**Architecture rules addition:**

```javascript
const ARCHITECTURE_RULES = {
  // existing rules ...
  application: ['cli', 'interfaces', 'supervisor'], // NEW: same restrictions as application-services
};
```

**Unknown layer handling:** Files that remain `'unknown'` after classification must fail the validator with a message, rather than being silently skipped. This ensures new directories added to `src/` cannot evade architecture rules:

```javascript
if (layer === 'unknown') {
  errors.push(`Unclassified source file "${relativePath}" — add it to the layer map in validate-architecture-rules.mjs`);
  continue;
}
```

The top-level `index.ts` (public barrel export) should be explicitly exempt:

```javascript
const EXEMPT_FILES = new Set(['index.ts']);
```

### 5.2 Prohibit `core` from Importing Upward Layers

**File:** `apps/control-plane/scripts/validate-architecture-rules.mjs`

**Problem:** The current `core` rule forbids `['cli', 'interfaces', 'supervisor']` but does not forbid `application-services`, `application-tools`, `mcp`, or `providers`. `core` should be a pure domain layer — no upward dependencies.

**Fix:** Expand the `core` forbidden list:

```javascript
'core': ['cli', 'interfaces', 'supervisor', 'application-services', 'application-tools', 'application', 'mcp', 'providers'],
```

Before adding this rule, verify that no existing `core/` file currently imports from those layers. If any do, they must be refactored to eliminate the import first.

**Verification command:**

```bash
grep -r "from.*application\|from.*mcp\|from.*providers" apps/control-plane/src/core/ --include="*.ts"
```

---

## 6. Acceptance Criteria

### V1 — MCP Contract Hardening

- [ ] `npm run validate:mcp-contracts` fails if any catalog tool with `mutating: true` is not in `TOOL_BEHAVIOR_METADATA`
- [ ] `npm run validate:mcp-contracts` fails if any input or output schema fails AJV compilation
- [ ] `npm run validate:mcp-contracts` prints warnings for orphan schema files not in the `knownShared` list
- [ ] `npm run validate:mcp-contracts` fails if any `supported_roles` value is not in the known role set
- [ ] `cost.record` tool has `TOOLS.COST_RECORD` added to `MUTATING_TOOLS` in `tool-metadata.ts`
- [ ] All existing passing behaviors of `validate:mcp-contracts` continue to pass

### V2 — Docker MCP Contract

- [ ] `npm run validate:docker-mcp` fails if the Dockerfile `FROM` does not pin to `node:22` or higher
- [ ] `npm run validate:docker-mcp` fails if the entrypoint script does not contain `exec "$@"`
- [ ] All existing passing behaviors of `validate:docker-mcp` continue to pass

### V3 — Architecture Rules

- [ ] `npm run validate:architecture` fails for any file in `application/adapters/` that imports from `cli`, `interfaces`, or `supervisor`
- [ ] `npm run validate:architecture` fails for any file in `core/` that imports from `application-services`, `application-tools`, `application`, `mcp`, or `providers`
- [ ] `npm run validate:architecture` fails for any unclassified source file (not in the exempt list)
- [ ] `index.ts` is explicitly exempted from architecture classification
- [ ] All existing passing behaviors of `validate:architecture` continue to pass

---

## 7. Implementation Order

Implement in this order to avoid false CI failures:

1. **Fix `cost.record` in `tool-metadata.ts`** — adds `TOOLS.COST_RECORD` to `MUTATING_TOOLS` (code fix, not validator fix)
2. **Verify `core/` has no upward imports** — grep check before changing rules
3. **V3: Classify `application/` layer and `unknown` fail-loudly** — low risk, additive classification
4. **V3: Expand `core` forbidden list** — only after step 2 confirms no violations exist
5. **V1: Add `mutating` cross-check** — only after step 1 ensures it passes immediately
6. **V1: Add AJV schema compilation check** — compile all schemas, fix any that fail
7. **V1: Add orphan schema detection** — populate `knownShared` with the 3 known orphans
8. **V1: Add `supported_roles` validation** — straightforward set membership check
9. **V2: Add Node version pin check** — regex against Dockerfile `FROM` line
10. **V2: Add `exec "$@"` check** — regex against entrypoint script

---

## 8. Files Affected

| File                                                          | Change Type                                   |
| ------------------------------------------------------------- | --------------------------------------------- |
| `apps/control-plane/scripts/validate-mcp-contracts.ts`        | Add checks 3.1–3.4                            |
| `apps/control-plane/scripts/validate-architecture-rules.mjs`  | Add checks 5.1–5.2                            |
| `apps/control-plane/scripts/validate-docker-mcp-contract.mjs` | Add checks 4.1–4.2                            |
| `apps/control-plane/src/application/tools/tool-metadata.ts`   | Add `TOOLS.COST_RECORD` to `MUTATING_TOOLS`   |
| `apps/control-plane/test/tool-metadata.spec.ts`               | Add tests for `COST_RECORD` mutating behavior |
| `spec-files/progress.md`                                      | Update with completed milestone               |
