# AI Configuration Consolidation

Date: 2026-05-22
Status: implemented in the `models` worktree on 2026-05-23

## Problem

CTM currently exposes provider configuration in two places:

- Top navigation: `AI Providers` sits next to `WALL-E` and opens the richer model/provider page with gateways, provider cards, model counts, route policy, tests, scans, scorecards, defaults, and model catalog information.
- Settings / Wall-E Setup: `AI`, which shows the existing provider health/setup surface plus an `Open AI Providers` deep link.

That split is confusing because both surfaces use nearly the same words but do not mean exactly the same thing. Users cannot tell whether they are configuring:

- coding agents such as Claude, Codex/OpenAI, Google/Gemini, and Wall-E Coding Agent;
- model providers such as Anthropic, OpenAI, Google Gemini, DeepSeek, Ollama, Fireworks, and Portkey-discovered providers;
- gateways such as Portkey; or
- model routing/testing defaults used by CTM and Wall-E jobs.

The existing `AI Providers` page also contains important operational information. It should not be removed or reduced to a settings toggle page.

## Product Model

Use these terms consistently:

- **Coding Agents**: execution runtimes that own sessions. Examples: Claude Code, Codex/OpenAI, Gemini, Wall-E Coding Agent.
- **AI Providers**: model access backends used by CTM jobs, Wall-E jobs, and Wall-E Coding Agent. Examples: Anthropic, OpenAI, Google Gemini, Ollama, Fireworks via Portkey.
- **Gateways**: shared routing/discovery layers. Example: Portkey.
- **Models**: individual model routes exposed by AI Providers, including direct routes and gateway routes.

The user configures two separate but related things:

1. **Coding agents**: Which execution runtimes can run coding sessions, how they authenticate, and what default model/route they use.
2. **Model access**: Which AI providers, gateways, models, health tests, routing policies, and job defaults CTM/Wall-E can use.

## Revised IA Decision

Keep the current CTM navigation shape:

- Top navigation is `Sessions`, `Prompts`, `Wall-E`, `AI Providers`, `More`, and the settings gear.
- The `More` menu no longer owns `AI Providers`; provider configuration is a primary destination.
- The page title remains `AI Providers`.
- Settings remains a gear icon. It should not become a new top-level AI navigation tab.

`AI Providers` is the source of truth for provider access, model routing, model operations, and provider testing. Settings only summarizes this state and deep-links into the `AI Providers` page.

## User Questions Resolved

### Where CTM and Wall-E provider defaults are configured

Configure CTM and Wall-E model access on the `AI Providers` page:

- A compact top routing overview shows CTM/Wall-E defaults, coding-agent coverage, gateway state, and operations state without adding another full management card.
- Provider credentials, gateway routes, tests, rescans, and remove actions live in the provider cards below.
- Settings shows the same defaults as a read-only summary and links to this page.

### Where Routing, Shadow Training, and Scorecard go

These should stay on the `AI Providers` page, grouped under a `Model Operations` section below provider cards:

- `Routing`: one combined surface for task-specific route pins and tier fallback inspection.
  - `Task Overrides`: editable per-task Wall-E route pins, for example Chat, Think, Coding, Coding Review, Extract, and Embeddings. These win first.
  - `Tier Fallbacks`: read-only tier mapping/default fallback inspection. These apply only when a task has no explicit override.
- `Shadow Training`: training/evaluation collection and enablement.
- `Scorecard`: model/provider performance, benchmark, latency, cost, and quality results.

They should not move into Settings because they are operational model-management surfaces, not app setup preferences.

### Whether Settings summary/deep link is consistent

Settings should keep a normal settings-card layout. The implemented first step keeps the existing health/setup controls for compatibility, then adds a summary card and deep link at the top so users have one obvious canonical path into the `AI Providers` page. It should show:

- current CTM, Wall-E, and Wall-E Coding Agent defaults;
- provider health counts;
- last test time;
- gateway status;
- one primary `Open AI Providers` action.

This keeps Settings consistent as a status/setup overview while avoiding two competing provider editors.

### How users manually add providers

Manual provider add stays on `AI Providers` through the existing `+ Add Provider` action. The dialog should be clearer:

1. Choose provider: Anthropic, OpenAI, Google Gemini, DeepSeek, Moonshot, Ollama, or Custom OpenAI-compatible.
2. Choose connection: Direct API key, CLI/OAuth where supported, or Portkey gateway.
3. Enter credential/base URL/config fields.
4. Test connection.
5. Save.

For Portkey-discovered providers that CTM does not know natively, users can add them from the Portkey catalog or as a custom OpenAI-compatible route.

## Proposed AI Page Structure

The `AI Providers` page should avoid full-width summary panels above the work surface. The current structure is:

1. **Routing Overview**
   - One compact chip row.
   - Shows CTM/Wall-E defaults, coding-agent coverage, gateway state, and operations state.
   - Chips switch to the backing tab rather than duplicating controls.

2. **Section Tabs**
   - `Providers`
   - `Gateways`
   - `Model Operations`
   - `Model Catalog`
   - Providers is the default because it is the main source-of-truth work surface.
   - The Providers tab owns provider count, search, add-provider action, and routing overview chips.

3. **Gateways**
   - Portkey Gateway.
   - Connection state, sync state, imported providers, imported models.
   - Actions: Edit gateway, Sync now, Apply to Portkey providers, Disable as default, Remove.
   - Shows supported/imported providers discovered from Portkey without requiring those providers to already exist as CTM direct providers.

4. **Providers**
   - This preserves the current rich provider page.
   - Provider health, testing, rescans, credential/route state, model counts, capabilities, pricing ranges, access route policy.
   - Provider cards remain the primary management surface because there are not many providers.
   - Each card keeps per-connection Test/Re-scan/Remove.
   - Provider cards should show whether the provider can be used for:
     - CTM jobs
     - Wall-E jobs
     - Wall-E Coding Agent
   - The model catalog remains below as an inspection/detail area.

5. **Model Operations**
   - Routing, with editable Task Overrides persisted through the Wall-E task-default API plus read-only Tier Fallbacks in the same card.
   - Shadow Training.
   - Scorecard.
   - Training review / promotion information when present.

6. **Model Catalog**
   - Long model inventory.
   - Collapsed/searchable by default.
   - Includes pricing, capabilities, route source, and duplicate-route inspection.

The long **Model Catalog** should remain in the AI page, after provider cards, and stay collapsed or searchable by default. It is useful for model-level inspection and routing ambiguity, but provider cards are still the decision surface.

## Settings Page Change

Keep the Settings `AI` tab as a setup/status area with a deep link into AI Providers:

The summary should show:

- active CTM/Wall-E provider route;
- number of configured providers;
- number of failing providers;
- last provider test time;
- a direct link to `AI -> AI Providers`.

It should not contain another independent provider-card editor.

## Navigation Labels

Preferred labels:

- Top nav: `AI Providers`
- Page title: `AI Providers`
- Compact row: `CTM / Wall-E`, `Coding agents`, `Gateway`, `Operations`
- Section: `Gateways`
- Section: `Providers`
- Section: `Model Operations`
- Subsection: `Model Catalog`

Avoid:

- using `AI Provider` singular in Settings;
- using `Provider` for coding agents.

## Key UX Rules

1. If a user wants to configure session runners, they go to **Coding Agents**.
2. If a user wants to configure model access, credentials, provider tests, model counts, or routing, they go to **AI Providers**.
3. If a user wants to connect Portkey, they go to **Gateways**.
4. Settings can point to AI configuration, but it should not duplicate it.
5. Existing testing and model information stay visible in the AI Providers section.
6. Wall-E Coding Agent should be represented in Coding Agents but consume routes from AI Providers.
7. Provider test failures should appear in both Overview and the relevant provider card, with one canonical test action.

## Implementation Notes

- The production `AI Providers` page now renders a title followed by tabs for `Providers`, `Gateways`, `Model Operations`, and `Model Catalog`; the provider summary/search/actions live inside the Providers tab.
- `Providers` preserves the rich provider cards, model counts, Portkey route policy, Test/Re-scan/Remove actions, and manual `+ Add Provider` path.
- Providers renders exactly two ownership groups: `Manual + Gateway` and `Gateway Only`. A canonical provider appears once; if a user-added provider also has gateway routes, those routes fold into the `Manual + Gateway` card rather than creating a duplicate provider card.
- Settings keeps the current topbar/menu style, renames `AI Provider` to `AI`, and keeps a compact `AI Configuration` card with an `Open AI Providers` action.
- Manual add now includes `Custom OpenAI-compatible`, backed by a generic OpenAI-compatible runtime path instead of a ExampleCorp-specific provider hack.

## Availability And Loading Contract

Wall-E remains the owner for model providers, gateways, model registry, routing defaults, shadow config, and scorecards. CTM should not copy ownership of those writes into its own database.

CTM does own a small **last-good read model** for first paint and failure fallback:

- Endpoint: `GET /api/models/overview`
- Cache file: CTM config directory, `model-overview-cache.json`
- Contents: providers, models, gateways, provider health, routing defaults, shadow config, catalog status, and warnings.
- Exclusions: API keys, encrypted credentials, and write-only secrets.

The AI Providers page follows this load order:

1. Ask CTM for `/api/models/overview`.
2. If CTM has a cached overview, render it immediately with a "last known catalog" banner and refresh Wall-E in the background.
3. If CTM has no cache, fetch live providers from Wall-E with a short timeout and render the provider surface as soon as that returns.
4. Fetch model registry, gateways, health, tier defaults, task defaults, and shadow config asynchronously after first paint.
5. Defer expensive operations data, such as scorecards, promotion history, and training review, until the user opens `Model Operations`.

This avoids the previous failure mode where the whole AI Providers page was blocked by a transient `ECONNREFUSED 127.0.0.1:<wall-e-port>` while the Wall-E supervisor was still starting. A cold or restarting Wall-E process can still make live data stale, but it should not turn a previously usable provider page into an empty/error page.

The contract is intentionally asymmetric:

- **Reads** may use stale CTM cache with a visible freshness warning.
- **Writes** still go to Wall-E owner APIs and must fail loudly if Wall-E is unavailable.
- **Refreshes** are best-effort and de-duplicated so multiple browser tabs do not stampede Wall-E during startup.

## Wall-E Endpoint Responsiveness Contract

The AI Providers page depends on Wall-E for live provider/catalog data, so Wall-E HTTP handlers must not perform event-loop-blocking discovery work. The specific failure mode observed on 2026-05-23 was:

1. CTM proxied model/provider requests to Wall-E.
2. Wall-E was alive, but the Node process sample showed the main thread spending time in `SyncProcessRunner::Spawn`.
3. While that synchronous child process was running, unrelated Wall-E routes such as status/model reads could appear unresponsive from CTM.

The fix is structural:

- HTTP request handlers may call local databases synchronously when the query is bounded and already part of the route's data ownership.
- HTTP request handlers should not call `execFileSync`, shell startup files, credential CLIs, or other unbounded subprocess discovery.
- Provider environment discovery is now batched through one async shell lookup, cached for a short TTL, and de-duplicated while in flight.
- Explicit detection/setup actions may wait for async discovery, but the event loop remains available to serve status and cached read requests.

This keeps the useful behavior where CTM/Wall-E can discover API keys from a user's shell configuration, without letting a slow shell startup file or credential helper make Wall-E look dead.

## Open Questions

- Should `AI Providers` eventually become a full top-level mode with its own keyboard shortcut?
- Should the default landing section be `Overview` or `AI Providers` for returning users?
- Should Wall-E Setup keep any provider-specific form fields for first-run onboarding, or should first-run setup deep-link into the AI page immediately?
- Should provider cards expose job-default assignment inline, or should defaults live only in Overview?

## Discussion Recommendation

Start with the least disruptive consolidation:

1. Keep the current topbar/menu style.
2. Promote `AI Providers` to primary nav after `WALL-E`.
3. Keep the top of the AI Providers page compact; put provider overview/search/actions inside the Providers tab rather than above all tabs.
4. Keep Gateways, provider cards, Model Operations, and Model Catalog on that same page, but expose them as sibling tabs.
5. Change Settings `AI Provider` to `AI` and keep the existing setup/status shape with a link to `AI Providers`.
