# Portkey Gateway Provider UX

Date: 2026-05-17
Updated: 2026-05-20
Status: revised and implemented

## Context

Portkey is a gateway in front of model APIs. Its OpenAI-compatible integration points clients at `https://api.portkey.ai/v1` and sends Portkey headers such as `x-portkey-api-key`, optional virtual-key/config headers, and provider-qualified model ids such as `@openai-prod/gpt-4o`. Its Claude Code and Codex integrations keep the same idea: the user configures a Portkey transport, then Portkey routes to an upstream provider.

Wall-E already has two separate ideas that should not be collapsed:

- Provider runtime: the adapter shape Wall-E uses to format requests and parse responses, such as `openai`, `anthropic`, `google`, `deepseek`, or local providers.
- Auth/runtime shortcuts: `claude_cli`, `codex_cli`, and local proxy paths. These are not HTTP gateways and should stay visually distinct.

Portkey belongs on a third axis: **connection / transport**. A provider instance can be "OpenAI via Portkey" or "Anthropic via Portkey" while still using the OpenAI or Anthropic runtime adapter.

## Research Notes

Portkey's current public docs support the implementation shape:

- The OpenAI-compatible setup is still "change base URL to `https://api.portkey.ai/v1` and use a Portkey API key".
- The universal API supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages against the gateway, so Portkey should be modeled as transport rather than as a standalone upstream provider.
- The model listing endpoint is `GET /v1/models`, requires `x-portkey-api-key`, and can filter by `provider`/`ai_service`. Its response includes provider-qualified ids such as `@ai-provider-slug/gpt-5`, which CTM must preserve instead of filtering through the built-in catalog.
- Portkey Models is also a public model/pricing catalog. Provider pricing JSON is available at `https://configs.portkey.ai/pricing/{provider}.json`, individual model pricing is available at `https://api.portkey.ai/model-configs/pricing/{provider}/{model}`, and the open catalog repository publishes the current provider list. This is provider-agnostic and must not be limited to provider families CTM already has direct keys for.

## Revised UX Model

The `#models` page should use this vocabulary:

- **Provider**: upstream API/runtime family. Examples: OpenAI, Anthropic, Google Gemini. Provider cards are the primary overview because CTM has few provider families.
- **Connection**: how requests reach the provider. Examples: Direct API key, Portkey gateway, Claude CLI subscription, Codex CLI subscription, OAuth proxy.
- **Route**: a concrete model plus provider instance. Examples: `gpt-5.5 · OpenAI · Direct`, `gpt-5.5 · OpenAI · Portkey prod`.
- **Gateway**: shared transport/account configuration that can be enabled per provider family. Portkey is the first gateway.
- **Portkey Catalog Provider**: a provider family from Portkey's public model catalog, even when CTM has no direct adapter/key for that provider.
- **Model Catalog**: the long model inventory underneath provider cards. It is collapsed by default because the provider view is the decision surface.

This keeps the user from seeing Portkey as a competing "model provider" next to OpenAI and Anthropic. It also explains why a Claude model can be reachable through Portkey while Claude CLI remains a separate subscription-backed route.

## 2026-05-20 Product Decision

Portkey is enabled and disabled at the **provider** level, not the individual model level.

The provider card owns the access policy:

- `Auto`: prefer direct provider API when configured, otherwise use Portkey if the provider is gateway-only.
- `Prefer Direct`: use direct provider API when available; fall back to Portkey only when direct access is absent.
- `Prefer Portkey`: use the Portkey connection for that provider when available; fall back to direct only when no Portkey route exists.

The Gateway section owns account-level operations:

- connect/edit the Portkey gateway credential
- sync the Portkey model catalog
- apply Portkey to all providers verified by model sync
- apply Portkey to all providers imported from Portkey's catalog
- disable Portkey as the default by returning provider policies to `Auto`
- remove all Portkey gateway connections

The Model Catalog remains an inspection surface. It shows imported/discovered models, capabilities, pricing, source, and verification status, but the default access decision belongs on provider cards.

## `#models` Page Behavior

### Provider Cards

Provider cards remain grouped by upstream provider:

- OpenAI
  - Direct API key
  - Codex CLI subscription
  - Portkey gateway
- Anthropic
  - Direct API key
  - Claude CLI subscription
  - OAuth proxy
  - Portkey gateway
- Google Gemini
  - Direct API key
  - Portkey gateway when supported by the configured route
- Fireworks AI, Together AI, Mistral AI, and other Portkey-only providers
  - Portkey gateway only unless CTM later adds a native direct adapter

Each saved provider instance should display a compact connection chip:

- `Direct`
- `Portkey`
- `Claude CLI`
- `Codex CLI`
- `OAuth proxy`

The chip is metadata, not the provider type.

Each cloud provider card also exposes an **Access Route** control:

- `Use Portkey` when a Portkey route exists for that provider.
- `Prefer Direct` for direct-first routing.
- `Auto` for direct-first with gateway fallback.
- `Enable Portkey` when the provider has no Portkey route yet.

For a provider that exists only through Portkey's catalog, the card must not show a direct route control. It should render as Portkey-only. Unknown Portkey catalog provider families use the OpenAI-compatible runtime against the Portkey gateway and provider-qualified model ids such as `@fireworks-ai/mixtral-8x7b`.

This gives users a discoverable per-provider control without forcing route choices across hundreds of model rows.

### Gateway Section

The Models page includes a first-class **Gateways** section above provider cards.

The Portkey gateway card displays:

- imported Portkey provider count
- imported model count
- last successful sync or last sync error
- `Connect Portkey` / `Edit Gateway`
- `Sync now`
- `Apply to Portkey providers`
- `Disable as default`
- `Remove`

Portkey model discovery runs hourly in the Wall-E daemon, runs on startup after the daemon is listening, and can be triggered manually from `Sync now`. Sync has two data sources:

- The connected gateway `/models` response. This proves what the configured gateway route returns for the user's credential/header set.
- The public Portkey Models catalog. This is used to import provider families and model/pricing rows that CTM does not know directly, such as Fireworks AI.

Sync must not restrict Portkey provider imports to CTM's built-in provider list. It should fetch Portkey's current public catalog provider list, fall back to the documented provider list when the catalog index is unavailable, and import raw/provider-qualified Portkey model ids, including providers and model ids not present in CTM's built-in supported-model catalog.

Corporate/devbox Portkey configurations can point at a company gateway host instead of public `api.portkey.ai`. CTM should preserve the gateway URL from local configuration/logs and should not assume that a valid Portkey header set works against the public Portkey host. CTM must not hardcode any company's gateway URL as a fallback.

### Add Provider And Gateway Dialogs

The normal Add Provider dialog asks for upstream first, then connection:

1. Provider: OpenAI, Anthropic, Google, DeepSeek, Ollama.
2. Connection: Direct API key or Portkey gateway.
3. For Direct: show the provider API key and optional base URL.

The Gateway section uses a different Portkey-specific dialog. It must not ask users to choose an upstream provider or provider slug. Portkey is connected once with:

- Portkey API key
- Gateway URL, defaulting to `https://api.portkey.ai/v1` for manual Portkey accounts, or the detected devbox/corporate gateway URL when CTM can read a local devbox Claude configuration
- optional config id
- optional virtual key
- optional local label for the Portkey account/config

After saving this single gateway config, CTM stores one gateway credential row. The next Portkey sync creates internal runtime routes for providers discovered from the configured gateway and from Portkey's public model catalog. Provider cards then control whether each discovered provider uses direct API access, Portkey, or automatic fallback. Providers with no direct CTM connection render as Portkey-only.

If CTM detects a devbox Claude Portkey header file, the UI may offer a one-click "Use" action. That action stays server-side: the browser receives only masked metadata and booleans such as whether a virtual key exists; raw custom headers and API keys must never be returned by `/api/models/providers` or `/api/models/detect-keys`.

Default labels should make the route obvious:

- `OpenAI via Portkey`
- `Anthropic via Portkey`
- `OpenAI direct`

Advanced custom headers can still exist, but common Portkey fields should not require users to write JSON.

### Model Provider Section

The Model Providers section is collapsed by default. The page still shows the Gateways section first because gateway setup is the shared routing control. Searching provider cards expands the Model Providers section so matches are visible.

### Model List

The Model Catalog section is collapsed by default. Searching models expands the catalog so search results remain visible.

The model list should show duplicate model ids as separate routes only when necessary:

- If a model exists once, show `GPT-5.5` with a subtle `OpenAI · Direct` chip.
- If the same model exists multiple times, show separate rows:
  - `GPT-5.5` `OpenAI · Direct`
  - `GPT-5.5` `OpenAI · Portkey prod`

The saved value should prefer `model_registry.id`, not only the bare model id.

Gateway-discovered model rows use:

- `source = portkey`
- `gateway_type = portkey`
- `verification_status = verified` only when CTM's built-in catalog recognizes the model id; otherwise `unverified`
- `last_seen_at` from the latest Portkey sync

Public Portkey catalog model rows use:

- `source = portkey_catalog`
- `gateway_type = portkey`
- `verification_status = catalog`
- provider-qualified model ids such as `@fireworks-ai/mixtral-8x7b`
- pricing converted from Portkey's cents-per-token values into CTM's dollars-per-1M-token fields

Catalog pruning must not delete gateway-discovered or Portkey-catalog rows just because CTM's built-in catalog does not know the id yet.

## Conflict Rules

1. Exact `model_registry.id` wins.
2. Explicit task/model defaults win over provider route policy.
3. Provider route policy resolves duplicate direct and Portkey rows for the same provider family.
4. If a bare model id maps to exactly one enabled route, it is safe to resolve.
5. If a bare model id maps to multiple enabled routes and provider route policy cannot disambiguate, return an ambiguity error that names the available routes.
6. Never select the newest provider row as a silent tie-breaker when direct and Portkey routes share the same model id.

## Implementation Phases

### Phase 1: Documentation

Document the UX and routing model, including the `#models` page behavior and duplicate-model conflict rules.

### Phase 2: Runtime Support

- Add OpenAI custom-header support so OpenAI-through-Portkey can send Portkey headers.
- Add shared Portkey metadata helpers for naming, default base URLs, and route labels.
- Add public Portkey Models catalog sync independent of CTM's existing provider list.
- Allow unknown Portkey-only provider families to use the OpenAI-compatible runtime against the Portkey gateway.
- Extend provider detection so `PORTKEY_API_KEY` is treated as a gateway credential, not as Anthropic-only auto-detection.
- Add tests for OpenAI Portkey headers and duplicate route resolution.

### Phase 3: `#models` UX

- Add a connection selector to the Add Provider dialog.
- Replace raw custom-header JSON for common Portkey setup with focused fields.
- Render connection chips on provider/model rows.
- Keep CLI subscription auth clearly separated from Portkey gateway routes.

### Phase 4: Verification

- Unit-test provider adapters and model routing.
- Browser-test `http://localhost:3456/#models` or a dev CTM instance for the Add Provider dialog, direct provider setup, and Portkey setup.
- Use local HTTP traffic for test connection requests with fake keys against a local mock gateway so request headers can be inspected without sending secrets.

## Verification Results

The implemented provider-level gateway path is covered by:

- Portkey sync tests that import unknown gateway model ids, import Portkey-only catalog provider families such as Fireworks AI, mark CTM-known ids as `verified`, mark unknown gateway ids as `unverified`, and prove catalog pruning preserves gateway/catalog rows.
- Provider route-policy tests that prefer Portkey or direct connections per provider and fall back when the requested route is unavailable.
- Chat routing tests that resolve duplicate direct/Portkey bare model ids using the provider policy instead of raising avoidable ambiguity.
- Model admin HTTP tests for gateway summaries and provider route policy updates.
- Static UI tests for the providerless Portkey Gateway dialog, provider Access Route controls, collapsed-by-default Model Providers, and collapsed-by-default Model Catalog.
- Playwright render tests for the Models page gateway card, provider route buttons, Portkey-only provider cards, collapsed provider/catalog expansion, and provider policy POST behavior.
- Full CTM regression tests.

## References

- https://portkey.ai/docs/integrations/libraries/openai-compatible
- https://portkey.ai/docs/api-reference/inference-api/models/models
- https://portkey.ai/docs/product/ai-gateway/universal-api
- https://portkey.ai/docs/product/model-catalog/portkey-models
- https://portkey.ai/docs/integrations/libraries/claude-code
- https://portkey.ai/docs/integrations/libraries/codex
