# Avis Gateway

Smart model routing for Avis. The gateway automatically picks the best model from any configured provider based on what you're asking it to do.

## What It Does

Avis already supports 20+ providers (Anthropic, OpenAI, Google, OpenRouter, Vercel AI Gateway, HuggingFace, xAI, Mistral, etc.). The gateway adds intelligence on top:

1. **Task Classification** -- Heuristically classifies each message into categories: coding, reasoning, creative, factual, vision, conversation, tool-heavy, long-context
2. **Model Profiling** -- Maps each available model's strengths, speed, quality, and cost
3. **Smart Routing** -- Picks the best model for the task, considering category match, complexity, cost, and model health
4. **Health Tracking** -- Monitors latency and failures per model, avoids unhealthy models
5. **Fallback Chain** -- Provides alternative models if the primary choice fails

## Important: Avis Remains Avis

The gateway changes which model backs a response, but Avis's identity comes from the system prompt, memory, and personality -- not the model. You're always talking to Avis.

## How to Enable

The gateway is **disabled by default**. To enable:

```
# In settings.json (~/.avis/agent/settings.json)
{
  "gateway": {
    "enabled": true
  }
}
```

Or through the TUI settings.

## Configuration

```json
{
  "gateway": {
    "enabled": true,
    "qualityBias": 0.3,
    "maxCostPerMToken": 0,
    "excludeProviders": [],
    "pinnedModels": [],
    "useHealthData": true,
    "minConfidence": 0.4
  }
}
```

| Setting            | Default | Description                                            |
| ------------------ | ------- | ------------------------------------------------------ |
| `enabled`          | `false` | Whether automatic routing is active                    |
| `qualityBias`      | `0.3`   | Bias toward quality (-1 = fastest, 1 = best quality)   |
| `maxCostPerMToken` | `0`     | Max cost per million tokens (0 = no limit)             |
| `excludeProviders` | `[]`    | Providers to never route to                            |
| `pinnedModels`     | `[]`    | Models to prefer (get +50 score bonus)                 |
| `useHealthData`    | `true`  | Use runtime health data for decisions                  |
| `minConfidence`    | `0.4`   | Below this confidence, fall back to your default model |

## Task Categories

| Category       | Example Prompts                                                       |
| -------------- | --------------------------------------------------------------------- |
| `coding`       | "write a function", "fix this bug", "refactor the component"          |
| `reasoning`    | "analyze the trade-offs", "explain why this fails", "walk me through" |
| `creative`     | "draft an email", "brainstorm names", "write a story"                 |
| `factual`      | "what is X?", "one word answer", "capital of France"                  |
| `vision`       | Any prompt with attached images                                       |
| `conversation` | "hey", "where were we", "thanks"                                      |
| `tool-heavy`   | "find all files", "replace across codebase", "install and configure"  |
| `long-context` | "review the entire file", "go through the codebase"                   |

## Model Profiles

The gateway has built-in profiles for major model families:

- **Anthropic**: Opus 4.6 (highest quality), Sonnet 4.5 (balanced), Sonnet 4 (fast+good), Haiku 3.5 (fastest)
- **OpenAI**: GPT-5.x (highest quality), o4-mini (reasoning), GPT-4o (balanced), GPT-4o-mini (fast)
- **Google**: Gemini 3/2.5 Pro (quality), Gemini 2.5 Flash (fast)
- **xAI**: Grok 4 (balanced)
- **Mistral**: Devstral (fast coding)
- **Open Models**: Kimi K2, Qwen, DeepSeek, Llama (via OpenRouter/HuggingFace)

Unknown models get a conservative default profile (coding-capable, medium speed/quality).

## Architecture

```
User Input
    |
    v
[Task Classifier] -- heuristic, no LLM call
    |
    v
[Router] -- scores all profiled models
    |         considers: strengths, speed, quality, cost, health
    |
    v
[Best Model] --> Agent sends prompt to this model
    |
    v
[Health Tracker] -- records latency/success/failure
```

The gateway is in `packages/coding-agent/src/core/gateway/`:

- `types.ts` -- Type definitions and config
- `classifier.ts` -- Task classification heuristics
- `profiles.ts` -- Model capability profiles
- `router.ts` -- Scoring and routing engine
- `health.ts` -- Runtime health tracking
- `gateway.ts` -- Main gateway class
- `index.ts` -- Public API

## When Disabled

When the gateway is disabled (default), it's completely transparent. The agent uses whatever model you've selected, exactly as before. No overhead.
