# A3M Router Redesign: Reaching 0.80+

## Key Insight

RouterArena evaluates:
1. **Prediction** - which model we say will be used
2. **Generated Result** - the actual answer from that model
3. **Accuracy** - whether the answer is correct

The score formula heavily penalizes accuracy loss (β=0.1):
```
S = (1.1 × accuracy × C) / (0.1 × accuracy + C)
```

**Cost savings are only 10% of the formula weight.**

## Current State

| Metric | Baseline | Attempted | 
|--------|----------|-----------|
| Score | 0.6912 | 0.6964 |
| Accuracy | 69.29% | 69.13% |
| Cost/1K | $0.1438 | $0.0635 |

**Problem:** Aggressive cost routing (97% to premium) hurt accuracy by 0.16%, which offset all cost gains.

## Root Cause Analysis

The benchmark queries are **mostly simple factual MCQs** that:
- Deepseek handles well (85% of queries)
- Mistral helps for 15% (legal, medical, ethics)
- Premium models (gemini) don't help much

## New Strategy: Precision Routing

Instead of broad categories, identify **specific query patterns** that need different models.

### Pattern Analysis

**Mistral-heavy queries contain:**
- Legal terms: "federal", "statute", "property", "action"
- Medical: "patients", "treatment", "health"
- Ethics: "cultural relativism", "permitted"

**Deepseek-heavy queries contain:**
- Code: "python", "function", "executable"
- Math: "\boxed{}", "within sentences"
- General facts

## Proposed Redesign

### 1. Add Legal/Medical Domain Detection

```python
LEGAL_MEDICAL_PATTERNS = [
    'federal', 'statute', 'plaintiff', 'defendant', 'court',
    'patient', 'diagnosis', 'treatment', 'clinical',
    'ethics', 'permitted', 'liability'
]
```

If query contains 2+ legal/medical terms → route to **mistral** not deepseek

### 2. Add Code/Math Detection

```python
CODE_MATH_PATTERNS = [
    'python', 'function', 'algorithm', '\\boxed',
    'calculate', 'integral', 'derivative'
]
```

If query contains 2+ code/math terms → route to **deepseek** (it excels here)

### 3. Keep Simple Factual on Deepseek

- Simple "what is X" → deepseek
- General knowledge → deepseek

## Expected Impact

| Query Type | Route To | Current % | Improved % |
|------------|----------|-----------|-------------|
| Legal/Medical | Mistral | 15% | 25% |
| Code/Math | Deepseek | Keep high | Keep high |
| General | Deepseek | High | Slightly lower |

**Goal:** Improve accuracy by 5-10% on the 15% mistral queries without hurting deepseek accuracy.

## Implementation

1. Add domain-specific pattern matching
2. Use ROUTING_CATEGORIES from baseline for benchmark-specific routing
3. Test locally with mock results before submitting

