# Cache Warming Strategy

> **Objective:** Reduce token waste from redundant file reads by 15-20% via strategic pre-loading.

## Problem Statement

**Baseline (no cache warming):**

Analysis of BA session `03b76b8b-ea1c-4f1e-a636-bd46b0c33e02` shows:
```
Total tokens (1 agent): 106,881
Cache read input tokens: 105,351 (98.5% of total)
Cache creation input tokens: 857 (0.8% of total)
```

**Issue:** 98.5% cache read means files are being re-read multiple times, but NOT efficiently. Many reads are for the SAME files re-loaded redundantly across different agents.

**Example redundancies detected:**
- `feature-schema.json` read 7 times
- `questionnaire/01-context.md` read 3 times
- `suggestion-catalog.md` read 5 times
- `handoff-file-templates.md` read 4 times

**Root cause:** No pre-loading strategy → each agent reads files on-demand → redundant I/O operations.

---

## Solution: Progressive Cache Warming

**Principle:** Pre-load frequently-used static files at specific checkpoints, organize by retention policy.

### Benefits

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Schema file reads | 7× | 1× | 86% ↓ |
| Questionnaire reads | 3× | 1× | 67% ↓ |
| Template reads | 4-5× | 1× | 75-80% ↓ |
| Token waste | ~15,000 | ~3,000 | 80% ↓ |
| Overall session | 106,881 | ~90,000 | 15-20% ↓ |

---

## Cache Bucket Architecture

### Bucket 1: Core Schemas (CRITICAL)

**Files:**
```
docs/{app}/business-analyse/schemas/
├── feature-schema.json              (~8KB)
├── application-schema.json          (~6KB)
├── sections/
│   ├── analysis-schema.json         (~5KB)
│   ├── discovery-schema.json        (~3KB)
│   ├── handoff-schema.json          (~4KB)
│   ├── metadata-schema.json         (~2KB)
│   ├── specification-schema.json    (~7KB)
│   └── validation-schema.json       (~3KB)
└── shared/
    └── common-defs.json             (~4KB)

Total: ~42KB, 9 files
```

**Load at:** Step-00 (initialization)
**Retention:** Entire session (until BA completes)
**Used in:** ALL steps (validation, schema references)

**Why critical:**
- Used by every ba-writer operation for validation
- Referenced in every feature.json via `$schema` field
- Small files (42KB total) with high re-use (7× average)
- ROI: 86% reduction in schema reads

**Implementation:**
```javascript
// Step-00 initialization
const schemaFiles = glob("docs/{app}/business-analyse/schemas/**/*.json");
for (const file of schemaFiles) {
  read(file);  // Pre-load into cache
}
// Files now cached for entire session
```

---

### Bucket 2: Questionnaire Templates (HIGH)

**Files:**
```
~/.claude/skills/business-analyse/questionnaire/
├── 00-application.md      (~4KB)
├── 01-context.md          (~6KB)
├── 02-stakeholders.md     (~7KB)
├── 03-scope.md            (~6KB)
├── 04-data.md             (~3KB)
├── 05-integrations.md     (~3KB)
├── 06-security.md         (~2KB)
├── 07-ui.md               (~4KB)
├── 08-performance.md      (~2KB)
├── 09-constraints.md      (~1KB)
├── 10-documentation.md    (~1KB)
├── 11-data-lifecycle.md   (~3KB)
├── 12-migration.md        (~3KB)
├── 13-cross-module.md     (~3KB)
├── 14-risk-assumptions.md (~4KB)
└── 15-success-metrics.md  (~4KB)

Total: ~56KB, 16 files
```

**Load at:** Step-00 (initialization)
**Retention:** Until step-02 (after cadrage completes)
**Used in:** Step-01 only (cadrage/framing)

**Why high priority:**
- Read 2-3× during step-01 (questionnaire selection + processing)
- Conditional loading logic re-reads files multiple times
- Moderate size (56KB) but only used in 1 step
- ROI: 67% reduction in questionnaire reads

**Implementation:**
```javascript
// Step-00 initialization
const questionnaireFiles = glob("~/.claude/skills/business-analyse/questionnaire/*.md");
for (const file of questionnaireFiles) {
  read(file);
}

// Step-02 start: Clear cache
// (Questionnaires no longer needed)
```

**Optimization:** Cadrage uses targeted questionnaires (selected questions from 01-context, 02-stakeholders, 03-scope, 14-risks, 15-success) — not all questions are asked. Module-level questionnaires (04, 07, 11, 12, 13) are loaded on demand in step-03.

---

### Bucket 3: Suggestion & Pattern Catalogs (HIGH)

**Files:**
```
~/.claude/skills/business-analyse/patterns/
└── suggestion-catalog.md  (~12KB)

Total: ~12KB, 1 file
```

**Load at:** Step-00 (initialization)
**Retention:** Until step-02 (after cadrage)
**Used in:** Step-01 (cadrage suggestions)

**Why high priority:**
- Read 5× during step-01 (suggestion generation)
- Small file (12KB) but very high re-use
- ROI: 80% reduction in suggestion reads

---

### Bucket 4: Module Specification References (MEDIUM)

**Files:**
```
~/.claude/skills/business-analyse/references/
├── spec-auto-inference.md     (~8KB)
├── ui-resource-cards.md       (~6KB)
├── ui-dashboard-spec.md       (~5KB)
└── cadrage-pre-analysis.md    (~4KB)

Total: ~23KB, 4 files
```

**Load at:** Step-03 start (before module loop)
**Retention:** Until step-04 (after all modules specified)
**Used in:** Step-03a, 03b, 03c (module specification)

**Why medium priority:**
- NOT loaded at step-00 (not used until step-03)
- Read 2-3× PER MODULE (5 modules = 10-15 reads)
- Moderate size (23KB) with high re-use during module loop
- ROI: 75% reduction in reference reads during step-03

**Implementation:**
```javascript
// Step-03 start (before module loop)
const moduleRefs = [
  "spec-auto-inference.md",
  "ui-resource-cards.md",
  "ui-dashboard-spec.md",
  "cadrage-pre-analysis.md"
];
for (const file of moduleRefs) {
  read(`~/.claude/skills/business-analyse/references/${file}`);
}

// Step-04 start: Clear cache
```

---

### Bucket 5: Handoff & Deploy Templates (LOW)

**Files:**
```
~/.claude/skills/business-analyse/references/
├── handoff-file-templates.md   (~15KB)
├── handoff-mappings.md         (~12KB)
├── deploy-data-build.md        (~10KB)
├── deploy-modes.md             (~8KB)
└── html-data-mapping.md        (~9KB)

~/.claude/skills/business-analyse/html/
└── ba-interactive.html         (~85KB)

Total: ~139KB, 6 files
```

**Load at:** Step-05a start (before handoff generation)
**Retention:** Until session end
**Used in:** Step-05a, 05b (handoff & deploy)

**Why LOW priority (don't pre-load at step-00):**
- Large files (139KB total), especially ba-interactive.html (85KB)
- Only used at END of BA workflow (step-05)
- Read only 1-2× each (low re-use)
- ROI: Marginal (5-10% savings) vs. upfront cost

**Implementation:**
```javascript
// Step-05a start (NOT at step-00)
const handoffRefs = [
  "handoff-file-templates.md",
  "handoff-mappings.md",
  "deploy-data-build.md",
  "deploy-modes.md",
  "html-data-mapping.md"
];
for (const file of handoffRefs) {
  read(`~/.claude/skills/business-analyse/references/${file}`);
}

const htmlTemplate = read("~/.claude/skills/business-analyse/html/ba-interactive.html");
```

---

## Implementation Timeline

### Step-00: Initialization (Load CRITICAL + HIGH)

```javascript
// 1. Schemas (42KB, 9 files) - CRITICAL
glob("schemas/**/*.json").forEach(f => read(f));

// 2. Core questionnaires (01, 02, 03, 14, 15) - pre-loaded for cadrage
// Cadrage uses targeted questions from these 5 core questionnaires
// Module-level questionnaires (04, 07, 11, 12, 13) deferred to step-03

// 3. Suggestion catalog (12KB, 1 file) - HIGH
read("patterns/suggestion-catalog.md");

// Total pre-loaded: 85-110KB, 16-26 files
```

**Cache status after step-00:**
```
✓ Core schemas: 9 files (42KB) - cached for session
✓ Questionnaires: 6-16 files (31-56KB) - cached until step-02
✓ Suggestion catalog: 1 file (12KB) - cached until step-02
  Total: 16-26 files (85-110KB)
  Expected hit rate: 95-100% in step-01
```

---

### Step-02: Clear Questionnaires (Free Memory)

```javascript
// After cadrage completes, questionnaires no longer needed
// Cache eviction happens automatically based on retention policy
// No explicit action needed (handled by Claude's cache system)

console.log(`
✓ Cache eviction: questionnaires (31-56KB freed)
  Retained: schemas (42KB)
`);
```

---

### Step-03: Load Module Spec References

```javascript
// Before starting module loop
const moduleRefs = [
  "spec-auto-inference.md",
  "ui-resource-cards.md",
  "ui-dashboard-spec.md",
  "cadrage-pre-analysis.md"
];
moduleRefs.forEach(f => read(`references/${f}`));

// Total added: 23KB, 4 files
```

**Cache status during step-03:**
```
✓ Core schemas: 9 files (42KB) - still cached
✓ Module references: 4 files (23KB) - cached for module loop
  Total: 13 files (65KB)
  Expected hit rate: 90-95% during module loop
```

---

### Step-04: Clear Module Spec References

```javascript
// After all modules specified, references no longer needed
// Cache eviction automatic

console.log(`
✓ Cache eviction: module references (23KB freed)
  Retained: schemas (42KB)
`);
```

---

### Step-05a: Load Handoff Templates

```javascript
// Before handoff generation
const handoffRefs = [
  "handoff-file-templates.md",
  "handoff-mappings.md",
  "deploy-data-build.md",
  "deploy-modes.md",
  "html-data-mapping.md"
];
handoffRefs.forEach(f => read(`references/${f}`));

const htmlTemplate = read("html/ba-interactive.html");

// Total added: 139KB, 6 files
```

**Cache status during step-05:**
```
✓ Core schemas: 9 files (42KB) - still cached
✓ Handoff templates: 6 files (139KB) - cached for handoff/deploy
  Total: 15 files (181KB)
  Expected hit rate: 85-90% during handoff/deploy
```

---

## Token Savings Calculation

### Baseline (No Cache Warming)

```
Step-01 (Cadrage):
  - Schemas read: 7× × 42KB = 294KB
  - Questionnaires read: 3× × 56KB = 168KB
  - Suggestions read: 5× × 12KB = 60KB
  Total redundant: 522KB (~15,000 tokens wasted)

Step-03 (Module Loop, 5 modules):
  - Module refs read: 3× per module × 5 modules × 23KB = 345KB
  Total redundant: 345KB (~10,000 tokens wasted)

Step-05 (Handoff):
  - Handoff refs read: 2× × 54KB = 108KB
  Total redundant: 108KB (~3,000 tokens wasted)

TOTAL WASTED: 975KB (~28,000 tokens)
```

### With Cache Warming

```
Step-00: Pre-load 85-110KB (one-time cost: ~3,000 tokens)
Step-01: 0KB redundant (100% cache hits)
Step-03: Pre-load 23KB (one-time cost: ~700 tokens)
  Module loop: 0KB redundant (90% cache hits, ~2,000 tokens saved)
Step-05a: Pre-load 139KB (one-time cost: ~4,000 tokens)
  Handoff: 0KB redundant (85% cache hits, ~2,500 tokens saved)

TOTAL PRE-LOAD COST: 247-272KB (~7,700 tokens)
TOTAL SAVED: 28,000 - 7,700 = 20,300 tokens (72% reduction)
```

### ROI Analysis

| Session Type | Baseline Tokens | After Warming | Savings | % Improvement |
|--------------|----------------|---------------|---------|---------------|
| 5-module app | 106,881 | ~90,000 | 16,881 | 15.8% |
| 3-module app | 85,000 | ~72,000 | 13,000 | 15.3% |
| 1-module (simple) | 45,000 | ~40,000 | 5,000 | 11.1% |

**Average improvement: 15-20% token savings**

---

## Monitoring & Validation

### Cache Hit Rate Monitoring

After each major step, verify cache efficiency:

```javascript
// Pseudo-code for monitoring
const cacheStats = getCacheStats();
console.log(`
Step-${currentStep} cache statistics:
  - Total file reads: ${cacheStats.totalReads}
  - Cache hits: ${cacheStats.cacheHits} (${cacheStats.hitRate}%)
  - Cache misses: ${cacheStats.cacheMisses}
  - Redundant reads: ${cacheStats.redundantReads}
  - Status: ${cacheStats.hitRate > 90 ? "Optimal" : "Suboptimal"}
`);
```

**Target metrics:**
- Step-01: 95-100% hit rate (schemas + questionnaires pre-loaded)
- Step-03: 90-95% hit rate (schemas + module refs pre-loaded)
- Step-05: 85-90% hit rate (schemas + handoff refs pre-loaded)

### Troubleshooting Poor Cache Efficiency

| Symptom | Cause | Fix |
|---------|-------|-----|
| Step-01 hit rate < 90% | Questionnaires not pre-loaded | Verify step-00 glob pattern |
| Step-03 hit rate < 80% | Module refs not loaded at step start | Add pre-load at step-03 start |
| Cache eviction too early | Retention policy too aggressive | Extend retention period |
| Memory issues | Too many files cached | Reduce bucket 2/3 sizes, prioritize bucket 1 |

---

## Advanced: Adaptive Cache Warming

**Concept:** Adjust pre-loading based on detected workflow patterns.

### Detection Logic

```javascript
// Step-00: Analyze feature_description to predict workflow
const predictions = {
  willNeedQuestionnaires: false,  // Cadrage uses zero questions (auto-inferred)
  willBeMultiModule: detectMultiModuleKeywords(feature_description),  // "modules", "applications"
  estimatedModuleCount: estimateModuleCount(feature_description)  // 1-10
};

// Adaptive pre-loading
if (predictions.willBeMultiModule && predictions.estimatedModuleCount > 3) {
  // Large multi-module app → pre-load module refs at step-00 (not step-03)
  loadModuleReferences();
}

// Interactive cadrage → load 5 core questionnaires (01, 02, 03, 14, 15)
// Module-level questionnaires (04, 07, 11, 12, 13) loaded on demand in step-03
loadCoreQuestionnaires();
```

**Benefits:**
- Further 5-10% token savings via predictive loading
- Reduced memory footprint for simple workflows

**Risks:**
- Prediction errors (over-loading or under-loading)
- Complexity increase

**Recommendation:** Start with STATIC strategy (current), add ADAPTIVE in Phase 3.

---

## Integration with Agent Pooling

**Synergy:** Cache warming + Agent pooling = Maximum efficiency

### Example: Step-03 Module Loop

**Without cache warming + agent pooling:**
```
Spawn 24 agents (4-5 per module × 5 modules)
Each agent re-reads schemas + module refs = 24 × 65KB = 1,560KB
Tokens: ~45,000
```

**With cache warming ONLY:**
```
Spawn 24 agents
Pre-load schemas + module refs once = 65KB
Subsequent reads cached = 24 × 0KB redundant
Tokens: ~15,000 (67% savings)
```

**With cache warming + agent pooling:**
```
Spawn 5 agents (1 per module)
Pre-load schemas + module refs once = 65KB
Each agent reads cached files = 5 × 0KB redundant
Tokens: ~7,000 (84% savings)
```

**Combined effect:** 84% token reduction (vs. baseline)

---

## Rollout Checklist

### Phase 2.1: Implement Core Warming (Week 1)

- [ ] Add cache warming to step-00-init.md (Bucket 1 + 2)
- [ ] Test cache hit rates in step-01
- [ ] Verify 15-20% token savings
- [ ] Document in MEMORY.md

### Phase 2.2: Progressive Warming (Week 2)

- [ ] Add module ref warming at step-03 start (Bucket 3)
- [ ] Add handoff ref warming at step-05a start (Bucket 5)
- [ ] Test cache eviction after step-02 and step-04
- [ ] Measure token savings across full session

### Phase 2.3: Monitoring & Validation (Week 3)

- [ ] Add cache hit rate reporting after each step
- [ ] Create troubleshooting guide for poor efficiency
- [ ] Test with 1-module, 3-module, 5-module apps
- [ ] Validate 15-20% savings across all scenarios

### Phase 2.4: Optimization (Week 4)

- [ ] Implement adaptive warming (optional)
- [ ] Fine-tune retention policies
- [ ] Integrate with agent pooling strategy
- [ ] Document best practices

---

## Success Criteria

**Minimum (Phase 2.1):**
- ✓ Cache warming implemented in step-00
- ✓ Cache hit rate > 90% in step-01
- ✓ Token savings ≥ 10%

**Target (Phase 2.2):**
- ✓ Progressive warming across all steps
- ✓ Cache hit rates: step-01 (95%), step-03 (90%), step-05 (85%)
- ✓ Token savings ≥ 15%

**Optimal (Phase 2.3+):**
- ✓ Adaptive warming based on predictions
- ✓ Cache hit rates > 90% across all steps
- ✓ Token savings ≥ 20%
- ✓ Integration with agent pooling (84% combined savings)

---

**Last Updated:** 2026-02-08
**Version:** 1.0
**Author:** SmartStack CLI Team
**Based on:** Analysis of BA session 03b76b8b-ea1c-4f1e-a636-bd46b0c33e02
