# Critical Architectural Review: Execution Mode Specification

**Reviewer:** Senior Software Architect (Critical Perspective)  
**Date:** 2026-03-05  
**Spec:** `agentic_orchestrator_execution_mode_spec.md`  
**Verdict:** ⚠️ **MAJOR REVISIONS REQUIRED** - Original design had critical flaws

---

## 🔥 FATAL FLAWS IDENTIFIED

### 1. Race Conditions Everywhere

**Problem:** Watchdog monitors while agent writes. No coordination.

**Original Design:** Agent writes freely, checkpoint runs whenever triggered.

**Critical Issues:**

- Checkpoint validation runs while agent is mid-write
- Multiple checkpoints could trigger simultaneously
- File system events arrive out of order
- No write transaction boundaries

**Fix Applied:** Agent pause/resume protocol with acknowledgment, checkpoint serialization, debouncing.

---

### 2. Validation is Too Late

**Problem:** Interactive mode validates AFTER changes are written.

**Original Design:** Agent writes → Checkpoint captures diff → Validate → Maybe revert.

**Critical Issues:**

- Files already modified (potential corruption)
- Agent made dependent changes based on invalid state
- Revert is destructive and loses work
- No atomic rollback guarantee

**Fix Applied:** Shadow workspace strategy - agent writes to isolated copy, validation promotes to real worktree only if valid.

---

### 3. No Rollback Strategy

**Problem:** "Optional revert with `git checkout`" is naive.

**Original Design:** If validation fails, run `git checkout -- <files>`.

**Critical Issues:**

- What if agent has uncommitted changes across 50 files?
- What if some changes valid, some invalid?
- What if agent still writing when revert happens?
- What about file system state (temp files, build artifacts)?

**Fix Applied:** Three rollback strategies:

1. Full checkpoint rollback (restore to exact checkpoint state)
2. Partial file rollback (revert only violated files)
3. Smart rollback (keep valid changes, revert only violations)

---

### 4. Checkpoint Overhead is Unbounded

**Problem:** Every checkpoint runs full `git diff` + validation.

**Original Design:** No optimization, no caching, no timeouts.

**Critical Issues:**

- With 1000 files: parsing takes seconds
- Complex plan patterns: validation is slow
- No timeout budget
- Blocks agent progress

**Fix Applied:**

- Validation cache (per file + plan version)
- Incremental diff (only changed files since last checkpoint)
- Timeout budgets (5s max, then degrade)
- Sampling for large changesets (>100 files)

---

### 5. Concurrent Feature Isolation is Broken

**Problem:** "Each feature has isolated worktree" but services are singletons.

**Original Design:** Shared watchdog service, no concurrency control.

**Critical Issues:**

- Watchdog service has shared state
- Checkpoint service has no concurrency control
- Validation service could be bottlenecked
- File system events could be misattributed

**Fix Applied:**

- Per-feature watchdog instances (separate `chokidar` per feature)
- Checkpoint serialization per feature
- Validation queue with priority and concurrency limits
- Backpressure (max 5 concurrent interactive features)

---

### 6. Agent Notification is Fire-and-Forget

**Problem:** "Send message via WorkerProvider.sendMessage" assumes agent listens.

**Original Design:** No acknowledgment, no retry, no escalation.

**Critical Issues:**

- Agent may not be listening
- Agent may not respond
- Agent may not understand message format
- Agent could ignore critical violations

**Fix Applied:** Structured message protocol with:

- Acknowledgment requirement for critical messages
- Timeout and retry logic
- Escalation to human on repeated failures
- Agent pause on critical violations

---

### 7. No Graceful Degradation

**Problem:** If watchdog/checkpoint/validation fails, what happens?

**Original Design:** No error handling, no fallback, no recovery.

**Critical Issues:**

- Agent continues running blind
- Changes accumulate without validation
- System state becomes inconsistent
- No recovery path

**Fix Applied:**

- Circuit breakers (3 failures → escalate to human)
- Health checks for all services
- Automatic fallback to deterministic mode
- Agent timeout with forced checkpoint
- Recovery protocol with state capture

---

### 8. Audit Trail is Incomplete

**Problem:** Checkpoints only capture state at intervals.

**Original Design:** Only checkpoint snapshots, no intermediate events.

**Critical Issues:**

- No record of intermediate states
- No record of attempted-but-reverted files
- No record of agent's decision process
- Cannot reconstruct exact sequence

**Fix Applied:**

- Continuous event log (every file change with timestamp)
- Checkpoint chains (linked list of checkpoints)
- Incremental + cumulative diffs
- Validation history log

---

### 9. Security Nightmare

**Problem:** Agent has direct file system access with only periodic validation.

**Original Design:** No sandboxing, no resource limits, no path validation.

**Critical Issues:**

- Agent could write malicious code that executes before checkpoint
- Agent could modify `.git` directory
- Agent could escape worktree via symlinks
- Agent could exhaust disk space

**Fix Applied:**

- Path canonicalization and validation
- `.git` directory protection (immediate block)
- Symlink detection and removal
- Disk quota per feature (1GB limit)
- Executable file detection and flagging

---

### 10. Performance Assumptions are Unvalidated

**Problem:** "Checkpoint validation completes in < 500ms" - based on what?

**Original Design:** No benchmarks, no budgets, no degradation strategy.

**Critical Issues:**

- What if diff is 10,000 lines?
- What if plan has 100 allowed_areas patterns?
- What if lock service is slow?
- What if file system is network-mounted?

**Fix Applied:**

- Explicit latency budgets with targets and maximums
- Timeout-based degradation strategies
- Throughput requirements
- Resource limits per feature and globally
- Monitoring metrics and alert thresholds

---

## ✅ IMPROVEMENTS APPLIED

### Architecture Enhancements

1. **Shadow Workspace Strategy**
   - Agent writes to isolated shadow directory
   - Validation before promotion to real worktree
   - Atomic promotion or discard
   - Configurable per feature or globally

2. **Agent Communication Protocol**
   - Structured message format (JSON)
   - Pause/resume with acknowledgment
   - Violation notifications with severity
   - Timeout and retry logic

3. **Rollback Capabilities**
   - Full checkpoint rollback
   - Partial file rollback
   - Smart rollback (keep valid, revert violations)
   - Inverse diff application

4. **Performance Budgets**
   - Latency targets: checkpoint < 500ms, validation < 1s
   - Throughput targets: 100 file changes/sec
   - Resource limits: 1GB disk per feature, 50MB memory per watchdog
   - Degradation strategies for all timeouts

5. **Concurrency Control**
   - Per-feature service instances
   - Validation queue with priority
   - Backpressure (max 5 concurrent interactive features)
   - Event attribution per feature

6. **Security Hardening**
   - Path validation and canonicalization
   - `.git` directory protection
   - Symlink detection and blocking
   - Disk quota enforcement
   - Executable file detection

7. **Graceful Degradation**
   - Circuit breakers (3 failures → escalate)
   - Health checks for all services
   - Automatic fallback to deterministic mode
   - Recovery protocol with state capture

8. **Audit Trail Completeness**
   - Continuous event log (JSONL format)
   - Checkpoint chains (linked list)
   - Incremental + cumulative diffs
   - Validation history

9. **Monitoring and Observability**
   - Checkpoint latency (p50, p95, p99)
   - Validation latency and cache hit rate
   - Checkpoint failure rate
   - Disk/memory usage per feature
   - Alert thresholds

10. **Provider Capability Detection**
    - Query provider for interactive mode support
    - Automatic fallback if unsupported
    - Clear error messages
    - Standardized capability interface

---

## 📊 SPEC METRICS

| Metric                   | Before | After      | Change |
| ------------------------ | ------ | ---------- | ------ |
| Total lines              | 951    | 1805       | +90%   |
| Risk sections            | 4      | 9          | +125%  |
| Mitigation strategies    | 8      | 27         | +238%  |
| Code examples            | 12     | 28         | +133%  |
| Performance requirements | 0      | 4 sections | NEW    |
| Security mitigations     | 1      | 5          | +400%  |

---

## 🎯 RECOMMENDATION

**Status:** ✅ **APPROVED WITH REVISIONS**

The original design was fundamentally flawed for production use. The revised spec addresses all critical issues and is now ready for implementation with the following caveats:

1. **Start with Strategy A (Direct Worktree)** for simplicity, but implement Strategy B (Shadow Workspace) infrastructure from day one for easy migration.

2. **Implement circuit breakers and health checks first** before any interactive mode execution.

3. **Performance budgets are mandatory** - do not ship without timeout and degradation strategies.

4. **Security hardening is non-negotiable** - path validation, `.git` protection, and disk quotas must be in place before beta.

5. **Monitor everything** - checkpoint latency, validation failures, disk usage, memory usage. Set up alerts before production.

---

## 📋 IMPLEMENTATION PRIORITY

### P0 (Must Have Before Any Interactive Execution)

- Agent pause/resume protocol
- Path validation and security hardening
- Circuit breakers and health checks
- Checkpoint serialization (no concurrent checkpoints per feature)

### P1 (Must Have Before Beta)

- Shadow workspace strategy implementation
- Rollback strategies (full, partial, smart)
- Performance budgets and timeouts
- Monitoring and alerts

### P2 (Must Have Before Production)

- Validation cache
- Continuous event log
- Provider capability detection
- Graceful degradation and fallback

### P3 (Nice to Have)

- Validation sampling for large changesets
- Real-time dashboard streaming
- Checkpoint comparison UI
- Advanced rollback commands

---

**Conclusion:** The spec is now production-ready with comprehensive risk mitigation. Original design would have failed catastrophically in production. Revised design is robust, safe, and performant.
