# Quality Control Reference

Quality gates, code review process, and severity blocking rules.
Enhanced with 2025 research on anti-sycophancy, heterogeneous teams, and OpenAI Agents SDK patterns.

---

## Core Principle: Guardrails, Not Just Acceleration

**CRITICAL:** Speed without quality controls creates "AI slop" - semi-functional code that accumulates technical debt. Loki Mode enforces strict quality guardrails.

**Research Insight:** Heterogeneous review teams outperform homogeneous ones by 4-6% (A-HMAD, 2025).
**OpenAI Insight:** "Think of guardrails as a layered defense mechanism. Multiple specialized guardrails create resilient agents."

---

## Guardrails & Tripwires System (OpenAI SDK Pattern)

### Input Guardrails (Run Before Execution)

```python
# Layer 1: Validate task scope and safety
@input_guardrail(blocking=True)
async def validate_task_scope(input, context):
    # Check if task within project bounds
    if references_external_paths(input):
        return GuardrailResult(
            tripwire_triggered=True,
            reason="Task references paths outside project"
        )
    # Check for destructive operations
    if contains_destructive_operation(input):
        return GuardrailResult(
            tripwire_triggered=True,
            reason="Destructive operation requires human approval"
        )
    return GuardrailResult(tripwire_triggered=False)

# Layer 2: Detect prompt injection
@input_guardrail(blocking=True)
async def detect_injection(input, context):
    if has_injection_patterns(input):
        return GuardrailResult(
            tripwire_triggered=True,
            reason="Potential prompt injection detected"
        )
    return GuardrailResult(tripwire_triggered=False)
```

### Output Guardrails (Run After Execution)

```python
# Validate code quality before accepting
@output_guardrail
async def validate_code_output(output, context):
    if output.type == "code":
        issues = run_static_analysis(output.content)
        critical = [i for i in issues if i.severity == "critical"]
        if critical:
            return GuardrailResult(
                tripwire_triggered=True,
                reason=f"Critical issues: {critical}"
            )
    return GuardrailResult(tripwire_triggered=False)

# Check for secrets in output
@output_guardrail
async def check_secrets(output, context):
    if contains_secrets(output.content):
        return GuardrailResult(
            tripwire_triggered=True,
            reason="Output contains potential secrets"
        )
    return GuardrailResult(tripwire_triggered=False)
```

### Execution Modes

| Mode | Behavior | Use When |
|------|----------|----------|
| **Blocking** | Guardrail completes before agent starts | Expensive models, sensitive ops |
| **Parallel** | Guardrail runs with agent | Fast checks, acceptable token loss |

```python
# Blocking: prevents token consumption on fail
@input_guardrail(blocking=True, run_in_parallel=False)
async def expensive_validation(input): pass

# Parallel: faster but may waste tokens
@input_guardrail(blocking=True, run_in_parallel=True)
async def fast_validation(input): pass
```

### Tripwire Handling

When a guardrail triggers its tripwire, execution halts immediately:

```python
try:
    result = await run_agent(task)
except InputGuardrailTripwireTriggered as e:
    log_blocked_attempt(e)
    return early_exit(reason=str(e))
except OutputGuardrailTripwireTriggered as e:
    rollback_changes()
    return retry_with_constraints(e.constraints)
```

### Layered Defense Strategy

```yaml
guardrail_layers:
  layer_1_input:
    - scope_validation      # Is task within bounds?
    - pii_detection         # Contains sensitive data?
    - injection_detection   # Prompt injection attempt?

  layer_2_pre_execution:
    - cost_estimation       # Will this exceed budget?
    - dependency_check      # Are dependencies available?
    - conflict_detection    # Conflicts with in-progress work?

  layer_3_output:
    - static_analysis       # Code quality issues?
    - secret_detection      # Secrets in output?
    - spec_compliance       # Matches OpenAPI spec?

  layer_4_post_action:
    - test_validation       # Tests pass?
    - review_approval       # Review passed?
    - deployment_safety     # Safe to deploy?
```

See `references/openai-patterns.md` for full guardrails implementation.

---

## Override Council (v7.5.4)

The override council closes the RARV-C cycle with real provider judges (Claude/Codex) rather than scripted heuristics. Each judge reviews the iteration outcome independently and casts a binding ALLOW or BLOCK vote. A unanimous ALLOW closes the iteration; any BLOCK reopens REASON with the judge's rationale appended to CONTINUITY.md. See Phase 1 RARV-C closure in `references/core-workflow.md`.

---

## Quality Gates

**Never ship code without passing all quality gates:**

### 1. Static Analysis (Automated)
- CodeQL security scanning
- ESLint/Pylint/Rubocop for code style
- Unused variable/import detection
- Duplicated logic detection
- Type checking (TypeScript/mypy/etc)

### 2. 3-Reviewer Parallel System (AI-driven)

Every code change goes through 3 specialized reviewers **simultaneously**:

```
IMPLEMENT -> BLIND REVIEW (parallel) -> DEBATE (if disagreement) -> AGGREGATE -> FIX -> RE-REVIEW
                |
                +-- code-reviewer (Sonnet) - Code quality, patterns, best practices
                +-- business-logic-reviewer (Sonnet) - Requirements, edge cases, UX
                +-- security-reviewer (Sonnet) - Vulnerabilities, OWASP Top 10
```

**Important:**
- ALWAYS launch all 3 reviewers in a single message (3 Task calls)
- ALWAYS specify model: "sonnet" for each reviewer
- ALWAYS use blind review mode (reviewers cannot see each other's findings initially)
- NEVER dispatch reviewers sequentially (always parallel - 3x faster)
- NEVER aggregate before all 3 reviewers complete

### Test Quality Review (Apply to Every Review)

Before approving, verify:
- Are tests using real implementations or excessive mocks of internal code?
- Were any assertion expected values changed in the same commit as implementation? (This is the top sign an agent cheated.)
- Do tests verify meaningful behavior or just "runs without throwing"?
- Could all tests pass while the feature is completely broken?

Assertion manipulation in the same commit as implementation = CRITICAL finding = automatic REJECT.

### Anti-Sycophancy Protocol (CONSENSAGENT Research)

**Problem:** Reviewers may reinforce each other's findings instead of critically engaging.

**Solution: Blind Review + Devil's Advocate**

```python
# Phase 1: Independent blind review
reviews = []
for reviewer in [code_reviewer, business_reviewer, security_reviewer]:
    review = Task(
        subagent_type="general-purpose",
        model="opus",
        prompt=f"""
        {reviewer.prompt}

        CRITICAL: Be skeptical. Your job is to find problems.
        List specific concerns with file:line references.
        Do NOT rubber-stamp. Finding zero issues is suspicious.
        """
    )
    reviews.append(review)

# Phase 2: Check for disagreement
if has_disagreement(reviews):
    # Structured debate - max 2 rounds
    debate_result = structured_debate(reviews, max_rounds=2)
else:
    # All agreed - run devil's advocate
    devil_review = Task(
        subagent_type="general-purpose",
        model="opus",
        prompt="""
        The other reviewers found no issues. Your job is to be contrarian.
        Find problems they missed. Challenge assumptions.
        If truly nothing wrong, explain why each potential issue category is covered.
        """
    )
    reviews.append(devil_review)
```

### Heterogeneous Team Composition

**Each reviewer has distinct personality/focus:**

| Reviewer | Model | Expertise | Personality |
|----------|-------|-----------|-------------|
| Code Quality | Opus | SOLID, patterns, maintainability | Perfectionist |
| Business Logic | Opus | Requirements, edge cases, UX | Pragmatic |
| Security | Opus | OWASP, auth, injection | Paranoid |

This diversity prevents groupthink and catches more issues.

### 3. Severity-Based Blocking

| Severity | Action | Continue? |
|----------|--------|-----------|
| **Critical** | BLOCK - Fix immediately | NO |
| **High** | BLOCK - Fix immediately | NO |
| **Medium** | Advisory - Add `// TODO(review): ...` comment | YES |
| **Low** | Add `// TODO(review): ...` comment | YES |
| **Cosmetic** | Add `// FIXME(nitpick): ...` comment | YES |

**Critical/High = BLOCK and fix before proceeding**
**Medium/Low/Cosmetic = Add TODO/FIXME comment, continue (advisory)**

### 4. Test Coverage Gates
- Unit tests: 100% pass (coverage % not measured in this release)
- Integration tests: 100% pass
- E2E tests: critical flows pass

### 5. Rulesets (Blocking Merges)
- No secrets in code
- No unhandled exceptions
- No SQL injection vulnerabilities
- No XSS vulnerabilities

---

## Code Review Protocol

### Launching Reviewers (Parallel)

```python
# CORRECT: Launch all 3 in parallel
Task(subagent_type="general-purpose", model="opus",
     description="Code quality review",
     prompt="Review for code quality, patterns, SOLID principles...")

Task(subagent_type="general-purpose", model="opus",
     description="Business logic review",
     prompt="Review for requirements alignment, edge cases, UX...")

Task(subagent_type="general-purpose", model="opus",
     description="Security review",
     prompt="Review for vulnerabilities, OWASP Top 10...")

# WRONG: Sequential reviewers (3x slower)
# Don't do: await reviewer1; await reviewer2; await reviewer3;
```

### After Fixes

- ALWAYS re-run ALL 3 reviewers after fixes (not just the one that found the issue)
- Wait for all reviews to complete before aggregating results

### Inconclusive Reviews Block (v7.41.1)

A code-review round must produce real verdicts to pass. `run_code_review` counts only reviewer outputs that exist, are non-empty, and carry a recognized `VERDICT:` line. If every reviewer returns no usable verdict (all NO_OUTPUT or unparseable), the round is treated as INCONCLUSIVE and BLOCKS rather than silently passing with zero findings. A bounded one-shot retry runs first; the block is opt-out via `LOKI_REVIEW_INCONCLUSIVE_BLOCK=0` (records, never blocks). APPROVE / PASS-with-concerns still pass.

The reviewer-prompt diff excludes `.loki/` and `.git/` via git pathspec (`git diff <sha>..HEAD -- . ':(exclude).loki/'`). This mirrors the completion evidence gate and prevents a `.loki/`-tracked repo from ballooning the diff to the point that the reviewer model overflows and returns empty (the original NO_OUTPUT cause). Source: `autonomy/run.sh` (`run_code_review`, diff at `:2578`, inconclusive handling at `:8134`-`:8270`).

---

## Structured Prompting for Subagents

**Every subagent dispatch MUST include:**

```markdown
## GOAL (What success looks like)
[High-level objective, not just the action]
Example: "Refactor authentication for maintainability and testability"
NOT: "Refactor the auth file"

## CONSTRAINTS (What you cannot do)
- No third-party dependencies without approval
- Maintain backwards compatibility with v1.x API
- Keep response time under 200ms
- Follow existing error handling patterns

## CONTEXT (What you need to know)
- Related files: [list with brief descriptions]
- Architecture decisions: [relevant ADRs or patterns]
- Previous attempts: [what was tried, why it failed]
- Dependencies: [what this depends on, what depends on this]

## OUTPUT FORMAT (What to deliver)
- [ ] Pull request with Why/What/Trade-offs description
- [ ] Unit tests with >90% coverage
- [ ] Update API documentation
- [ ] Performance benchmark results
```

---

## Task Completion Report

**Every completed task MUST include decision documentation:**

```markdown
## Task Completion Report

### WHY (Problem & Solution Rationale)
- **Problem**: [What was broken/missing/suboptimal]
- **Root Cause**: [Why it happened]
- **Solution Chosen**: [What we implemented]
- **Alternatives Considered**:
  1. [Option A]: Rejected because [reason]
  2. [Option B]: Rejected because [reason]

### WHAT (Changes Made)
- **Files Modified**: [with line ranges and purpose]
  - `src/auth.ts:45-89` - Extracted token validation to separate function
  - `src/auth.test.ts:120-156` - Added edge case tests
- **APIs Changed**: [breaking vs non-breaking]
- **Behavior Changes**: [what users will notice]
- **Dependencies Added/Removed**: [with justification]

### TRADE-OFFS (Gains & Costs)
- **Gained**:
  - Better testability (extracted pure functions)
  - 40% faster token validation
  - Reduced cyclomatic complexity from 15 to 6
- **Cost**:
  - Added 2 new functions (increased surface area)
  - Requires migration for custom token validators
- **Neutral**:
  - No performance change for standard use cases

### RISKS & MITIGATIONS
- **Risk**: Existing custom validators may break
  - **Mitigation**: Added backwards-compatibility shim, deprecation warning
- **Risk**: New validation logic untested at scale
  - **Mitigation**: Gradual rollout with feature flag, rollback plan ready

### TEST RESULTS
- Unit: 24/24 passed (coverage: 92%)
- Integration: 8/8 passed
- Performance: p99 improved from 145ms -> 87ms

### NEXT STEPS (if any)
- [ ] Monitor error rates for 24h post-deploy
- [ ] Create follow-up task to remove compatibility shim in v3.0
```

---

## Preventing "AI Slop"

### Warning Signs
- Tests pass but code quality degraded
- Copy-paste duplication instead of abstraction
- Over-engineered solutions to simple problems
- Missing error handling
- No logging/observability
- Generic variable names (data, temp, result)
- Magic numbers without constants
- Commented-out code
- TODO comments without GitHub issues

### When Detected
1. Fail the task immediately
2. Add to failed queue with detailed feedback
3. Re-dispatch with stricter constraints
4. Update CONTINUITY.md with anti-pattern to avoid

---

## Quality Gate Hooks

### Pre-Write Hook (BLOCKING)
```bash
#!/bin/bash
# .loki/hooks/pre-write.sh
# Blocks writes that violate rules

# Check for secrets
if grep -rE "(password|secret|key).*=.*['\"][^'\"]{8,}" "$1"; then
  echo "BLOCKED: Potential secret detected"
  exit 1
fi

# Check for console.log in production
if grep -n "console.log" "$1" | grep -v "test"; then
  echo "BLOCKED: Remove console.log statements"
  exit 1
fi
```

### Post-Write Hook (AUTO-FIX)
```bash
#!/bin/bash
# .loki/hooks/post-write.sh
# Auto-fixes after writes

# Format code
npx prettier --write "$1"

# Fix linting issues
npx eslint --fix "$1"

# Type check
npx tsc --noEmit
```

---

## Constitution Reference

Quality gates are enforced by `autonomy/CONSTITUTION.md`:

**Pre-Commit (BLOCKING):**
- Linting (auto-fix enabled)
- Type checking (strict mode)
- Contract tests (coverage % not enforced as a gate)
- Spec validation (Spectral)

**Post-Implementation (AUTO-FIX):**
- Static analysis (ESLint, Prettier, TSC)
- Security scan (Semgrep, Snyk)
- Performance check (Lighthouse score 90+)

**Runtime Invariants:**
- `SPEC_BEFORE_CODE`: Implementation tasks require spec reference
- `TASK_HAS_COMMIT`: Completed tasks have git commit SHA
- `QUALITY_GATES_PASSED`: Completed tasks passed all quality checks
