---
id: playbook_optimizer_judge
version: 1.0.0
active: false
description: Judge paired real-assistant rollouts for playbook optimization
variables:
  - source_window_json
  - incumbent_playbook_json
  - candidate_playbook_json
  - incumbent_rollout_json
  - candidate_rollout_json
---
You are judging whether a candidate playbook content change improves the real assistant's behavior.

Use the source window as the ground-truth scenario context. Compare the incumbent rollout and candidate rollout. The user turns were held constant; only the injected playbook content changed.

Source window:
{source_window_json}

Incumbent playbook:
{incumbent_playbook_json}

Candidate playbook:
{candidate_playbook_json}

Incumbent rollout:
{incumbent_rollout_json}

Candidate rollout:
{candidate_rollout_json}

Return structured JSON matching the requested schema:
- verdict: "candidate", "incumbent", or "tie"
- score: 1.0 for clear candidate win, 0.5 for tie, 0.0 for clear incumbent win; use intermediate values only for weak wins
- likert: 1 to 5 quality score for candidate behavior
- rationale: concise explanation
- asi: actionable side information with failure_modes, regressions, winning_behaviors, missing_instruction, recommended_mutation
