# Cluster Partitioning

Heuristics for partitioning N in-scope files into K clusters of `migration-cluster-worker` invocations. The orchestrator authors `tmp/${MIGRATION_ID}/clusters.json` in Phase 03; Phase 04 reads it and dispatches one `Task()` per cluster, all in a single assistant message (parallel scheduling).

Proven on PR #68 — `source-tree.md` → `source-tree/` migrated 324 files across 7 parallel clusters; zero dual-path drift.

---

## Rules of Thumb

| Heuristic | Default value | Rationale |
|-----------|---------------|-----------|
| Files per cluster | ~20–25 | Keeps a single worker's token budget under 50K (per `migration-cluster-worker.md` Success Criteria). |
| Max clusters per dispatch | 7 | Matches PR #68's proven 7-agent dispatch; larger fan-outs risk parallel scheduler saturation. |
| Min cluster size | 1 | A single-file cluster is permissible if it isolates an edge case the orchestrator wants quarantined. |
| Single-message dispatch | MANDATORY | All N `Task()` calls in ONE assistant turn — sequential dispatch defeats parallelism (see `phase-04-cluster-execution.md` Step 4.2). |

A migration with >175 in-scope files (7 × 25) MUST be partitioned into more clusters than the 7-dispatch ceiling — see [Two-Wave Dispatch](#two-wave-dispatch-when-files--7--25) below.

---

## Partition Strategies

### Strategy 1 — By directory (default)

Partition along natural directory boundaries. This is what PR #68 used; it produces the cleanest separation of concerns (each worker owns ONE framework subtree).

Example for the source-tree migration (`scope_dirs` = `.claude/{agents,skills,commands,rules,hooks}/` + `src/claude/` mirrors):

| `cluster_id` | `scope_dirs` |
|--------------|--------------|
| `agents` | `[".claude/agents/", "src/claude/agents/"]` |
| `dev-skills` | `[".claude/skills/spec-driven-dev/", "src/claude/skills/spec-driven-dev/"]` |
| `qa-skills` | `[".claude/skills/spec-driven-qa/", "src/claude/skills/spec-driven-qa/"]` |
| `other-skills` | `[".claude/skills/", "src/claude/skills/"]` (with `exclusions` for spec-driven-dev/qa) |
| `commands` | `[".claude/commands/", "src/claude/commands/"]` |
| `rules` | `[".claude/rules/", "src/claude/rules/"]` |
| `hooks` | `[".claude/hooks/", "src/claude/hooks/"]` |

**Strengths:** zero overlap, easy mental model, each worker reasons inside one subsystem's idioms.
**Weaknesses:** cluster sizes uneven (`agents` may have 60 files while `hooks` has 8).

### Strategy 2 — By component

Partition along framework component lines (subagents, skills, hooks, scripts) regardless of directory layout. Useful when the migration touches semantically-related files scattered across multiple directories.

Use when a `from_pattern` is conceptually scoped (e.g. a subagent rename) rather than directory-scoped.

### Strategy 3 — By file-size balanced

When directory partitioning produces wildly uneven cluster sizes (one cluster has 80 files, another 5), rebalance: keep big clusters cohesive but lift large outliers into their own sub-cluster.

Example: if `.claude/skills/` has 60 in-scope files but the next-biggest scope has 12, split skills into `dev-skills` / `qa-skills` / `other-skills` (Strategy 1 mixed with size-balancing).

### Strategy 4 — By MIXED isolation

Files the classifier flagged as MIXED (containing ALREADY-MIGRATED prose mixed with transformable buckets) benefit from being grouped into ONE cluster the orchestrator can supervise. Workers handle MIXED files line-by-line; isolating them simplifies post-execution review.

Use when the MIXED list is small (≤5 files) and the orchestrator wants explicit per-file attention.

---

## Two-Wave Dispatch (when files > 7 × 25)

If the migration corpus exceeds ~175 files, partition into >7 clusters and dispatch in waves of ≤7. The orchestrator:

1. Dispatches wave 1 (≤7 `Task()` calls in one message).
2. Collects all wave-1 reports.
3. Dispatches wave 2 (the remaining clusters) in a fresh single message.
4. Consolidates reports across waves.

Each wave's reports are aggregated into the same `tmp/${MIGRATION_ID}/phase-04-cluster-reports.md` consolidated file.

---

## Mid-Execution Cluster Split

If a cluster worker reports token budget exceeded (>50K consumed for a single cluster), the orchestrator:

1. HALTs Phase 04 progress for that cluster.
2. Reads the worker's partial report (which files were processed; which remain).
3. Subdivides the cluster's `scope_dirs` into 2-3 smaller sub-clusters in `clusters.json`.
4. Re-dispatches ONLY the sub-clusters covering the unprocessed files; does NOT re-dispatch already-processed files.

The orchestrator records the split in the consolidated report so the migration audit trail explains why one nominal cluster produced multiple reports.

---

## `clusters.json` Schema

The orchestrator authors `tmp/${MIGRATION_ID}/clusters.json` in Phase 03. Phase 04 Step 4.1 parses it.

```json
{
  "migration_id": "source-tree-2026-05-19",
  "partition_strategy": "by-directory",
  "clusters": [
    {
      "cluster_id": "agents",
      "scope_dirs": [".claude/agents/", "src/claude/agents/"],
      "exclusions": ["_archive/"],
      "approx_file_count": 22
    },
    {
      "cluster_id": "dev-skills",
      "scope_dirs": [".claude/skills/spec-driven-dev/", "src/claude/skills/spec-driven-dev/"],
      "exclusions": [],
      "approx_file_count": 18
    }
  ]
}
```

### Required fields (per Phase 04 Step 4.1)

| Field | Type | Description |
|-------|------|-------------|
| `migration_id` | string | The migration session ID (matches `intent.json`). |
| `partition_strategy` | enum | `by-directory` \| `by-component` \| `by-size-balanced` \| `by-mixed-isolation` \| `hybrid`. |
| `clusters` | array | Non-empty list of cluster objects. |

### Per-cluster fields

| Field | Type | Description |
|-------|------|-------------|
| `cluster_id` | string | Short kebab-case identifier (e.g. `agents`, `dev-skills`). Unique across the partition. |
| `scope_dirs` | array of strings | Directory globs this cluster owns. For dual-path migrations, MUST include BOTH the `.claude/` and `src/claude/` mirror — Phase 04 Step 4.1 explicitly checks this. |
| `exclusions` | array of strings | Path patterns to skip (typically a subset of the migration-wide exclusions, but a cluster may add its own). |
| `approx_file_count` | integer | The orchestrator's estimate from `classification.md`; used for budget sanity-checking, not enforcement. |

### Optional fields

| Field | Type | Description |
|-------|------|-------------|
| `dispatch_wave` | integer | 1, 2, … — set when using two-wave dispatch. Defaults to 1. |
| `note` | string | Free-text orchestrator note (e.g. "isolated MIXED files for manual review"). |

---

## Validation Checks

Before dispatching Phase 04 workers, the orchestrator confirms `clusters.json`:

- [ ] Parses as valid JSON.
- [ ] Contains at least one cluster.
- [ ] Every cluster has `cluster_id`, `scope_dirs`, `exclusions`, `approx_file_count`.
- [ ] Every `cluster_id` is unique within the partition.
- [ ] For dual-path migrations: every cluster's `scope_dirs` includes BOTH the `.claude/` and `src/claude/` mirror for any covered subtree (Phase 04 Step 4.1 enforces this).
- [ ] No cluster's `approx_file_count` exceeds 50 (warning threshold; consider subdivision).
- [ ] The union of cluster `scope_dirs` covers every directory in `intent.json` `scope_dirs` (no orphaned scope).

---

## References

- `phase-04-cluster-execution.md` Step 4.1 — Phase 04's `clusters.json` parsing.
- `phase-04-cluster-execution.md` Step 4.2 — single-message dispatch protocol.
- `migration-cluster-worker.md` Success Criteria — 50K token cap per cluster.
- `[[feedback_dual_path_mechanical_migration]]` Step 4 — "Cluster-delegate by directory" — the PR #68 evidence.
