# Sync Protocol Testing & Verification

This document outlines the testing strategy, verification results, and comprehensive guidelines for testing the Symmetric Sync Protocol.

💣💥 **USE THESE GUIDELINES TO MITIGATE COMPLEXITY EXPLOSION** 💣💥

Sync needs thorough testing and complexity increases exponentially. Ridiculously exponentially. I mean you won't believe just how vastly, hugely, mind-bogglingly fast...

---

## Testing Strategy

We use `respec-gib` (a BDD-style framework) with **In-Memory Simulation (`InnerSpace`)** to verify logic. This allows us to test complex graph scenarios without network or disk I/O overhead.

### Key Test Files

To see the most up-to-date test files for sync, search inside `libs/core-gib/src/sync` for `*.respec.mts` files.

**Basic Scenarios**:
*   `sync-innerspace.respec.mts`: Basic Push/Pull scenarios
*   `sync-innerspace-dest-ahead.respec.mts`: Receiver has newer data scenarios
*   `sync-innerspace-multiple-timelines.respec.mts`: Multiple independent timelines in one saga
*   `sync-innerspace-partial-update.respec.mts`: Smart diff verification (delta-only transfers)
*   `sync-innerspace-deep-updates.respec.mts`: Timelines with extensive history (past link traversal)
*   `sync-innerspace-constants.respec.mts`: Synchronization of stones (constants/non-TJPs)

**Conflict Resolution**:
*   `sync-conflict-basic-divergence.respec.mts`: Single timeline divergence and merge
*   `sync-conflict-basic-multitimelines.respec.mts`: Multiple timelines with conflicts
*   `sync-conflict-adv-multitimelines.respec.mts`: Advanced multi-timeline, multi-round sync
*   `sync-conflict-text-merge.respec.mts`: LCS-based text merging for `data.text` fields

**Identity & Security**:
*   `sync-innerspace-dest-ahead-withid.respec.mts`: Session identity test (🚧 **currently failing**)

---

## Verification Matrix

### 1. Basic Scenarios
| Feature | Description | Status | Test File |
| :--- | :--- | :--- | :--- |
| **Push** | Source syncs a new timeline to an empty Dest. | ✅ Verified | `sync-innerspace.respec.mts` |
| **Pull** | Dest requests data from Source (Reverse Sync). | ⏳ Pending | `sync-innerspace.respec.mts` |
| **Dest Ahead** | Dest has newer data; Source receives update. | ✅ Verified | `sync-innerspace-dest-ahead.respec.mts` |

### 2. Complex Graph Scenarios
| Feature | Description | Status | Test File |
| :--- | :--- | :--- | :--- |
| **Multi-Timeline** | Syncing multiple separate timelines. | ✅ Verified | `sync-innerspace-multiple-timelines.respec.mts` |
| **Deep Updates** | Syncing deep dependency chains (Ancestors). | ✅ Verified | `sync-innerspace-deep-updates.respec.mts` |
| **Smart Diff** | Sender skips data Receiver already has. | ✅ Verified | `sync-innerspace-partial-update.respec.mts` |
| **Constants** | Syncing immutable stones (no TJP). | ✅ Verified | `sync-innerspace-constants.respec.mts` |

### 3. Conflict Resolution & Edge Cases
| Feature | Description | Status | Test File |
| :--- | :--- | :--- | :--- |
| **Hash Validation** | Verifying integrity checks (ib/gib). | ✅ Verified | Built-in to all tests |
| **Basic Conflicts** | Divergent branches (single timeline). | ✅ Verified | `sync-conflict-basic-divergence.respec.mts` |
| **Multi-Timeline Conflicts** | Multiple timelines diverging. | ✅ Verified | `sync-conflict-basic-multitimelines.respec.mts` |
| **Advanced Conflicts** | Complex multi-round scenarios. | ✅ Verified | `sync-conflict-adv-multitimelines.respec.mts` |
| **Text Merge Conflicts** | LCS-based text merging. | ✅ Verified | `sync-conflict-text-merge.respec.mts` |

### 4. Security & Identity
| Feature | Description | Status | Test File |
| :--- | :--- | :--- | :--- |
| **Session Keystones** | Ephemeral identity for sync sessions. | 🚧 **Non-functional** | `sync-innerspace-dest-ahead-withid.respec.mts` |

---

## Recent Verifications

### Sync Constants (2026-01-08)
*   **Goal**: Ensure constants (ibGibs without `tjp` or `past`) are synced.
*   **Result**: ✅ Passed.
*   **Fixes**: Updated `handleInitFrame` to request missing stones; updated `handleAckFrame` to include "tip" stones in payload even without dependencies.

### Text Merge Conflicts (2026-01-14)
*   **Goal**: Verify LCS-based text merging for divergent `data.text` fields.
*   **Result**: ✅ Passed.
*   **Implementation**: Uses Longest Common Subsequence algorithm for three-way merge.

---

## Testing Guidelines

### Core Concepts

#### Rounds

Tests are organized into **rounds**. Each round represents a complete cycle of:
1. **State Setup**: Establish expected initial conditions
2. **Actions**: Perform mutations, relations, and sync operations
3. **Verification**: Assert final state expectations

The **first round** establishes common history between source and dest, and subsequent rounds perform one or more op(s) and sync(s).

#### Broad Test Scenarios

Here are the broad category of ops that can occur to timelines WRT syncing:
* **Fast-Forward (Source Ahead)**: Source has newer versions that dest lacks
* **Fast-Forward (Dest Ahead)**: Dest has newer versions that source lacks
* **Conflict**: Both source and dest have divergent edits from a common ancestor
* **No Change**: Timeline exists on both sides with no edits
* **Stones**: Stones are treated slightly differently in the sync engine right now, so these must also be tested in various permutations.

---

## Variable Naming Convention

Variable names are a COMBINATION of BOTH snake_case AND camelCase:

**Format**: `round_timeline_version_location_commentsHereInCamelCase_possiblyOtherComments`

### Components

* **Round**: `r1`, `r2`, `r3`, etc. (round number)
* **Timeline**: Greek letter (`alpha`, `beta`, `gamma`, `delta`, `epsilon`, etc.)
* **Version**: `v0`, `v1`, `v2`, etc. (from test POV, NOT necessarily matching `data.n`)
* **Location**: `source` or `dest` (where the ibgib was **created**, not where it currently exists)
* **Comments**: Optional descriptive suffixes (can chain multiple)

### Examples

```typescript
r1_alpha_v0_source                           // Round 1, alpha root, created on source
r1_alpha_v1_source_common                    // Round 1, alpha v1, common to both (pre-divergence)
r1_beta_v0_dest                              // Round 1, beta root, created on dest
r1_alpha_v2_source_sansBeta                  // Round 1, alpha v2 on source, before beta relation
r1_alpha_v3_source_rel8beta                  // Round 1, alpha v3 on source, after adding beta relation
r1_delta_v1_source_createdInR1_rel8edFromAlpha  // Round 1, delta v1 with multiple comments
r2_alpha_v4_dest_postConflict               // Round 2, alpha v4 on dest, after conflict resolution
```

---

## Test Structure

### File Organization

```typescript
await respecfully(sir, `Test Suite Name`, async () => {
    // Setup Spaces, Coordinators, Peer, etc.

    // #region Round 1 - Common History Establishment

    // domain create/edits required before our verify pre state

    const r1_alpha_v0_source_yada = await testTransformer.create({...});
    const r1_beta_v0_source_yada = await testTransformer.create({...});

    await respecfully(sir, `r1 verify pre`, async () => {
        await ifWe(sir, 'alpha exists in source and NOT exist on dest (or whatever)', async () => {
            // timeline-specific assertions
        });
        await ifWe(sir, 'beta exists in source and NOT exist on dest (or whatever)', async () => {
            // timeline-specific assertions
        });
    });

    // do the sync(s) with possible other edit(s)
    const r1_syncSaga = await senderCoordinator.sync({ ... });

    await respecfully(sir, `r1 verify post`, async () => {
        await ifWe(sir, 'alpha synced correctly', async () => {
            // timeline-specific assertions
        });
        await ifWe(sir, 'beta synced correctly', async () => {
            // timeline-specific assertions
        });
    });

    // #endregion Round 1 - Common History Establishment

    // #region Round 2 - Some divergent edits are synced

    // domain create/edits required before our verify pre state

    await respecfully(sir, `r2 verify pre`, async () => {
        await ifWe(sir, 'latest alpha only on source', async () => {
            // timeline-specific assertions
        });
        await ifWe(sir, 'beta still synced on source/dest', async () => {
            // timeline-specific assertions
        });
    });

    // do the sync(s) with possible other edit(s) which can be lengthy, so
    // encapsulate in regions as needed

    // #region r2 edits

    const r2_alpha_v1_source_rel8dToBeta = testTransformer.rel8({...});

    // #endregion r2 edits

    const r2_syncSaga = await senderCoordinator.sync({ ... });

    await respecfully(sir, `r2 verify post`, async () => {
        await ifWe(sir, 'alpha synced correctly in some specific way', async () => {
            // timeline-specific assertions
        });
        await ifWe(sir, 'beta synced correctly in some specific way', async () => {
            // timeline-specific assertions
        });
    });

    // #endregion Round 2 - Common History Establishment

});
```

### Region Usage

Use `// #region` and `// #endregion` tags to encapsulate **timeline edit groupings** within a round. This allows collapsing code sections to manage complexity.

Always include the round in a region tag, and the endregion tag should have the same exact text as the begin region tag.

Pattern examples:
* `// #region r{N} setup` - For initial timeline creation
* `// #region r{N} edits` - For mutations/relations within a round

### Verification Blocks

* **`respecfully`**: Outer verification block per round with text "r{N} verify pre/post"
* **`ifWeMight`**: Per-timeline verification within a round
  * NOTE: Graphs should be used for verification of sync, not individual addrs/ibgibs in most (all?) cases
* **`iReckon`**: Individual assertions

Example:
```typescript
await respecfully(sir, `r1 verify results`, async () => {
    await ifWe(sir, 'alpha exists on both sides', async () => {
        iReckon(sir, alphaOnSource).asTo('alpha on source').isGonnaBeTruthy();
        iReckon(sir, alphaOnDest).asTo('alpha on dest').isGonnaBeTruthy();
    });

    await ifWe(sir, 'dependency graphs match', async () => {
        const graphsMatch = graphsAreEquivalent({ graphA, graphB });
        iReckon(sir, graphsMatch).asTo('graphs equivalent').isGonnaBeTrue();
    });
});
```

---

## Multiple Syncs Per Round

A single round can have **multiple sync calls**. This is useful when:
* Creating multiple timeline pairs (some on source, some on dest)
* Testing specific sync scenarios in isolation

Key principle: Verify **final state** at end of round, not intermediate states between syncs. We don't want to get bogged down trying to test every detail after every sync. The broad coverage should handle this.

```typescript
// Round 1: Establish common history
// Sync 1: alpha and beta (created on source)
await senderCoordinator.sync({ domainIbGibs: [alpha, beta], ... });

// Sync 2: gamma and delta (created on dest, reverse direction)
await receiverCoordinator.sync({ domainIbGibs: [gamma, delta], ... });

// Single verification checks all 4 timelines
await respecfully(sir, `r1 verify all timelines synced`, async () => {
    // check alpha, beta, gamma, delta all exist on both sides
});
```

---

## Domain-Level Focus

Advanced tests focus on **domain-level correctness**, not protocol mechanics:

❌ **Don't capture**: `baseAddr`, `orphanAddr`, temporal ordering details
❌ **Don't use**: `delay()` calls to control base/orphan selection
✅ **Do focus on**: Data field correctness, relation preservation, dependency completeness

---

## Complexity Management

### Principles

1. **Self-Documenting Names**: Variable names should tell the story
2. **Region Collapsibility**: Use regions to hide complexity
3. **Round Isolation**: Each round is conceptually independent
4. **Minimal Commenting**: Rely on naming and structure over heavy comments
5. **Test Helpers**: Use helper functions to reduce boilerplate (see below)

### Test Transformer

Tests are making relatively arbitrary fork/mut8/rel8 calls. To help reduce implementation detail noise, the `TestTransformer` helper class was created.

These methods return a `TestStepInfo` which itself has references to the `ibGib` and `addr`, as well as other metadata. So the variable `r2_alpha_v2_source_arbitraryMut8` does NOT reference an ibgib, rather, it references one of these test step info objects.

```typescript
const r2_beta_v0_source = await testTransformer.create({
    ib: 'beta',
    in: 'source',
    data: { fieldA: 'beta field A created on source' },
    name: 'r2_beta_v0_source'
});
const r2_alpha_v2_source_arbitraryMut8 = await testTransformer.mut8({
    ibGib: r1_alpha_v1_source_common.ibGib, // refers to previous step.ibGib
    in: 'source',
    strField: 'fieldA',
    name: 'r2_alpha_v2_source_arbitraryMut8'
});
```

See `test-helpers.mts` and `test-types.mts` for implementations.

---

## Scenarios to Test

### Basic Scenarios
* Fast-forward (source ahead)
* Fast-forward (dest ahead)
* Simple conflict (different fields edited)
* No changes

### Advanced Scenarios
* **Divergent edits on related ibgibs**: Alpha conflicts while beta (rel8'd from alpha) also conflicts
* **One-sided new timelines**: Timeline exists only on source or dest initially
* **Transitive dependencies**: Delta rel8'd from alpha, alpha rel8'd from beta
* **Reverse dependencies**: Timeline created on dest, source adds rel8n to it
* **Removal conflicts**: Source removes rel8n while dest mut8s the rel8'd ibgib
* **Chain edits**: Mutation → rel8 → mutation again on same timeline in one round
* **Text edits**: Edit `ibGib.data.text`. If divergent, uses LCS-based merge. See `sync-conflict-text-merge.respec.mts`.

### Security Scenarios
* **Session identity creation**: Verify ephemeral keystone generation (🚧 pending implementation)
* **Signature verification**: (🚧 pending implementation)
* **Authorization checks**: Verify identity-based permissions (🚧 pending implementation)

---

## Summary

These guidelines aim to **minimize complexity** in necessarily complex integration tests through:
* Consistent naming conventions
* Round-based mental model
* Region-based code organization
* Domain-level focus
* Helper function abstraction
* Clear verification patterns

When in doubt, prioritize **readability** and **maintainability** over brevity.
