# CancellationToken Implementation - Manual Testing Instructions

#---------------------------------------
IMPORTANT! IMPORTANT! IMPORTANT!
If you want to use this you must first copy it over to a loaded area. This default place is 
just for future use and will note be connected to the coade base as the package file is in a higher folder.
#---------------------------------------

## Files Created

1. **Test Files**:
   - `Assets/Tests/CancellationTokenTests.cs` (main cancellation tests)
   - `Assets/Tests/BatchCancellationTests.cs` (batch execution tests)
   - `Assets/Tests/SetupTeardownCancellationTests.cs` (setup/teardown lifecycle tests)
2. **Assembly Definition**: `Assets/Tests/CancellationTokenTests.asmdef`
3. **This Document**: `TESTING_INSTRUCTIONS.md`

## Pre-Testing Setup

### Step 1: Open Unity
1. Open the CADcog project in Unity 6000.2
2. Wait for Unity to compile the new test files

### Step 2: Open MooseRunner Window
1. In Unity, go to `Window > MooseRunner` (or wherever the MooseRunner window menu is)
2. The MooseRunner window should open showing your test assemblies

### Step 3: Locate the Tests
1. In MooseRunner, look for the `CancellationTokenTests` assembly
2. Expand it to see the test classes:
   - `CancellationTokenTests` (main tests - in CancellationTokenTests.cs)
   - `BatchCancellationTests` (batch cancellation tests - in BatchCancellationTests.cs)
   - `SetupTeardownCancellationTests` (setup/teardown tests - in SetupTeardownCancellationTests.cs)

---

## Manual Testing Checklist

### ✅ Test 1: Basic Task Cancellation

**What to do:**
1. Select `Test1_BasicTaskCancellation_ShouldStopImmediately`
2. Click **Run** button in MooseRunner
3. Observe console message: `[Test1] Test started - will delay for 10 seconds`
4. **Within 1-2 seconds, click the STOP button**
5. Observe the results

**Expected Results:**
- ✅ Test stops immediately (doesn't wait 10 seconds)
- ✅ Console shows: `Test Test1_BasicTaskCancellation_ShouldStopImmediately was cancelled`
- ✅ Console shows: `[Test1] SUCCESS: Cancellation was properly caught by test method`
- ✅ Test status shows "Failed" (because cancelled tests currently marked as failed)
- ❌ **FAIL IF**: Test continues for 10 seconds

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 2: Cooperative Loop Cancellation

**What to do:**
1. Select `Test2_CooperativeLoop_ShouldExitEarly`
2. Click **Run**
3. Watch console iterations: `[Test2] Iteration 0`, `[Test2] Iteration 1`, etc.
4. **After ~5 iterations, click STOP**

**Expected Results:**
- ✅ Loop exits immediately
- ✅ Remaining 95 iterations don't execute
- ✅ Console shows cancellation message
- ❌ **FAIL IF**: All 100 iterations complete

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 3: Multiple Operations Cancellation

**What to do:**
1. Select `Test3_MultipleOperations_AllCancelTogether`
2. Click **Run**
3. Observe console: `[Test3] Starting 3 parallel 5-second delays`
4. **Within 1-2 seconds, click STOP**

**Expected Results:**
- ✅ All three operations cancel together
- ✅ Test stops immediately
- ✅ Console shows cancellation message
- ❌ **FAIL IF**: Test waits 5 seconds

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 4: UniTask Cancellation (if UNITASK_PRESENT)

**What to do:**
1. Check if UniTask is installed in your project
2. If not installed, skip this test
3. If installed, select `Test4_UniTaskCancellation_ShouldStopImmediately`
4. Click **Run**
5. **Within 1-2 seconds, click STOP**

**Expected Results:**
- ✅ UniTask cancels immediately
- ✅ Console shows: `[Test4] SUCCESS: UniTask cancellation was properly caught`
- ✅ Same behavior as Test 1 but for UniTask
- ❌ **FAIL IF**: Test runs for 10 seconds

**Status:** [ ] PASS [ ] FAIL [ ] SKIPPED (no UniTask)

**Notes:**
_______________________________________

---

### ✅ Test 5: Backward Compatibility (Func<bool>)

**What to do:**
1. Select `Test5_BackwardCompatibility_FuncBoolStillWorks`
2. Click **Run**
3. Observe iterations in console
4. **After 2-3 iterations, click STOP**

**Expected Results:**
- ✅ Test stops using legacy mechanism
- ✅ Console shows: `[Test5] SUCCESS: Stopped via Func<bool> at iteration X`
- ✅ No errors or warnings
- ✅ Proves old pattern still works
- ❌ **FAIL IF**: Stop button doesn't work

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 6: No Parameters Test

**What to do:**
1. Select `Test6_NoParameters_RunsNormally`
2. Click **Run**
3. **DO NOT click STOP** - let it complete
4. Observe it runs for 2 seconds and completes

**Expected Results:**
- ✅ Test completes normally
- ✅ Console shows: `[Test6] Test completed successfully`
- ✅ Test passes (green)
- ⚠️ Note: Stop button won't work for tests without cancellation parameters

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 7: Long Running Test

**What to do:**
1. Select `Test7_LongRunning_CancellableAnytime`
2. Click **Run**
3. Wait for random number of seconds (e.g., 7 seconds)
4. Click **STOP**

**Expected Results:**
- ✅ Test stops immediately at any point
- ✅ Doesn't complete the full 30 seconds
- ✅ Console shows cancellation message
- ❌ **FAIL IF**: Test continues after clicking Stop

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 8: Timeout Attribute

**What to do:**
1. Select `Test8_Timeout_ShouldFailAfter2Seconds`
2. Click **Run**
3. **DO NOT click STOP** - let the timeout trigger automatically
4. Observe the test behavior

**Expected Results:**
- ✅ Test starts and logs: `[Test8] Starting test with 2-second timeout, but will try to delay for 10 seconds`
- ✅ Test automatically FAILS after 2 seconds (doesn't wait 10 seconds)
- ✅ Console shows: `Test Test8_Timeout_ShouldFailAfter2Seconds timed out`
- ✅ Console shows: `[Test8] Task was cancelled - timeout triggered!`
- ✅ **Console ERROR shows: `Test Test8_Timeout_ShouldFailAfter2Seconds did timeout`** (red error in console)
- ✅ Test status shows "Failed" (red)
- ❌ **FAIL IF**: Test runs for more than 3 seconds
- ❌ **FAIL IF**: Test completes successfully (green)

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 9: Batch Cancellation

**What to do:**
1. Select ALL THREE tests in `BatchCancellationTests`:
   - `BatchTest1_TwoSeconds`
   - `BatchTest2_TwoSeconds`
   - `BatchTest3_TwoSeconds`
2. Click **Run All** (or select batch run)
3. Wait for `BatchTest1` to complete (~2 seconds)
4. **During `BatchTest2`, click STOP**

**Expected Results:**
- ✅ BatchTest1 completes and shows "Passed" (green)
- ✅ BatchTest2 cancels immediately and shows "Failed" (red)
- ✅ BatchTest3 NEVER starts and stays "NotRun" (gray)
- ✅ Console shows: `[Batch1] Completed`, then `[Batch2] Starting`, then cancellation message
- ✅ Console shows: `Test BatchTest2_TwoSeconds was stopped and marked as Failed`
- ❌ **FAIL IF**: BatchTest3 starts or shows anything other than gray/NotRun
- ❌ **FAIL IF**: BatchTest3 shows "Starting" in console

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 10: Setup Cancellation

**What to do:**
1. Select `SetupTeardownCancellationTests > TestMethod`
2. Click **Run**
3. Observe console: `[Setup] Setup started - 5 second delay`
4. **During setup (within 2 seconds), click STOP**

**Expected Results:**
- ✅ Setup cancels immediately (doesn't wait 5 seconds)
- ✅ Console shows: `[Setup] Setup started - 5 second delay`
- ✅ Console does NOT show: `[Setup] Setup completed`
- ✅ **Test method NEVER runs** - skips directly to TearDown
- ✅ Console does NOT show: `[TestMethod] Test is running`
- ✅ TearDown runs for cleanup (may show `[Teardown] Teardown started`)
- ✅ Test marked as "Failed" (red) - cancelled during setup
- ❌ **FAIL IF**: Console shows `[TestMethod] Test is running` (test should be skipped)
- ❌ **FAIL IF**: Setup runs for full 5 seconds

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 11: OneTimeSetUp Cancellation

**What to do:**
1. Select `SetupTeardownCancellationTests > TestMethod`
2. Click **Run**
3. Observe console: `[OneTimeSetUp] OneTimeSetUp started - 5 second delay`
4. **During OneTimeSetUp (within 2 seconds), click STOP**

**Expected Results:**
- ✅ OneTimeSetUp cancels immediately (doesn't wait 5 seconds)
- ✅ Console shows: `[OneTimeSetUp] OneTimeSetUp started - 5 second delay`
- ✅ Console does NOT show: `[OneTimeSetUp] OneTimeSetUp completed`
- ✅ Console DOES show: `[OneTimeTearDown] OneTimeTearDown started` **immediately**
- ✅ Console DOES show: `[OneTimeTearDown] OneTimeTearDown completed`
- ✅ Console does NOT show: `[Setup] Setup started` (Setup skipped)
- ✅ Console does NOT show: `[TestMethod] Test is running` (Test skipped)
- ✅ Console does NOT show: `[Teardown] Teardown started` (TearDown skipped - Setup never ran)
- ✅ Test marked as "Failed" (red) - cancelled during OneTimeSetUp
- ❌ **FAIL IF**: Setup, Test, or TearDown execute after OneTimeSetUp cancelled
- ❌ **FAIL IF**: OneTimeTearDown doesn't run
- ❌ **FAIL IF**: TearDown executes (should be skipped because Setup never ran)

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 12: OneTimeTearDown Execution After Failure

**What to do:**
1. Run Test 11 first (cancel during OneTimeSetUp)
2. Verify that OneTimeTearDown ran during the same execution (not when switching classes)
3. Observe the console messages

**Expected Results:**
- ✅ Console shows `[OneTimeTearDown] OneTimeTearDown started` **during the same execution** (not on class switch)
- ✅ Console shows `[OneTimeTearDown] OneTimeTearDown completed`
- ✅ Console does NOT show `[Teardown] Teardown started` (TearDown skipped)
- ✅ OneTimeTearDown executes even though OneTimeSetUp failed
- ✅ Execution flow: OneTimeSetUp (cancelled) → OneTimeTearDown → Abort
- ❌ **FAIL IF**: OneTimeTearDown doesn't run immediately
- ❌ **FAIL IF**: OneTimeTearDown waits for class switch instead of running immediately
- ❌ **FAIL IF**: TearDown executes (should be skipped)

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

### ✅ Test 13: Re-run After OneTimeSetUp Failure

**What to do:**
1. Run `SetupTeardownCancellationTests > TestMethod`
2. Click STOP during OneTimeSetUp
3. Wait for execution to complete
4. Run the SAME test again (`SetupTeardownCancellationTests > TestMethod`)
5. **This time, let it complete without clicking STOP**
6. Observe console

**Expected Results:**
- ✅ Console shows `[OneTimeSetUp] OneTimeSetUp started` **again** on the second run
- ✅ OneTimeSetUp runs from scratch (doesn't skip because state was reset)
- ✅ If not cancelled, console shows: `[OneTimeSetUp] OneTimeSetUp completed`
- ✅ If not cancelled, test proceeds normally: Setup → Test → TearDown
- ✅ Test completes successfully (green) on second run if not cancelled
- ❌ **FAIL IF**: OneTimeSetUp is skipped on second run (would indicate state wasn't reset)
- ❌ **FAIL IF**: Console shows error about OneTimeSetUp already having run

**Status:** [ ] PASS [ ] FAIL

**Notes:**
_______________________________________

---

## Post-Testing Validation

### Check for Memory Leaks
1. Run several tests with cancellation
2. Check Unity Profiler for memory allocation
3. Verify `CancellationTokenSource` objects are being disposed

**Status:** [ ] NO LEAKS [ ] LEAKS DETECTED

**Notes:**
_______________________________________

### Check Console for Errors
1. Review all console messages
2. Look for any unexpected errors
3. Verify no stack traces from our changes

**Status:** [ ] NO ERRORS [ ] ERRORS FOUND

**Notes:**
_______________________________________

---

## Known Limitations (Expected Behavior)

1. **Cancelled tests show as "Failed"** - This is expected. We're using the existing `MethodTestStatus.Failed` for cancelled tests. In the future, we can add a "Skipped" status.

2. **Tests without parameters cannot be cancelled** - This is expected. Only tests with `CancellationToken` or `Func<bool>` parameters can be stopped.

3. **Coroutines still use old mechanism** - This is expected. IEnumerator tests continue to use the `_stopTestNow` flag and `StopCoroutine()`.

## Important Fix Applied

**ISSUE FIXED**: Previously, Test5 (Func<bool>) and coroutine tests would be marked as **Passed** when stopped because they exited cleanly without throwing exceptions.

**FIX**: Now ALL tests (CancellationToken, Func<bool>, and coroutines) are properly marked as **Failed** when the Stop button is clicked. This happens in `TestMethodExecutor.StopExecution()` which now tracks and marks the currently executing test as Failed.

**What This Means**: Test5 should now correctly show as "Failed" when you click Stop, just like Tests 1-4.

## Important Fix #2: Batch Test Blocking

**ISSUE FIXED**: Previously, when running batch tests and clicking Stop during Test2, Test3 would still start after Test2 completed.

**ROOT CAUSE**:
1. Race condition in `TestExecutorAPI.StopTestExecution()` - batch flag was set AFTER method executor stop, allowing callbacks to fire first
2. Missing stop checks in recursive callbacks - callbacks would start the next test without checking if stop was requested

**FIXES APPLIED**:
1. **Reordered stop calls** in `TestExecutorAPI.cs`: Batch flag now set BEFORE stopping method executor
2. **Added stop checks** in `TestBatchRunner.cs`: All 4 recursive callbacks now check `_stopTestNow` before starting next test

**What This Means**:
- When you stop during Test2, Test2 will be marked as Failed (red)
- Test3 will NOT start - it stays gray (NotRun)
- This applies to all batch execution scenarios (methods, classes, assemblies)

---

## Important Fix #3: OneTimeTearDown Execution After OneTimeSetUp Failure

**ISSUE FIXED**: Previously, when OneTimeSetUp was cancelled or failed, OneTimeTearDown would never run, causing resource leaks.

**ROOT CAUSE**:
1. When OneTimeSetUp failed, code would set `_lastTestClassRun = null` immediately
2. This prevented OneTimeTearDown from executing (check requires `_lastTestClassRun != null`)
3. But setting to null is NECESSARY - otherwise re-running the test would skip OneTimeSetUp (thinking it already succeeded)
4. TearDown would run even though Setup never ran (nothing to tear down)

**FIX APPLIED**:
**File**: `TestLifecycleRunner.cs` (lines 118-126)

**New behavior when OneTimeSetUp fails**:
1. Run **OneTimeTearDown immediately** for cleanup (before resetting state)
2. Then set `_lastTestClassRun = null` to reset state
3. Then call `onComplete()` to abort execution
4. **Skip Setup, Test, AND TearDown** entirely (Setup never ran, nothing to tear down)

**What This Means**:
- When OneTimeSetUp is cancelled, OneTimeTearDown runs immediately for resource cleanup
- TearDown is now correctly skipped (Setup never ran)
- State is reset, allowing OneTimeSetUp to run again on next execution
- No resource leaks from partial OneTimeSetUp execution
- Correct lifecycle: OneTimeSetUp (failed) → OneTimeTearDown → Reset → Abort

---

## Troubleshooting

### Tests don't appear in MooseRunner
- **Solution**: Check Unity Console for compilation errors
- **Solution**: Verify `CancellationTokenTests.asmdef` was created correctly
- **Solution**: Reimport the Assets/Tests folder

### Stop button doesn't work
- **Solution**: Verify you're in Play Mode (MooseRunner only supports stopping in Play Mode)
- **Solution**: Check that test has `CancellationToken` parameter
- **Solution**: Review TestMethodExecutor.cs changes are correct

### Compilation errors
- **Solution**: Verify `using System.Threading;` was added to TestMethodExecutor.cs
- **Solution**: Check Unity version is 6000.2 or compatible
- **Solution**: Review all edits were applied correctly

---

## Summary Checklist

After completing all tests, verify:

- [ ] All CancellationToken tests can be cancelled immediately
- [ ] Timeout attribute automatically cancels tests after specified duration
- [ ] Backward compatibility maintained (Func<bool> still works)
- [ ] Batch cancellation stops current test and skips remaining
- [ ] Setup/Teardown cancellation works correctly
- [ ] OneTimeSetUp cancellation triggers OneTimeTearDown immediately
- [ ] OneTimeTearDown runs even when OneTimeSetUp fails
- [ ] TearDown is skipped when OneTimeSetUp fails (Setup never ran)
- [ ] Re-running test after OneTimeSetUp failure runs OneTimeSetUp again (state reset)
- [ ] No memory leaks detected
- [ ] No unexpected errors in console
- [ ] Tests without parameters still run normally

---

## Next Steps

Once manual testing is complete:

1. **Document Results**: Fill in all the [ ] PASS/FAIL checkboxes above
2. **Report Issues**: If any tests fail, document exactly what happened
3. **Review Implementation**: If everything passes, the implementation is validated
4. **Consider Future Enhancements**:
   - Add `MethodTestStatus.Skipped` enum value for cancelled tests
   - Add timeout support via linked CancellationTokenSource
   - Update UI to show cancellation vs failure distinctly

---

## Questions?

If you encounter any issues or have questions about the tests:
1. Check the test code comments in `CancellationTokenTests.cs`
2. Review `plan.md` for detailed implementation details
3. Check Unity console for error messages
