# E2E Testing Guide

This guide explains Bangstack's end-to-end (E2E) testing infrastructure introduced in v0.4.3, which ensures generated code is actually written to disk and structured correctly.

## Table of Contents
1. [Overview](#overview)
2. [Why E2E Tests?](#why-e2e-tests)
3. [Test Infrastructure](#test-infrastructure)
4. [Running E2E Tests](#running-e2e-tests)
5. [Writing New E2E Tests](#writing-new-e2e-tests)
6. [Testing Best Practices](#testing-best-practices)
7. [Troubleshooting](#troubleshooting)

## Overview

Bangstack's E2E tests verify the entire code generation pipeline:
1. Parse workflow YAML
2. Generate code through the orchestrator
3. Write files to disk
4. Verify file existence and content

These tests catch critical bugs that unit tests miss, such as:
- Files not being written to disk (like the v0.4.1 bug)
- Incorrect file paths or permissions
- Missing dependencies in generated code
- Integration issues between generators

## Why E2E Tests?

### The v0.4.1 Bug Story

In v0.4.1, all unit tests passed, but generated files weren't being written to disk. The issue was that generators returned relative paths instead of absolute paths, and the file writer silently failed.

**Unit tests** checked that generators returned the correct content, but didn't verify files were actually written.

**E2E tests** would have caught this immediately by checking `fs.existsSync()` on generated files.

### What E2E Tests Verify

- ✅ Files are actually written to disk
- ✅ All file paths are absolute (not relative)
- ✅ File content matches generator output
- ✅ Directory structure is correct
- ✅ Generated code compiles without errors
- ✅ Integration between multiple generators works

## Test Infrastructure

### Test Structure

E2E tests are located in `src/e2e/`:

```
src/e2e/
├── file-generation-smoke.test.ts     # Smoke test for basic file generation
├── database-config.e2e.test.ts       # Database configuration tests
├── auth-config.e2e.test.ts           # Authentication tests
└── event-driven.e2e.test.ts          # Event-driven architecture tests
```

### Smoke Test Pattern

The file generation smoke test verifies the basics work:

```typescript
import * as fs from 'fs';
import * as path from 'path';
import { GenerationOrchestrator } from '../generators/generation-orchestrator';
import { Workflow } from '../types/workflow';

describe('E2E: File Generation Smoke Test', () => {
  const testOutputDir = '/tmp/bangstack-e2e-test';
  let orchestrator: GenerationOrchestrator;

  beforeEach(() => {
    // Clean up test directory before each test
    if (fs.existsSync(testOutputDir)) {
      fs.rmSync(testOutputDir, { recursive: true, force: true });
    }

    orchestrator = new GenerationOrchestrator();
  });

  afterEach(() => {
    // Clean up test directory after each test
    if (fs.existsSync(testOutputDir)) {
      fs.rmSync(testOutputDir, { recursive: true, force: true });
    }
  });

  it('should actually write files to disk with absolute paths', async () => {
    const minimalWorkflow: Workflow = {
      workflow: {
        id: 'e2e-smoke-test',
        version: '1.0.0',
        name: 'E2E Smoke Test',
      },
      stack: {
        api: {
          type: 'backend',
          technology: 'typescript/express',
          config: { port: 3000 },
        },
      },
      stages: {
        local: {
          infrastructure: {
            databases: {},
            services: {},
          },
        },
      },
    };

    // Generate code
    const result = await orchestrator.generate(minimalWorkflow, {
      outputDirectory: testOutputDir,
      dryRun: false,
      verbose: false,
    });

    // Verify files were actually generated
    expect(result.files.length).toBeGreaterThan(0);

    // Verify all returned paths are absolute
    result.files.forEach(file => {
      expect(path.isAbsolute(file.path)).toBe(true);
    });

    // Verify files actually exist on disk
    const filesOnDisk: string[] = [];
    result.files.forEach(file => {
      if (fs.existsSync(file.path)) {
        filesOnDisk.push(file.path);
      }
    });

    // At least some files should exist (this catches the v0.4.1 bug)
    expect(filesOnDisk.length).toBeGreaterThan(0);

    // Verify the files are in the correct directory
    filesOnDisk.forEach(filePath => {
      expect(filePath.startsWith(testOutputDir)).toBe(true);
    });

    // Verify files written count matches
    expect(result.written.length).toBeGreaterThan(0);
  });

  it('should write files with content matching returned data', async () => {
    const workflow: Workflow = {
      // ... workflow definition
    };

    const result = await orchestrator.generate(workflow, {
      outputDirectory: testOutputDir,
      dryRun: false,
      verbose: false,
    });

    // Verify at least one file was written
    expect(result.written.length).toBeGreaterThan(0);

    // Pick the first file and verify content matches
    const firstFile = result.files[0];
    if (fs.existsSync(firstFile.path)) {
      const fileContent = fs.readFileSync(firstFile.path, 'utf-8');
      expect(fileContent).toBe(firstFile.content);
    }
  });
});
```

### Feature-Specific Tests

Feature tests verify specific generator functionality:

```typescript
// database-config.e2e.test.ts
it('should generate backend with database configuration', async () => {
  const workflow: Workflow = {
    workflow: {
      id: 'db-test',
      version: '1.0.0',
      name: 'Database Test',
    },
    stack: {
      api: {
        type: 'backend',
        technology: 'typescript/express',
        config: { port: 3000 },
      },
    },
    stages: {
      local: {
        infrastructure: {
          databases: {
            main: {
              type: 'postgresql',
              config: { host: 'localhost', port: 5432 },
            },
          },
          services: {},
        },
      },
    },
  };

  const result = await orchestrator.generate(workflow, {
    outputDirectory: testOutputDir,
    dryRun: false,
    verbose: false,
  });

  // Verify files were generated
  expect(result.files.length).toBeGreaterThan(0);
  expect(result.written.length).toBeGreaterThan(0);

  // Verify files are in the correct directory
  result.files.forEach(file => {
    expect(file.path.startsWith(testOutputDir)).toBe(true);
  });
});
```

## Running E2E Tests

### Run All E2E Tests

```bash
# Run all E2E tests
npm test -- src/e2e

# Run with verbose output
npm test -- src/e2e --verbose
```

### Run Specific E2E Test

```bash
# Run smoke tests only
npm test -- src/e2e/file-generation-smoke.test.ts

# Run database tests only
npm test -- src/e2e/database-config.e2e.test.ts

# Run auth tests only
npm test -- src/e2e/auth-config.e2e.test.ts

# Run event-driven tests only
npm test -- src/e2e/event-driven.e2e.test.ts
```

### Run in Watch Mode

```bash
# Watch mode for rapid development
npm test -- src/e2e --watch
```

### CI/CD Integration

E2E tests run automatically in the prepublishOnly script:

```json
{
  "scripts": {
    "prepublishOnly": "npm run lint && npm test && npm run build"
  }
}
```

This ensures:
1. Code passes linting
2. All tests (including E2E) pass
3. TypeScript compiles successfully

## Writing New E2E Tests

### Step 1: Create Test File

Create a new test file in `src/e2e/`:

```typescript
// src/e2e/my-feature.e2e.test.ts
import * as fs from 'fs';
import * as path from 'path';
import { GenerationOrchestrator } from '../generators/generation-orchestrator';
import { Workflow } from '../types/workflow';

describe('E2E: My Feature', () => {
  const testOutputDir = '/tmp/bangstack-e2e-my-feature';
  let orchestrator: GenerationOrchestrator;

  beforeEach(() => {
    if (fs.existsSync(testOutputDir)) {
      fs.rmSync(testOutputDir, { recursive: true, force: true });
    }
    orchestrator = new GenerationOrchestrator();
  });

  afterEach(() => {
    if (fs.existsSync(testOutputDir)) {
      fs.rmSync(testOutputDir, { recursive: true, force: true });
    }
  });

  // Your tests here
});
```

### Step 2: Define Test Workflow

Create a workflow that uses your feature:

```typescript
it('should generate my feature', async () => {
  const workflow: Workflow = {
    workflow: {
      id: 'my-feature-test',
      version: '1.0.0',
      name: 'My Feature Test',
    },
    stack: {
      api: {
        type: 'backend',
        technology: 'typescript/express',
        config: { port: 3000 },
      },
    },
    stages: {
      local: {
        infrastructure: {
          databases: {},
          services: {},
        },
      },
    },
  };

  const result = await orchestrator.generate(workflow, {
    outputDirectory: testOutputDir,
    dryRun: false,
    verbose: false,
  });

  // Add assertions
});
```

### Step 3: Add Assertions

Verify files are generated correctly:

```typescript
// Verify files were generated
expect(result.files.length).toBeGreaterThan(0);
expect(result.written.length).toBeGreaterThan(0);

// Verify specific file exists
const myFilePath = path.join(testOutputDir, 'backend', 'src', 'my-file.ts');
expect(fs.existsSync(myFilePath)).toBe(true);

// Verify file content
const myFileContent = fs.readFileSync(myFilePath, 'utf-8');
expect(myFileContent).toContain('expected content');
expect(myFileContent).toContain('import { Something }');
expect(myFileContent).toContain('export function myFunction');

// Verify file paths are absolute
result.files.forEach(file => {
  expect(path.isAbsolute(file.path)).toBe(true);
});

// Verify files are in correct directory
result.files.forEach(file => {
  expect(file.path.startsWith(testOutputDir)).toBe(true);
});
```

### Step 4: Test Edge Cases

Test error conditions and edge cases:

```typescript
it('should handle missing configuration', async () => {
  const workflow: Workflow = {
    // ... workflow with missing config
  };

  await expect(
    orchestrator.generate(workflow, {
      outputDirectory: testOutputDir,
      dryRun: false,
      verbose: false,
    })
  ).rejects.toThrow('Expected error message');
});
```

## Testing Best Practices

### 1. Use Isolated Test Directories

Always use temporary directories to avoid conflicts:

```typescript
// Good - isolated directory
const testOutputDir = '/tmp/bangstack-e2e-my-test';

// Bad - shared directory
const testOutputDir = './generated';
```

### 2. Clean Up Before and After

Ensure clean state for each test:

```typescript
beforeEach(() => {
  if (fs.existsSync(testOutputDir)) {
    fs.rmSync(testOutputDir, { recursive: true, force: true });
  }
});

afterEach(() => {
  if (fs.existsSync(testOutputDir)) {
    fs.rmSync(testOutputDir, { recursive: true, force: true });
  }
});
```

### 3. Test File Existence First

Check file exists before reading content:

```typescript
// Good
if (fs.existsSync(filePath)) {
  const content = fs.readFileSync(filePath, 'utf-8');
  expect(content).toContain('something');
}

// Bad - will throw if file doesn't exist
const content = fs.readFileSync(filePath, 'utf-8');
```

### 4. Verify Absolute Paths

Always verify paths are absolute to catch v0.4.1-style bugs:

```typescript
result.files.forEach(file => {
  expect(path.isAbsolute(file.path)).toBe(true);
});
```

### 5. Test Minimal Workflows

Start with minimal workflows to isolate issues:

```typescript
// Minimal workflow for testing
const minimalWorkflow: Workflow = {
  workflow: {
    id: 'test',
    version: '1.0.0',
    name: 'Test',
  },
  stack: {
    api: {
      type: 'backend',
      technology: 'typescript/express',
      config: { port: 3000 },
    },
  },
  stages: {
    local: {
      infrastructure: {
        databases: {},
        services: {},
      },
    },
  },
};
```

### 6. Use Descriptive Test Names

Test names should describe what's being tested:

```typescript
// Good
it('should generate Prisma schema with PostgreSQL configuration', async () => {
  // ...
});

// Bad
it('should work', async () => {
  // ...
});
```

## Troubleshooting

### Test Failures

**Problem:** E2E test fails with "file not found"

**Solution:**
1. Check if files were generated:
```typescript
console.log('Files generated:', result.files.length);
console.log('Files written:', result.written.length);
console.log('First file:', result.files[0]);
```

2. Verify output directory:
```typescript
console.log('Output directory:', testOutputDir);
console.log('Directory exists:', fs.existsSync(testOutputDir));
console.log('Directory contents:', fs.readdirSync(testOutputDir));
```

### Path Issues

**Problem:** Paths are relative instead of absolute

**Solution:**
Check generator implementation ensures absolute paths:

```typescript
// In your generator
const absolutePath = path.resolve(outputDirectory, relativePath);
return {
  files: [{
    path: absolutePath,  // Must be absolute
    content: '...',
  }],
};
```

### Cleanup Issues

**Problem:** Test directory not cleaning up

**Solution:**
Add force flag to rmSync:

```typescript
if (fs.existsSync(testOutputDir)) {
  fs.rmSync(testOutputDir, {
    recursive: true,
    force: true,  // Important!
  });
}
```

### Permission Errors

**Problem:** EACCES or EPERM errors

**Solution:**
1. Use `/tmp` directory which is always writable
2. Check directory permissions
3. Ensure no files are open/locked

## Future Enhancements

Planned improvements for E2E testing:

1. **Compilation Tests**: Verify generated TypeScript compiles
2. **Runtime Tests**: Start generated servers and test endpoints
3. **Integration Tests**: Test generated code with real databases
4. **Performance Tests**: Measure generation time for large workflows
5. **Snapshot Tests**: Compare generated output with snapshots

## Summary

E2E tests are critical for ensuring Bangstack generates working code. They:

- ✅ Catch bugs that unit tests miss
- ✅ Verify the entire generation pipeline
- ✅ Ensure files are written correctly to disk
- ✅ Provide confidence for releases

Always run E2E tests before publishing:

```bash
npm run prepublishOnly  # Runs lint + test + build
```

## Additional Resources

- [Test Coverage Report](../README.md#-testing)
- [Contributing Guidelines](../CONTRIBUTING.md)
- [Generator Framework](./GENERATOR-FRAMEWORK.md)
- [API Reference](./API-REFERENCE.md)

---

**Questions?** Open an issue on [GitHub](https://github.com/mogullabs/bangstack/issues) or check the [discussions](https://github.com/mogullabs/bangstack/discussions).
