# Stages and Infrastructure Guide

## Philosophy

Stages define **environment-specific configurations** with support for complex, multi-service infrastructure. Each stage can have:

- Multiple domains (web, API, docs, admin, marketing)
- Multiple databases (primary, analytics, cache)
- Additional services (Redis, queues, storage, email, CDN)
- Background workers
- Smart defaults with override capability

---

## Basic Stage Configuration

### Minimal Configuration

```yaml
stages:
  local:
    infrastructure:
      databases:
        primary: "postgresql://localhost/myapp_dev"
```

**Smart Defaults Applied**:
- `config.log_level`: `info`
- `config.error_details`: `false`
- `config.cors.enabled`: `true`
- `config.rate_limiting.enabled`: `true` (100 req/min)

### Standard Three-Stage Setup

```yaml
stages:
  local:
    description: "Local development environment"
    infrastructure:
      domains:
        web: "localhost:3000"
        api: "localhost:3001"
      databases:
        primary: "postgresql://localhost/myapp_dev"
    config:
      log_level: debug
      development_tools: true
      error_details: true

  staging:
    description: "Staging environment for testing"
    infrastructure:
      domains:
        web: "staging.myapp.com"
        api: "api-staging.myapp.com"
      databases:
        primary: "{{ env.STAGING_DB_URL }}"
    config:
      log_level: info
      development_tools: true  # Available but requires auth
      error_details: true
    env:
      STAGING_DB_URL: "postgresql://..."

  production:
    description: "Production environment"
    infrastructure:
      domains:
        web: "myapp.com"
        api: "api.myapp.com"
      databases:
        primary: "{{ env.PROD_DB_URL }}"
    config:
      log_level: warn
      development_tools: false  # Disabled in production
      error_details: false  # Hide errors for security
      cors:
        origins: ["https://myapp.com"]
      rate_limiting:
        requests_per_minute: 1000
    env:
      PROD_DB_URL: "postgresql://..."
```

---

## Multi-Domain Infrastructure

### Web Application with Separate Services

```yaml
stages:
  production:
    infrastructure:
      domains:
        web: "app.mycompany.com"           # Main web app
        api: "api.mycompany.com"           # REST API
        docs: "docs.mycompany.com"         # Documentation site
        portal: "portal.mycompany.com"     # Customer portal
        admin: "admin.mycompany.com"       # Admin panel
        marketing: "mycompany.com"         # Marketing site

      databases:
        primary: "{{ env.PRIMARY_DB }}"
```

**Generated Infrastructure**:
```typescript
// src/config/domains.ts
export const DOMAINS = {
  web: process.env.DOMAIN_WEB || 'app.mycompany.com',
  api: process.env.DOMAIN_API || 'api.mycompany.com',
  docs: process.env.DOMAIN_DOCS || 'docs.mycompany.com',
  portal: process.env.DOMAIN_PORTAL || 'portal.mycompany.com',
  admin: process.env.DOMAIN_ADMIN || 'admin.mycompany.com',
  marketing: process.env.DOMAIN_MARKETING || 'mycompany.com',
};
```

### Custom Domains

```yaml
stages:
  production:
    infrastructure:
      domains:
        web: "app.example.com"
        api: "api.example.com"
        custom:
          webhooks: "webhooks.example.com"
          cdn: "cdn.example.com"
          status: "status.example.com"

      databases:
        primary: "{{ env.PRIMARY_DB }}"
```

---

## Multi-Database Infrastructure

### Separate Databases for Different Purposes

```yaml
stages:
  production:
    infrastructure:
      domains:
        web: "app.mycompany.com"

      databases:
        primary: "{{ env.PRIMARY_DB }}"         # Main application data
        analytics: "{{ env.ANALYTICS_DB }}"     # Analytics/reporting data
        cache: "redis://{{ env.REDIS_URL }}"    # Cache layer

      services:
        redis:
          provider: "elasticache"
          connection: "{{ env.REDIS_URL }}"
```

**Generated Database Connections**:
```typescript
// src/config/databases.ts
import { PrismaClient } from '@prisma/client';
import { createClient as createRedis } from 'redis';

export const db = new PrismaClient({
  datasources: {
    db: { url: process.env.PRIMARY_DB },
  },
});

export const analyticsDb = new PrismaClient({
  datasources: {
    db: { url: process.env.ANALYTICS_DB },
  },
});

export const cacheDb = createRedis({
  url: process.env.REDIS_URL,
});
```

### Custom Database Configurations

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"
        analytics: "{{ env.ANALYTICS_DB }}"
        custom:
          timeseries: "{{ env.TIMESERIES_DB }}"  # TimescaleDB for time-series data
          search: "{{ env.ELASTICSEARCH_URL }}"   # Elasticsearch for search
```

---

## Additional Services

### Redis for Caching and Sessions

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        redis:
          provider: "upstash"
          connection: "{{ env.UPSTASH_REDIS_URL }}"
          config:
            max_connections: 10
            ttl_default: 3600  # 1 hour default TTL
```

**Generated Service Client**:
```typescript
// src/services/cache.ts
import { Redis } from 'ioredis';

export const redis = new Redis(process.env.UPSTASH_REDIS_URL, {
  maxRetriesPerRequest: 3,
  enableReadyCheck: false,
});

// Usage
await redis.set('key', 'value', 'EX', 3600);
const value = await redis.get('key');
```

### Queue for Background Jobs

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        queue:
          provider: "bullmq"
          connection: "{{ env.REDIS_URL }}"
          config:
            concurrency: 5
            retry_attempts: 3

      workers:
        enabled: true
        concurrency: 2
        queues: ["email", "notifications", "analytics"]
```

**Generated Queue Client**:
```typescript
// src/services/queue.ts
import { Queue, Worker } from 'bullmq';

const connection = { url: process.env.REDIS_URL };

export const emailQueue = new Queue('email', { connection });
export const notificationsQueue = new Queue('notifications', { connection });

// Worker
const emailWorker = new Worker(
  'email',
  async (job) => {
    // Process email job
    await sendEmail(job.data);
  },
  { connection, concurrency: 2 }
);
```

### Storage for Files

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        storage:
          provider: "s3"
          connection: "{{ env.AWS_S3_BUCKET }}"
          config:
            region: "us-east-1"
            access_key: "{{ env.AWS_ACCESS_KEY }}"
            secret_key: "{{ env.AWS_SECRET_KEY }}"
```

**Generated Storage Client**:
```typescript
// src/services/storage.ts
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

export const s3 = new S3Client({
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY,
    secretAccessKey: process.env.AWS_SECRET_KEY,
  },
});

// Usage
export async function uploadFile(key: string, body: Buffer) {
  await s3.send(new PutObjectCommand({
    Bucket: process.env.AWS_S3_BUCKET,
    Key: key,
    Body: body,
  }));
}
```

### Email Service

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        email:
          provider: "sendgrid"
          connection: "{{ env.SENDGRID_API_KEY }}"
          config:
            from_email: "noreply@myapp.com"
            from_name: "MyApp"
```

**Generated Email Client**:
```typescript
// src/services/email.ts
import sgMail from '@sendgrid/mail';

sgMail.setApiKey(process.env.SENDGRID_API_KEY);

export async function sendEmail(to: string, subject: string, html: string) {
  await sgMail.send({
    to,
    from: {
      email: 'noreply@myapp.com',
      name: 'MyApp',
    },
    subject,
    html,
  });
}
```

### CDN Configuration

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        cdn:
          provider: "cloudflare"
          connection: "{{ env.CLOUDFLARE_ZONE }}"
          config:
            cache_ttl: 86400  # 24 hours
            min_file_size: 1024  # Cache files > 1KB
```

### Monitoring and Logging

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        monitoring:
          provider: "datadog"
          connection: "{{ env.DATADOG_API_KEY }}"
          config:
            service_name: "myapp"
            environment: "production"
            log_level: "warn"
```

---

## Background Workers

### Basic Worker Configuration

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        redis:
          provider: "upstash"
          connection: "{{ env.REDIS_URL }}"

      workers:
        enabled: true
        concurrency: 5
        queues: ["default"]
```

### Multiple Queue Workers

```yaml
stages:
  production:
    infrastructure:
      databases:
        primary: "{{ env.PRIMARY_DB }}"

      services:
        queue:
          provider: "bullmq"
          connection: "{{ env.REDIS_URL }}"

      workers:
        enabled: true
        concurrency: 10
        queues:
          - "email"           # Email sending
          - "notifications"   # Push notifications
          - "analytics"       # Analytics events
          - "reports"         # Report generation
        config:
          retry_attempts: 3
          retry_delay: 5000
```

**Generated Worker Process**:
```typescript
// src/workers/index.ts
import { Worker } from 'bullmq';

const workers = [
  new Worker('email', async (job) => {
    // Process email jobs
  }, { concurrency: 3 }),

  new Worker('notifications', async (job) => {
    // Process notification jobs
  }, { concurrency: 3 }),

  new Worker('analytics', async (job) => {
    // Process analytics jobs
  }, { concurrency: 2 }),

  new Worker('reports', async (job) => {
    // Process report jobs
  }, { concurrency: 2 }),
];
```

---

## Complete Example: SaaS Application

```yaml
stages:
  local:
    description: "Local development"
    infrastructure:
      domains:
        web: "localhost:3000"
        api: "localhost:3001"

      databases:
        primary: "postgresql://localhost/myapp_dev"

    config:
      log_level: debug
      development_tools: true
      error_details: true

  staging:
    description: "Staging environment"
    infrastructure:
      domains:
        web: "staging-app.myapp.com"
        api: "staging-api.myapp.com"
        docs: "staging-docs.myapp.com"

      databases:
        primary: "{{ env.STAGING_DB }}"
        analytics: "{{ env.STAGING_ANALYTICS_DB }}"

      services:
        redis:
          provider: "upstash"
          connection: "{{ env.STAGING_REDIS }}"

        storage:
          provider: "s3"
          connection: "{{ env.STAGING_S3_BUCKET }}"

        email:
          provider: "sendgrid"
          connection: "{{ env.SENDGRID_API_KEY }}"
          config:
            from_email: "staging@myapp.com"

      workers:
        enabled: true
        concurrency: 2
        queues: ["email", "notifications"]

    config:
      log_level: info
      development_tools: true
      error_details: true

  production:
    description: "Production environment"
    infrastructure:
      domains:
        web: "app.myapp.com"
        api: "api.myapp.com"
        docs: "docs.myapp.com"
        portal: "portal.myapp.com"
        admin: "admin.myapp.com"
        marketing: "myapp.com"
        custom:
          webhooks: "webhooks.myapp.com"
          status: "status.myapp.com"

      databases:
        primary: "{{ env.PROD_DB }}"
        analytics: "{{ env.PROD_ANALYTICS_DB }}"
        cache: "{{ env.PROD_REDIS }}"

      services:
        redis:
          provider: "elasticache"
          connection: "{{ env.PROD_REDIS }}"
          config:
            max_connections: 50

        queue:
          provider: "bullmq"
          connection: "{{ env.PROD_REDIS }}"
          config:
            concurrency: 10

        storage:
          provider: "s3"
          connection: "{{ env.PROD_S3_BUCKET }}"
          config:
            region: "us-east-1"

        search:
          provider: "algolia"
          connection: "{{ env.ALGOLIA_APP_ID }}"
          config:
            api_key: "{{ env.ALGOLIA_API_KEY }}"

        email:
          provider: "sendgrid"
          connection: "{{ env.SENDGRID_API_KEY }}"
          config:
            from_email: "noreply@myapp.com"
            from_name: "MyApp"

        cdn:
          provider: "cloudflare"
          connection: "{{ env.CLOUDFLARE_ZONE }}"

        monitoring:
          provider: "datadog"
          connection: "{{ env.DATADOG_API_KEY }}"
          config:
            service_name: "myapp"
            environment: "production"

      workers:
        enabled: true
        concurrency: 20
        queues:
          - "email"
          - "notifications"
          - "analytics"
          - "reports"
          - "exports"

    config:
      log_level: warn
      development_tools: false
      error_details: false
      cors:
        enabled: true
        origins:
          - "https://myapp.com"
          - "https://app.myapp.com"
      rate_limiting:
        enabled: true
        requests_per_minute: 1000

    env:
      PROD_DB: "postgresql://..."
      PROD_ANALYTICS_DB: "postgresql://..."
      PROD_REDIS: "redis://..."
      PROD_S3_BUCKET: "myapp-production"
      ALGOLIA_APP_ID: "..."
      ALGOLIA_API_KEY: "..."
      SENDGRID_API_KEY: "..."
      CLOUDFLARE_ZONE: "..."
      DATADOG_API_KEY: "..."
```

---

## Smart Defaults

### Default Behaviors by Stage

Bangstack applies sensible defaults based on stage name:

**Local Stage** (default overrides):
```yaml
config:
  log_level: debug
  development_tools: true
  error_details: true
  cors:
    origins: ["*"]
```

**Staging Stage** (default overrides):
```yaml
config:
  log_level: info
  development_tools: true  # Enabled but requires auth
  error_details: true
```

**Production Stage** (default overrides):
```yaml
config:
  log_level: warn
  development_tools: false
  error_details: false  # Hide for security
  cors:
    origins: []  # Must specify explicitly
```

### Override Smart Defaults

```yaml
stages:
  local:
    infrastructure:
      databases:
        primary: "postgresql://localhost/myapp"

    # Override local defaults
    config:
      log_level: warn  # Less verbose logging
      error_details: false  # Hide errors even in local
```

---

## Environment Variables

### Using Environment Variables

```yaml
stages:
  production:
    infrastructure:
      domains:
        web: "{{ env.DOMAIN_WEB }}"
        api: "{{ env.DOMAIN_API }}"

      databases:
        primary: "{{ env.DATABASE_URL }}"

      services:
        redis:
          connection: "{{ env.REDIS_URL }}"

    env:
      DOMAIN_WEB: "app.mycompany.com"
      DOMAIN_API: "api.mycompany.com"
      DATABASE_URL: "postgresql://..."
      REDIS_URL: "redis://..."
```

**Generated .env.production**:
```bash
DOMAIN_WEB=app.mycompany.com
DOMAIN_API=api.mycompany.com
DATABASE_URL=postgresql://...
REDIS_URL=redis://...
```

---

## Summary

**Infrastructure Components**:
- ✅ Multiple domains (web, api, docs, portal, admin, marketing, custom)
- ✅ Multiple databases (primary, analytics, cache, custom)
- ✅ Additional services (redis, queue, storage, search, email, cdn, monitoring)
- ✅ Background workers with queue configuration

**Smart Configuration**:
- ✅ Smart defaults by stage (local, staging, production)
- ✅ Override capability for all defaults
- ✅ Environment variable templating
- ✅ Per-stage config (log levels, CORS, rate limiting)

**Generated Code**:
- Database clients for all databases
- Service clients for all services
- Worker processes for background jobs
- Environment-specific configuration

This system supports **simple local development** while scaling to **complex multi-service production infrastructure**.
