# NexusCLI Pipeline Integration

**Version**: 0.1.0
**Created**: 2025-11-17
**Based on**: NexusChat development workflow (CLAUDE.md)

---

## 🔄 Pipeline Overview

NexusCLI follows a **5-phase pipeline** for distributed CLI orchestration, adapted from the NexusChat development workflow:

```
Analysis → Coordination → Execution → Test → Deploy
```

Each phase has specific responsibilities and integrates with the Control Plane and Node Wrappers.

---

## 📋 Phase Breakdown

### Phase 1: Analysis

**Purpose**: Understand what needs to be done, where, and validate feasibility.

**Control Plane Responsibilities**:

1. **Request Validation**
   - Parse job specification (tool, command, target nodes)
   - Validate command syntax
   - Check tool availability on target nodes
   - Verify user permissions

2. **Resource Assessment**
   - Query Node Registry for available nodes
   - Check node capacity (active jobs vs max concurrent)
   - Estimate execution time and resource requirements

3. **Risk Analysis**
   - Identify destructive commands (rm, mkfs, dd, etc.)
   - Flag production nodes for approval workflows
   - Check command whitelists/blacklists

**API Endpoints**:
```
POST /api/v1/jobs/validate
```

**Example**:
```json
// Request
{
  "tool": "bash",
  "command": "rm -rf /var/www/*",
  "nodeIds": ["prod-001"]
}

// Response
{
  "valid": false,
  "warnings": [
    {
      "type": "destructive_command",
      "message": "Command contains 'rm -rf' - requires approval",
      "severity": "critical"
    }
  ],
  "suggestions": [
    "Use 'rm -rf /var/www/tmp/*' for safer scoped deletion",
    "Add --dry-run flag to test first"
  ]
}
```

---

### Phase 2: Coordination

**Purpose**: Plan execution strategy, select nodes, queue jobs.

**Control Plane Responsibilities**:

1. **Node Selection**
   - Choose optimal node(s) based on:
     - Current load
     - Tool availability
     - Geographic location (latency)
     - Cost (if WellaNet integration)

2. **Job Scheduling**
   - Queue jobs if nodes are busy
   - Prioritize based on user roles
   - Schedule batch jobs with dependencies

3. **Session Initialization**
   - Create job records in storage
   - Generate unique jobIds
   - Set up SSE stream endpoints

**Algorithm** (Node Selection):
```javascript
function selectNode(jobSpec, availableNodes) {
  // 1. Filter by tool availability
  const capableNodes = availableNodes.filter(node =>
    node.capabilities.tools.includes(jobSpec.tool)
  );

  // 2. Filter by load threshold
  const readyNodes = capableNodes.filter(node =>
    node.activeJobs < node.capabilities.maxConcurrentJobs
  );

  // 3. Sort by load (ascending)
  readyNodes.sort((a, b) => a.activeJobs - b.activeJobs);

  // 4. Return least loaded node
  return readyNodes[0] || null;
}
```

**API Endpoints**:
```
POST /api/v1/jobs (creates and coordinates job)
GET /api/v1/nodes/optimal?tool=bash (suggest best node)
```

---

### Phase 3: Execution

**Purpose**: Execute command on remote node(s), stream real-time output.

**Workflow**:

```
Control Plane                    Wrapper (Node)
     │                               │
     │ 1. POST /jobs                 │
     ├──────────────────────────────▶│
     │   {jobId, tool, command}      │
     │                               │
     │ 2. Accept job                 │
     │◀──────────────────────────────┤
     │   {status: "accepted"}        │
     │                               │
     │                               │ 3. Spawn PTY
     │                               │    pty.spawn(bash, ['-c', cmd])
     │                               │
     │ 4. GET /jobs/:id/stream (SSE) │
     ├──────────────────────────────▶│
     │                               │
     │ 5. Stream events              │
     │◀──────────────────────────────┤
     │   data: {type:"status",...}   │
     │   data: {type:"output_chunk"} │
     │   ...                         │
     │   data: {type:"done"}         │
     │                               │
     │ 6. Close stream               │
     │◀──────────────────────────────┤
```

**Wrapper Execution Flow** (based on NexusChat claude-wrapper.js):

```javascript
async function executeJob({ jobId, command, workingDir, timeout }) {
  // 1. Spawn PTY process
  const ptyProcess = pty.spawn('/bin/bash', ['-c', command], {
    cwd: workingDir,
    env: process.env,
    cols: 80,
    rows: 30,
  });

  // 2. Attach output parser
  const parser = new OutputParser();
  let stdout = '';
  let stderr = '';

  ptyProcess.onData((data) => {
    stdout += data;

    // Parse and emit events
    const events = parser.parse(data);
    events.forEach(event => {
      emitSSE(jobId, event); // Send to Control Plane
    });
  });

  // 3. Handle timeout
  const timer = setTimeout(() => {
    ptyProcess.kill('SIGTERM');
    emitSSE(jobId, {
      type: 'error',
      error: 'Job timeout exceeded',
      timeout: timeout,
    });
  }, timeout);

  // 4. Wait for completion
  ptyProcess.onExit(({ exitCode }) => {
    clearTimeout(timer);

    // Save result
    saveJobResult(jobId, {
      exitCode,
      stdout: cleanAnsi(stdout),
      stderr: cleanAnsi(stderr),
      duration: Date.now() - startTime,
    });

    // Emit done event
    emitSSE(jobId, {
      type: 'done',
      exitCode,
    });
  });
}
```

**Output Parser** (adapted from NexusChat output-parser.js):

```javascript
class OutputParser {
  constructor() {
    this.state = 'idle';
    this.buffer = '';
  }

  parse(chunk) {
    const events = [];
    this.buffer += chunk;

    // Parse line by line
    const lines = this.buffer.split('\n');
    this.buffer = lines.pop(); // Keep incomplete line

    for (const line of lines) {
      // Detect tool execution
      if (/^Running (bash|git|docker):/.test(line)) {
        events.push({
          type: 'status',
          category: 'tool',
          message: line,
          icon: '🔧',
        });
      }

      // Detect errors
      if (/^Error:|FATAL:/.test(line)) {
        events.push({
          type: 'status',
          category: 'warning',
          message: line,
          icon: '⚠️',
        });
      }

      // Stream output
      events.push({
        type: 'output_chunk',
        stream: 'stdout',
        text: line + '\n',
      });
    }

    return events;
  }
}
```

---

### Phase 4: Test

**Purpose**: Validate execution results, assert expected outcomes.

**Validation Types**:

1. **Exit Code Validation**
   ```javascript
   if (job.result.exitCode !== 0) {
     throw new Error(`Job failed with exit code ${job.result.exitCode}`);
   }
   ```

2. **Output Validation** (regex patterns)
   ```javascript
   const expectedPattern = /Successfully deployed to production/;
   if (!expectedPattern.test(job.result.stdout)) {
     throw new Error('Expected success message not found in output');
   }
   ```

3. **Side-Effect Validation** (chained jobs)
   ```javascript
   // Example: After "deploy to prod", verify service is running
   const verifyJob = await JobManager.create({
     nodeId: 'prod-001',
     tool: 'bash',
     command: 'systemctl status myapp',
     parent: job.jobId, // Link to parent job
   });

   if (verifyJob.result.exitCode !== 0) {
     // Rollback parent job
     await JobManager.rollback(job.jobId);
   }
   ```

**Test Configuration** (per job):
```json
{
  "jobId": "job-123",
  "tool": "bash",
  "command": "npm run build",
  "tests": [
    {
      "type": "exit_code",
      "expected": 0
    },
    {
      "type": "output_contains",
      "pattern": "Build successful",
      "stream": "stdout"
    },
    {
      "type": "file_exists",
      "path": "/var/www/dist/index.html"
    }
  ]
}
```

**API Endpoints**:
```
POST /api/v1/jobs/:id/test (run tests on completed job)
GET /api/v1/jobs/:id/test-results
```

---

### Phase 5: Deploy

**Purpose**: Execute follow-up actions, trigger dependent jobs, update state.

**Deployment Patterns**:

1. **Sequential Chain**
   ```javascript
   // Build → Test → Deploy → Verify
   const pipeline = [
     { tool: 'bash', command: 'npm run build' },
     { tool: 'bash', command: 'npm test' },
     { tool: 'bash', command: 'rsync -av dist/ /var/www/prod/' },
     { tool: 'bash', command: 'systemctl restart nginx' },
   ];

   for (const step of pipeline) {
     const job = await JobManager.create(step);
     await JobManager.waitForCompletion(job.jobId);

     if (job.result.exitCode !== 0) {
       // Rollback previous steps
       await rollbackPipeline(pipeline.slice(0, pipeline.indexOf(step)));
       break;
     }
   }
   ```

2. **Parallel Fanout**
   ```javascript
   // Deploy to multiple nodes simultaneously
   const deployJobs = await Promise.all([
     JobManager.create({ nodeId: 'web-001', command: 'deploy.sh' }),
     JobManager.create({ nodeId: 'web-002', command: 'deploy.sh' }),
     JobManager.create({ nodeId: 'web-003', command: 'deploy.sh' }),
   ]);

   // Wait for all to complete
   await Promise.all(deployJobs.map(j => JobManager.waitForCompletion(j.jobId)));
   ```

3. **Conditional Branching**
   ```javascript
   const buildJob = await JobManager.create({ command: 'npm run build' });
   await JobManager.waitForCompletion(buildJob.jobId);

   if (buildJob.result.exitCode === 0) {
     // Success: deploy to production
     await JobManager.create({
       nodeId: 'prod-001',
       command: 'deploy-prod.sh',
       parent: buildJob.jobId,
     });
   } else {
     // Failure: notify team
     await JobManager.create({
       tool: 'bash',
       command: 'curl -X POST slack-webhook -d "Build failed"',
     });
   }
   ```

**API Endpoints**:
```
POST /api/v1/pipelines (create multi-job pipeline)
GET /api/v1/pipelines/:id (get pipeline status)
POST /api/v1/jobs/:id/rollback (rollback deployed job)
```

---

## 🔗 Pipeline Example: Full Deployment Workflow

### Scenario: Deploy web application to production

```javascript
// Phase 1: ANALYSIS
const analysisResult = await fetch('/api/v1/jobs/validate', {
  method: 'POST',
  body: JSON.stringify({
    tool: 'bash',
    command: 'cd /var/www/myapp && git pull && npm run build',
    nodeId: 'prod-web-001',
  }),
});
// → { valid: true, estimatedDuration: 120000, warnings: [] }

// Phase 2: COORDINATION
const buildJob = await fetch('/api/v1/jobs', {
  method: 'POST',
  body: JSON.stringify({
    nodeId: 'build-server',
    tool: 'bash',
    command: 'cd /var/www/myapp && npm run build',
    metadata: { pipeline: 'production-deploy' },
  }),
});
// → { jobId: "job-001", streamEndpoint: "/jobs/job-001/stream" }

// Phase 3: EXECUTION (real-time streaming)
const eventSource = new EventSource('/api/v1/jobs/job-001/stream');
eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);

  if (data.type === 'status') {
    console.log(`[${data.category}] ${data.message}`);
    // → [tool] Bash: npm run build
  }

  if (data.type === 'output_chunk') {
    console.log(data.text);
    // → > webpack compiled successfully
  }

  if (data.type === 'done') {
    console.log('✅ Build completed');
    eventSource.close();
  }
};

// Phase 4: TEST
const testResult = await fetch('/api/v1/jobs/job-001/test', {
  method: 'POST',
  body: JSON.stringify({
    tests: [
      { type: 'exit_code', expected: 0 },
      { type: 'file_exists', path: '/var/www/myapp/dist/index.html' },
    ],
  }),
});
// → { passed: true, results: [...] }

// Phase 5: DEPLOY
if (testResult.passed) {
  const deployJob = await fetch('/api/v1/jobs', {
    method: 'POST',
    body: JSON.stringify({
      nodeId: 'prod-web-001',
      tool: 'bash',
      command: 'rsync -av /var/www/myapp/dist/ /var/www/production/',
      parent: 'job-001', // Link to build job
    }),
  });

  // Verify deployment
  const verifyJob = await fetch('/api/v1/jobs', {
    method: 'POST',
    body: JSON.stringify({
      nodeId: 'prod-web-001',
      tool: 'bash',
      command: 'systemctl status nginx && curl -I http://localhost',
      parent: deployJob.jobId,
    }),
  });
}
```

---

## 🔄 Relationship to NexusChat Workflow

**NexusChat CLAUDE.md Pipeline** (adapted):

```
1. MODIFICA       → Analysis (validate changes)
2. TEST           → Coordination (select test environment)
3. DOCUMENTA      → Execution (run tests, generate docs)
4. COMMIT ATOMICO → Test (verify all tests pass)
5. PUSH           → Deploy (push to git, deploy to production)
```

**NexusCLI Equivalent**:

```
1. ANALYSIS       → What needs to be done? Where? How?
2. COORDINATION   → Which nodes? In what order? When?
3. EXECUTION      → Run commands, stream output
4. TEST           → Did it work? Assert expectations
5. DEPLOY         → Apply changes, rollback if needed
```

---

## 📊 Monitoring & Observability

### Metrics (per phase)

**Analysis**:
- Validation failures per hour
- Average validation time
- Most common validation errors

**Coordination**:
- Node selection time
- Queue depth
- Jobs pending vs executing

**Execution**:
- Average job duration (per tool)
- Success rate (exit code 0)
- Timeout rate

**Test**:
- Test pass rate
- Failed assertion types
- Rollback frequency

**Deploy**:
- Deployment success rate
- Rollback time
- Pipeline completion time

**API Endpoint**:
```
GET /api/v1/metrics/pipeline
```

**Response**:
```json
{
  "analysis": {
    "totalValidations": 1523,
    "validationFailures": 45,
    "avgValidationTime": 12
  },
  "execution": {
    "totalJobs": 8421,
    "successfulJobs": 8103,
    "failedJobs": 318,
    "avgDuration": 3421
  },
  "deploy": {
    "totalDeployments": 234,
    "successRate": 0.963,
    "avgRollbackTime": 8234
  }
}
```

---

## 🚀 Future Enhancements

### Pipeline Templates

**Predefined Workflows**:
```yaml
# templates/web-deploy.yaml
name: "Production Web Deployment"
phases:
  - analysis:
      validate: true
      approvers: ["admin@example.com"]

  - coordination:
      nodes: ["build-server"]

  - execution:
      steps:
        - { tool: "bash", command: "git pull" }
        - { tool: "bash", command: "npm install" }
        - { tool: "bash", command: "npm run build" }

  - test:
      tests:
        - { type: "exit_code", expected: 0 }
        - { type: "file_exists", path: "dist/index.html" }

  - deploy:
      strategy: "rolling"
      nodes: ["prod-web-001", "prod-web-002", "prod-web-003"]
      healthCheck: "curl -f http://localhost/health"
```

### AI-Assisted Analysis

**Future Integration**:
- Use Claude/GPT to analyze commands for risks
- Suggest safer alternatives
- Auto-generate test assertions
- Predict optimal node selection

---

## 📚 Related Documents

- [API & Wrapper Contract](./API_WRAPPER_CONTRACT.md) - API specification
- [Architecture Overview](./ARCHITECTURE.md) - System architecture
- [NexusChat CLAUDE.md](../../CLAUDE.md) - Original workflow

---

_Generated by Claude Code (Sonnet 4.5) - 2025-11-17_
