# Feedback Persistence Guide

**Version:** 1.0
**Date:** 2025-11-20
**Status:** Active

This guide explains how feedback sessions are stored, indexed, and retrieved in the DevForgeAI feedback system.

---

## Overview

The feedback persistence layer provides:
- **Durable storage** - Markdown files with structured metadata
- **Fast retrieval** - JSON index for efficient search
- **Cross-project portability** - ZIP export/import functionality
- **Version control friendly** - Plain text formats compatible with Git

---

## Storage Architecture

### Directory Structure

```
devforgeai/feedback/
├── sessions/                    # Individual feedback sessions
│   ├── 20251120-083045-dev-STORY-001.md
│   ├── 20251120-091523-qa-STORY-002.md
│   └── 20251120-143012-release-STORY-003.md
├── index.json                   # Searchable metadata index
├── config.yaml                  # User preferences (trigger mode, templates)
└── archive/                     # Exported feedback packages
    ├── feedback-export-20251120.zip
    └── feedback-export-20251115.zip
```

### File Naming Convention

**Pattern:** `{timestamp}-{operation}-{context}.md`

**Components:**
- `{timestamp}` - ISO 8601 format: `YYYYMMDD-HHMMSS`
- `{operation}` - Command/skill/subagent name (e.g., `dev`, `qa`, `test-automator`)
- `{context}` - Story ID or operation context (e.g., `STORY-001`, `EPIC-005`, `general`)

**Examples:**
- `20251120-083045-dev-STORY-001.md` - /dev command for STORY-001
- `20251120-091523-qa-STORY-002.md` - /qa validation for STORY-002
- `20251120-143012-release-production.md` - Production release feedback
- `20251121-101234-spec-driven-architecture-general.md` - Architecture skill, no story

---

## Session File Format

### Structure

```markdown
---
session-id: "20251120-083045-uuid4"
operation: "dev"
operation-type: "command"
status: "passed"
story-id: "STORY-001"
timestamp: "2025-11-20T08:30:45Z"
duration-ms: 1234567
template-used: "command-passed"
version: "1.0"
---

# Retrospective: /dev STORY-001

## What Went Well

TDD workflow was clear and well-structured. Tests caught 2 edge cases early that would have been bugs in production. Code review automation flagged cyclomatic complexity issue before commit.

## What Could Improve

Test generation took 15 minutes - could use more examples for mocking database calls. Refactoring phase felt rushed, could benefit from additional guidance on when to stop refactoring.

## Suggestions for Next Time

- Add database mocking examples to test-automator subagent
- Include "refactoring checklist" in Phase 3 to guide when enough is enough
- Consider automated performance benchmarking in Phase 4

## Efficiency Notes

Overall workflow: 4/5 (efficient)
Time spent: ~35 minutes (expected 20-25)
Delays: Dependency resolution (5min), test debugging (10min)

## Context (auto-populated)

**Operation:** /dev STORY-001
**Type:** command
**Status:** passed
**Duration:** 20m 34s
**Tests:** 45 passed, 0 failed
**Coverage:** 92.3%

## Performance Metrics (auto-populated)

**Phase Breakdown:**
- Phase 0 (Preflight): 2m 15s
- Phase 1 (Red): 15m 30s
- Phase 2 (Green): 8m 45s
- Phase 3 (Refactor): 5m 12s
- Phase 4 (Integration): 4m 23s
- Phase 5 (Commit): 1m 29s

## User Sentiment (auto-populated)

**Detected Sentiment:** Positive
**Confidence:** High (multiple positive indicators)
**Keywords:** "clear", "well-structured", "effective"

## Actionable Insights (auto-extracted)

1. **Add mocking examples** - Priority: MEDIUM - Owner: test-automator subagent
2. **Refactoring guidance** - Priority: LOW - Owner: spec-driven-dev skill
3. **Dependency resolution speedup** - Priority: HIGH - Owner: infrastructure

---

**Generated by:** spec-driven-feedback v1.0
**Template:** command-passed.yaml
**Session Duration:** 8m 32s
**Questions Answered:** 4/4 (0 skipped)
```

### YAML Frontmatter Fields

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `session-id` | string | Yes | Unique session identifier (timestamp-uuid4) |
| `operation` | string | Yes | Command/skill/subagent name |
| `operation-type` | enum | Yes | `command`, `skill`, or `subagent` |
| `status` | enum | Yes | `passed`, `failed`, `partial`, `completed` |
| `story-id` | string | No | Associated story (null if not story-based) |
| `timestamp` | string | Yes | ISO 8601 timestamp |
| `duration-ms` | number | Yes | Operation duration in milliseconds |
| `template-used` | string | Yes | Template ID (e.g., `command-passed`) |
| `version` | string | Yes | Feedback system version |

---

## Index Structure

### Purpose

The `index.json` file enables:
- Fast search without reading all session files
- Filtering by operation, status, date, story
- Trend analysis across multiple sessions
- Export selection by criteria

### Schema

```json
{
  "version": "1.0",
  "last-updated": "2025-11-20T14:30:12Z",
  "total-sessions": 127,
  "sessions": [
    {
      "session-id": "20251120-083045-uuid4",
      "filename": "20251120-083045-dev-STORY-001.md",
      "operation": "dev",
      "operation-type": "command",
      "status": "passed",
      "story-id": "STORY-001",
      "timestamp": "2025-11-20T08:30:45Z",
      "duration-ms": 1234567,
      "template-used": "command-passed",
      "keywords": ["TDD", "edge cases", "code review", "mocking"],
      "sentiment": "positive",
      "questions-answered": 4,
      "questions-skipped": 0
    },
    {
      "session-id": "20251120-091523-uuid4",
      "filename": "20251120-091523-qa-STORY-002.md",
      "operation": "qa",
      "operation-type": "command",
      "status": "failed",
      "story-id": "STORY-002",
      "timestamp": "2025-11-20T09:15:23Z",
      "duration-ms": 45000,
      "template-used": "command-failed",
      "keywords": ["coverage", "threshold", "violations"],
      "sentiment": "negative",
      "questions-answered": 5,
      "questions-skipped": 1
    }
  ]
}
```

### Index Maintenance

**When to update:**
- After every feedback session creation
- After session deletion
- After manual edits to session files
- After import from external package

**Update mechanism:**
```bash
# Rebuild index from session files
/feedback-reindex

# Index updated automatically on:
# - /feedback command completion
# - /import-feedback completion
```

---

## Storage Workflow

### Session Creation

1. **User completes operation** (e.g., /dev STORY-001)
2. **Hook triggers** (based on config: failures-only, always, specific-operations)
3. **Conversation initiated** (spec-driven-feedback skill)
4. **User provides responses** (answers questions)
5. **Template rendering** (responses mapped to sections)
6. **File written** (`devforgeai/feedback/sessions/{filename}.md`)
7. **Index updated** (`index.json` appended with new entry)

### Session Retrieval

**By Story ID:**
```bash
/feedback-search --story=STORY-001
# Returns all sessions where story-id = "STORY-001"
```

**By Operation:**
```bash
/feedback-search --operation=/dev
# Returns all sessions where operation = "dev"
```

**By Status:**
```bash
/feedback-search --status=failure
# Returns all sessions where status = "failed"
```

**By Date Range:**
```bash
/feedback-search --date=last-30-days
# Returns sessions from last 30 days
```

**By Keyword:**
```bash
/feedback-search --keyword="confusing error"
# Full-text search in session content
```

---

## Data Retention

### Default Policy

- **Active sessions:** Indefinite retention
- **Archived exports:** Manual cleanup (no auto-delete)
- **Index rebuilds:** Preserve all sessions unless explicitly deleted

### Manual Cleanup

**Delete old sessions:**
```bash
# Remove sessions older than 90 days
find devforgeai/feedback/sessions/ -name "*.md" -mtime +90 -delete

# Rebuild index after deletion
/feedback-reindex
```

**Archive before cleanup:**
```bash
# Export sessions before deletion
/export-feedback --date=before-2025-08-01 --sanitize

# Verify archive created
ls -lh devforgeai/feedback/archive/

# Safe to delete after archive verified
```

---

## Cross-Project Export

### Export Format

**ZIP Package Structure:**
```
feedback-export-20251120.zip
├── manifest.json                # Metadata about export
├── sessions/                    # Session markdown files
│   ├── 20251120-083045-dev-STORY-001.md
│   ├── 20251120-091523-qa-STORY-002.md
│   └── ...
├── index.json                   # Searchable index (same schema)
└── config.yaml                  # User configuration (sanitized)
```

### Manifest Schema

```json
{
  "export-version": "1.0",
  "export-date": "2025-11-20T14:30:12Z",
  "source-project": "MyProject",
  "session-count": 15,
  "date-range": {
    "start": "2025-11-01T00:00:00Z",
    "end": "2025-11-20T23:59:59Z"
  },
  "sanitized": true,
  "filters": {
    "date": "2025-11",
    "operation": null,
    "status": null,
    "story-id": null
  },
  "checksum": "sha256:abc123...",
  "devforgeai-version": "1.0.1"
}
```

### Export Command

```bash
# Export all sessions from November 2025, sanitized
/export-feedback --date=2025-11 --sanitize

# Export specific operation
/export-feedback --operation=/qa --date=last-90-days

# Export specific story
/export-feedback --story=STORY-001
```

**Sanitization:**
- Removes API keys, tokens, passwords from session content
- Replaces file paths with generic placeholders
- Preserves structure and insights
- Safe for sharing across teams/organizations

### Import Command

```bash
# Import from ZIP package
/import-feedback feedback-export-20251120.zip

# Import merges with existing sessions (no duplicates by session-id)
# Index automatically rebuilt after import
```

---

## Version Control Integration

### Git-Friendly Format

**Advantages:**
- Plain text markdown (easy diffs)
- YAML frontmatter (structured metadata)
- Human-readable content
- Merge-friendly (conflicts rare)

**Recommended .gitignore:**
```gitignore
# Ignore local feedback config (user-specific)
devforgeai/feedback/config.yaml

# Commit sessions and index (shared team insights)
!devforgeai/feedback/sessions/
!devforgeai/feedback/index.json

# Ignore archives (too large, available via export)
devforgeai/feedback/archive/
```

### Commit Strategy

**Option 1: Per-Session Commits**
```bash
# After feedback session
git add devforgeai/feedback/sessions/{session-file}.md
git add devforgeai/feedback/index.json
git commit -m "feedback: Add retrospective for STORY-001"
```

**Option 2: Batch Commits**
```bash
# Weekly or sprint-end
git add devforgeai/feedback/sessions/
git add devforgeai/feedback/index.json
git commit -m "feedback: Sprint 3 retrospectives (12 sessions)"
```

**Option 3: Gitignore All (Privacy)**
```gitignore
# Feedback is private, not shared
devforgeai/feedback/
```

---

## Performance Considerations

### Index Size

**Estimated growth:**
- 1 session = ~200 bytes in index
- 1000 sessions = ~200 KB index
- 10,000 sessions = ~2 MB index

**Recommendation:** Rebuild index if >5 MB or search performance degrades

### Search Performance

**Index-based search (fast):**
- Operation, status, story-id, date filtering
- O(n) scan of index (typically <100ms for 1000 sessions)

**Full-text search (slower):**
- Keyword search requires reading session files
- O(n) scan of all matching files
- Consider external search tools (ripgrep, Elasticsearch) for >10,000 sessions

---

## Backup and Recovery

### Backup Strategy

**Local backups:**
```bash
# Backup entire feedback directory
tar -czf feedback-backup-$(date +%Y%m%d).tar.gz devforgeai/feedback/

# Store backups outside project
mv feedback-backup-*.tar.gz ~/backups/
```

**Cloud backups:**
- Sync `devforgeai/feedback/` to cloud storage (Dropbox, Google Drive, S3)
- Encrypt if contains sensitive data
- Automate with cron/scheduled tasks

### Recovery

**Restore from backup:**
```bash
# Extract backup
tar -xzf feedback-backup-20251120.tar.gz -C devforgeai/

# Rebuild index
/feedback-reindex

# Verify sessions
/feedback-search --date=all
```

**Recover from Git:**
```bash
# Restore deleted session
git checkout HEAD -- devforgeai/feedback/sessions/{session-file}.md

# Rebuild index
/feedback-reindex
```

---

## Security and Privacy

### Sensitive Data

**What might be in sessions:**
- Error messages with stack traces (may reveal file paths)
- User comments (may mention colleagues, projects)
- Performance metrics (may reveal infrastructure details)

**Sanitization (when exporting):**
- Replace absolute paths with placeholders (`/path/to/project` → `{PROJECT_ROOT}`)
- Remove API keys, tokens, credentials (regex pattern matching)
- Redact email addresses and names (if configured)

### Access Control

**File system permissions:**
```bash
# Restrict access to feedback directory (owner only)
chmod 700 devforgeai/feedback/
chmod 600 devforgeai/feedback/sessions/*.md
```

**Team sharing:**
- Commit to Git only if team agrees (shared retrospectives)
- Use exports for selective sharing (specific stories/dates)
- Consider separate repos for sensitive feedback

---

## Troubleshooting

### Index Out of Sync

**Symptoms:**
- Search returns wrong results
- Session count mismatch
- Missing recent sessions in search

**Solution:**
```bash
/feedback-reindex
# Scans all session files, rebuilds index from scratch
```

### Session File Corruption

**Symptoms:**
- YAML frontmatter parse errors
- Missing required fields
- Malformed markdown

**Solution:**
```bash
# Validate session file
cat devforgeai/feedback/sessions/{session-file}.md | head -20

# Check YAML frontmatter
grep -A 10 "^---" {session-file}.md | grep -B 10 "^---"

# Repair manually or restore from Git/backup
```

### Duplicate Sessions

**Symptoms:**
- Same session appears multiple times in index
- Duplicate session-ids

**Cause:** Index update failed mid-operation, index manually edited

**Solution:**
```bash
# Rebuild index (deduplicates automatically)
/feedback-reindex
```

---

## Related Documentation

- **Feedback Question Templates:** `feedback-question-templates.md`
- **Feedback Analysis Patterns:** `feedback-analysis-patterns.md`
- **Feedback Export Formats:** `feedback-export-formats.md`
- **Template Format Specification:** `template-format-specification.md`

---

**The persistence layer is designed for simplicity, portability, and durability. Plain text formats ensure feedback data outlives any specific tool or framework. Version control integration makes feedback a first-class artifact in the development workflow.**
