<!-- Copyright (c) 2026 defconxt. All rights reserved. -->
<!-- Licensed under AGPL-3.0 — see LICENSE file for details. -->
<!-- NERF is a trademark of defconxt. -->

---
technique: performing-security-data-migration
subdomain: security-data-engineering
title: "Performing Security Data Migration"
description: "Techniques and methodologies for performing security data migration"
tags:
  - data-migration
  - ETL
  - platform-migration
  - data-integrity
  - compliance
metadata:
  mitre-attack: []
---

# Performing Security Data Migration

> Plan and execute migrations of security data between platforms, formats, and environments while maintaining data integrity, chain of custody, and compliance.

## Overview

Security data migration involves moving logs, alerts, detection rules, and forensic artifacts between SIEM platforms, cloud environments, or storage tiers. This technique covers migration planning, data validation, rollback strategies, and chain-of-custody preservation.

## Core Concepts

- **Schema mapping**: Translate field names and types between source and target platforms
- **Data validation**: Verify record counts, checksums, and content integrity post-migration
- **Incremental migration**: Migrate data in batches to minimize downtime and risk
- **Chain of custody**: Maintain cryptographic proof that data was not altered during migration
- **Rollback strategy**: Define clear rollback procedures for each migration phase

## Workflow

1. Inventory source data: volume, format, schema, and dependencies
2. Map source schema to target platform schema
3. Build and test migration pipeline in staging environment
4. Execute pilot migration with representative data subset
5. Validate pilot results against integrity checksums
6. Execute incremental production migration in phases
7. Run parallel validation during cutover window
8. Verify complete data transfer with reconciliation reports
9. Decommission source after validation hold period
10. Document migration for compliance and audit trail

## Implementation

```python
import hashlib
import json
from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, Optional


@dataclass
class MigrationRecord:
    record_id: str
    source_hash: str
    target_hash: str | None = None
    status: str = "pending"
    migrated_at: datetime | None = None


class SecurityDataMigrator:
    """Manage security data migration with integrity verification."""

    def __init__(self, source_name: str, target_name: str) -> None:
        self.source_name = source_name
        self.target_name = target_name
        self.records: list[MigrationRecord] = []

    def prepare_record(self, record_id: str, data: dict) -> MigrationRecord:
        source_hash = hashlib.sha256(json.dumps(data, sort_keys=True).encode()).hexdigest()
        record = MigrationRecord(record_id=record_id, source_hash=source_hash)
        self.records.append(record)
        return record

    def migrate_record(self, record_id: str, transformed_data: dict) -> bool:
        record = next((r for r in self.records if r.record_id == record_id), None)
        if not record:
            return False
        record.target_hash = hashlib.sha256(
            json.dumps(transformed_data, sort_keys=True).encode()
        ).hexdigest()
        record.status = "migrated"
        record.migrated_at = datetime.utcnow()
        return True

    def validate(self) -> dict:
        total = len(self.records)
        migrated = sum(1 for r in self.records if r.status == "migrated")
        failed = sum(1 for r in self.records if r.status == "failed")
        return {"total": total, "migrated": migrated, "failed": failed, "pending": total - migrated - failed}

    def reconciliation_report(self) -> list[dict]:
        return [
            {"id": r.record_id, "status": r.status, "integrity": r.source_hash == r.target_hash if r.target_hash else None}
            for r in self.records
        ]
```

## Best Practices

- Never migrate without a validated rollback plan—test rollback before production
- Use checksums at record level, not just batch level, to catch partial corruption
- Maintain parallel read access during migration to avoid detection gaps
- Document schema mappings for audit—regulators want proof of data integrity
- Plan for schema evolution—target platform may require field transformations
- Run migration during low-alert periods to minimize SOC disruption

## Related Techniques

- [Implementing Security Data Backup Strategy](../implementing-security-data-backup-strategy/SKILL.md)
- [Implementing Security Data Classification](../implementing-security-data-classification/SKILL.md)
