# Prune Old Data

import { Aside, Steps, Code } from "@astrojs/starlight/components";
import NotProductionReady from '@/components/NotProductionReady.astro';
import pruningFunctionCode from '@/../../core/supabase/tests/_shared/prune_data_older_than.sql.raw?raw';

As your flows accumulate, pgflow tables gather historical data. While valuable for auditing, keeping this data indefinitely can impact performance and storage costs.

<Aside type="danger" title="CRITICAL: Read before using this function">
This function is **destructive and irreversible**. Before using:

**Test thoroughly on staging first:**
- Verify retention period suits your flows
- Ensure no tasks with long `startDelay` will be affected
- Check database performance impact

**This deletes BOTH completed AND failed runs:**
- No way to preserve failed runs for investigation
- Can delete data before tasks execute if misconfigured (see [startDelay](/reference/configuration/step-execution/#startdelay))
- **Only automate after manual testing confirms safety**
</Aside>

<Aside type="note" title="What gets pruned">
When a run (completed or failed) exceeds the retention period, **ALL** associated records are deleted:

**Deleted:**
- Run records (`pgflow.runs`)
- Step states and tasks - all statuses (`pgflow.step_states`, `pgflow.step_tasks`)
- PGMQ messages - active queue (`pgmq.q_{flow_slug}`) and archived (`pgmq.a_{flow_slug}`)
- Inactive workers (based on `last_heartbeat_at`)

**Preserved:**
- Active runs (`status='started'`), regardless of age
- Flow and step definitions (never pruned)
</Aside>

## Using the Pruning Function

pgflow includes a pruning function that accepts an INTERVAL parameter specifying how much data to keep:

```sql
pgflow.prune_data_older_than(retention_interval INTERVAL)
```

Examples:
```sql
-- Keep 90 days of data (recommended default)
SELECT pgflow.prune_data_older_than(make_interval(days => 90));

-- Keep 30 days of data (only if no long start_delay)
SELECT pgflow.prune_data_older_than(make_interval(days => 30));

-- Keep 6 months of data using interval literals
SELECT pgflow.prune_data_older_than(INTERVAL '6 months');

-- Keep 1 year for compliance requirements
SELECT pgflow.prune_data_older_than(INTERVAL '1 year');
```

## Running periodically

Use pg_cron to schedule automated pruning. **Run during low-traffic periods** as this operation can be resource-intensive for large datasets. Monitor database performance during the first few runs to determine the optimal schedule.

### 1. Install the pruning function

<Aside type="caution">
This function is not yet included in default pgflow migrations. Install it manually by running the [pruning function SQL](#the-pruning-function) using psql or Supabase Studio.
</Aside>

### 2. Setup pg_cron schedule

Run it in Supabase Studio or include in a migration file:

```sql
-- Schedule weekly pruning (every Sunday at 2 AM)
-- This keeps 90 days of data (adjust based on your needs)
SELECT cron.schedule(
  'pgflow-prune-weekly',
  '0 2 * * 0', -- cron expression: minute hour day month weekday
  $$SELECT pgflow.prune_data_older_than(make_interval(days => 90))$$
);
```

##### Verify the scheduled job

```sql
SELECT * FROM cron.job;
```

## The Pruning Function

Run this SQL to install the pruning function:

<Code lang="sql" code={pruningFunctionCode} />