Skip to content

# Monitor flow execution

Not production ready

pgflow is currently in Alpha stage and not yet production ready.

This guide explains how to monitor your pgflow workflows during and after execution using SQL queries.

## View workflow run status

Section titled “View workflow run status”

To check the status of your workflow runs, query the `pgflow.runs` table:

    SELECT * FROM pgflow.runsWHERE flow_slug = 'your_flow_slug'ORDER BY started_at DESCLIMIT 10;

This shows the most recent runs with their status, inputs, outputs, and remaining steps:

    run_id        | flow_slug    | status    | input                | output                   | remaining_steps--------------+--------------+-----------+----------------------+--------------------------+-----------------<run_id UUID> | your_flow    | completed | {"param": "value"}   | {"result": "Success!"}   | 0<run_id UUID> | your_flow    | started   | {"param": "value2"}  | null                     | 3<run_id UUID> | your_flow    | failed    | {"param": "value3"}  | null                     | 5

Run statuses include:

- `started`: The run has been created and is executing steps
- `completed`: All steps have completed successfully
- `failed`: One or more steps have failed after max retries

## View step details for a specific run

Section titled “View step details for a specific run”

To check the status of individual steps within a run:

    SELECT ss.step_slug, ss.status, ss.remaining_deps, ss.remaining_tasks, st.outputFROM pgflow.step_states ssLEFT JOIN    pgflow.step_tasks st    ON        ss.run_id = st.run_id        AND ss.step_slug = st.step_slug        AND st.status = 'completed'WHERE ss.run_id = 'your-run-id-here';

This shows the status of each step with its output:

    step_slug    | status    | remaining_deps | remaining_tasks | output-------------+-----------+----------------+----------------+---------------------------process_data | completed | 0              | 0              | {"processed": true}send_email   | completed | 0              | 0              | "Email sent successfully"final_step   | created   | 2              | 1              | null

Step statuses include:

- `created`: The step has been created but may be waiting for dependencies
- `started`: The step has started execution (all dependencies are complete)
- `completed`: The step has completed successfully
- `failed`: The step has failed after max retries

## Monitor active tasks

Section titled “Monitor active tasks”

To see all tasks that are currently active (queued or being processed):

    SELECT  run_id,  step_slug,  status,  attempts_count,  message_id,  queued_at,  started_at,  last_worker_idFROM pgflow.step_tasksWHERE status IN ('queued', 'started')ORDER BY queued_at ASC;

This shows active tasks at different stages:

    run_id   | step_slug    | status  | attempts_count | message_id | queued_at           | started_at          | last_worker_id---------+--------------+---------+---------------+------------+--------------------+--------------------+---------------<run_id> | process_data | queued  | 1              | 123456     | 2023-05-01 12:00:00| null               | null<run_id> | send_email   | started | 1              | 123457     | 2023-05-01 12:01:00| 2023-05-01 12:02:00| worker-uuid-123

Active task statuses:

- `queued`: Task is ready to run, waiting for a worker to claim it
- `started`: Task is currently being processed by a worker (with `started_at` timestamp and `worker_id`)

## Get execution timeline for a run

Section titled “Get execution timeline for a run”

To see how long each step took to execute:

    SELECT  step_slug,  status,  created_at,  started_at,  completed_at,  failed_at,  EXTRACT(EPOCH FROM (COALESCE(completed_at, failed_at) - started_at)) AS duration_secondsFROM pgflow.step_statesWHERE run_id = 'your-run-id-here'ORDER BY created_at ASC;

## Troubleshoot failed steps

Section titled “Troubleshoot failed steps”

When a step fails, you can check for error details:

    SELECT  ss.step_slug,  st.attempts_count,  st.error_message,  st.queued_at,  st.failed_atFROM pgflow.step_states ssJOIN pgflow.step_tasks st ON  ss.run_id = st.run_id AND  ss.step_slug = st.step_slugWHERE ss.run_id = 'your-run-id-here'AND ss.status = 'failed';

Flow Visualization

In a real application, you’d likely want to create a dashboard to visualize your flows and their execution status.

pgflow stores all the information needed to build rich visualizations of your workflow execution, including:

- Step dependencies
- Execution times
- Retry attempts
- Inputs and outputs

This data is available through SQL queries to the pgflow schema tables.

## View step dependencies

Section titled “View step dependencies”

To understand the structure of your workflow and the dependencies between steps:

    SELECT  steps.step_slug,  array_agg(deps.dep_slug) as dependenciesFROM pgflow.steps stepsLEFT JOIN pgflow.deps deps ON  steps.flow_slug = deps.flow_slug AND  steps.step_slug = deps.step_slugWHERE steps.flow_slug = 'your_flow_slug'GROUP BY steps.step_slug;

## Next steps

Section titled “Next steps”
Organize Flows code Learn how to structure your pgflow code for maintainability and reusability
Change existing Flow options Learn how to safely update configuration options for existing flows
Version your Flows Learn how to safely update your flows without breaking existing runs