Understand the State of a dbt Schedule

This user guide will show you how you can monitor your dbt Schedules and their dependent pipelines. Understanding the state of your dbt Schedules will help you with any troubleshooting that may be required.

Overview

The dbt Schedule’s Overview page is the best place to view at-a-glance information about its current state. The header of the Overview page provides different data points about the schedule.

1. Time Since Last Run: Shows how long it’s been since the last build was started.
2. Last Run Time: Shows how long the last successful dbt build took.
3. Next Run Start Time: Shows when the next scheduled build will start.
4. Selector: Shows the selector that this schedule is using to determine which models to run.
5. Git Hash: Shows the git commit hash of the dbt project that this schedule is using.

To view more specific details about each run of your dbt Schedule, go to the Activities tab.

Activities

The Activities page provides a historical reel of color-coded events for each run of your dbt Schedule. These events can be selected and drilled into to get more information about each run, including the dependent pipeline activity and the dbt build logs.

Each dbt Schedule run consists of the following 3 steps:

1. Build Trigger
2. Pipeline Ingestion
3. dbt Build

Build Trigger

A dbt Schedule is triggered:

1. Immediately after the dbt schedule is created.
2. Immediately after a new commit is pushed to the main branch.
3. Immediately after manually triggering the schedule using the UI or the API.
4. According to the cron schedule you’ve set, assuming there is no other dbt run in progress.

Once the schedule is triggered, all dependent pipelines begin to check for new data.

Pipeline Ingestion

After a dbt Schedule is triggered, its dependent pipelines check for new data and start ETLing. In the Ingest Details tab of a specific activity you can see the progress of each pipeline. Selecting View logs will take you to that pipeline’s activity page where you can view details about that pipeline’s ETL activities.

If the dbt Schedule has no dependent pipelines, the Ingest Details tab will be greyed out and made inaccessible.

dbt Build

After all dependent pipelines have finished ingesting and loading data, the dbt build command that builds models in your destination is executed.

After the build completes, a new activity is added to the Run Logs which shows how many models were built.

You can review the dbt build log output by selecting the dbt Build Logs tab for an activity.

The dbt build logs provide information about what actions were completed by dbt during the build, including which models were built.

If a dbt build did not succeed, the logs will provide error messages about the build failure.