What is the function of defer in dbt Cloud?
Defer is a feature in dbt Cloud that allows users to run a specific subset of models or tests in a sandbox environment without the need to build their upstream parents first. This feature is designed to save time and computational resources, especially when testing a small number of models in a large project.
- Defer: This is a feature in dbt Cloud that allows users to run a subset of models or tests without building their upstream parents first. It's a way to save computational resources and time.
- Sandbox environment: This is an isolated computing environment in which a program or file can be executed without affecting the application it runs on.
- Upstream parents: These are the models or tests that are dependencies for the model or test you want to run. They usually need to be built first before the dependent model can be run.
How can you use defer in dbt Cloud?
To use defer in dbt Cloud, you need to select the Production environment checkbox in the Environment Settings page, toggle the Defer to production button on the command bar, and then edit and run a single model. Defer will then compare artifacts from the most recent successful production job, excluding CI jobs, and automatically obtain the data for the non-edited models from the production environment.
- Production environment: This is the environment where the final product is deployed and run. In dbt Cloud, you can select this environment in the Environment Settings page.
- Defer to production button: This is a button on the command bar in dbt Cloud that you need to toggle to use the defer feature.
- Artifacts: These are the outputs of a job, such as models, tests, and data, that defer compares from the most recent successful production job.
What are the benefits of using defer in dbt Cloud?
Using defer in dbt Cloud can save costs and time during testing of the dbt models. For instance, you can use Defer to rebuild only a single model that you've selected. For the other models, dbt will use the definitions and the latest data from your last successful production run.
- Cost savings: By using defer, you can avoid building all models and tests, which can save computational resources and therefore reduce costs.
- Time savings: Defer allows you to run a single model or test without building the upstream parents first, which can save time, especially in large projects.
- Efficiency: Defer uses the definitions and the latest data from your last successful production run for the non-selected models, which can increase efficiency.
How does defer compare to cloning in dbt Cloud?
Defer and cloning are both ways to save costs in the data warehouse. However, while clone copies an entire schema into the target schema, defer lazily references pre-built models in the source schema.
- Cloning: This is a method in dbt Cloud that involves copying an entire schema into the target schema. It's a way to save costs in the data warehouse.
- Defer: Unlike cloning, defer lazily references pre-built models in the source schema. This means it only uses the models it needs, which can save computational resources.
- Schema: This is a logical container for the data structures in a database. In dbt Cloud, both cloning and defer work with schemas.
What does defer exclude in dbt Cloud?
When using defer in dbt Cloud, it compares artifacts from the most recent successful production job, but it excludes CI jobs. This means that any changes made in a CI job will not be reflected when using defer.
- CI jobs: These are Continuous Integration jobs, which are a part of the software development process where changes are integrated and tested automatically. In dbt Cloud, these are excluded when using defer.
- Artifacts: These are the outputs of a job, such as models, tests, and data, that defer compares from the most recent successful production job.
- Production job: This is the job that produces the final product. In dbt Cloud, defer compares artifacts from the most recent successful production job.
