Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] full refresh of failed model with new partition schema fails on __dbt_tmp #1342

Open
2 tasks done
martenlindblad opened this issue Sep 11, 2024 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@martenlindblad
Copy link

martenlindblad commented Sep 11, 2024

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

If I change partition columns of an incremental model and forget to manually do a full refresh the dbt job will throw an error. This is expected. I will then do a full refresh of the model using dbt build --select <model> --full-refresh --target prod --debug and the model is marked as success.
What is not apparent is that the __dbt_tmp table will linger until the next time the full build is run.
The dbt build job will fail on the next run, however, due to the fact that bigquery doesn't allow partition changes in a create or replace statement.

Expected Behavior

A full refresh of a model will drop the __dbt_tmp table.

Steps To Reproduce

  1. Change partition schema on incremental model
  2. Allow model to fail in dbt build job
  3. Do a manual --full-refresh
  4. Run the dbt build job again

Relevant log output

12:52:35    Database Error in model <model> (models/<model>.sql)
  Cannot replace a table with a different partitioning spec. Instead, DROP the table, and then recreate it. New partitioning spec is interval(type:day,field:p_date) and existing spec is none
12:52:35  

Environment

- OS: Alpine (container python:3.11-slim)
- Python: 3.11
- dbt-core: 1.8.4
- dbt-bigquery: 1.8.2

Additional Context

No response

@martenlindblad martenlindblad added bug Something isn't working triage labels Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants