Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Incremental merge strategy not working (ambiguous column error) #242

Open
1 task done
PilotKresse opened this issue Oct 23, 2024 · 0 comments
Open
1 task done
Labels
bug Something isn't working

Comments

@PilotKresse
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When using incremental strategies with the incremental model, building the model fails on an incremental run with an error (column ... is ambiguous)

The reason is probably that in the generated MERGE INTO statment, the columns are not prefixed with DBT_INTERNAL_SOURCE in the statements following "when not matched then insert"

` merge into [TABLE] as DBT_INTERNAL_DEST
using [TABLE] as DBT_INTERNAL_SOURCE
on (
DBT_INTERNAL_SOURCE.id = DBT_INTERNAL_DEST.id
) when matched then update set
"id" = DBT_INTERNAL_SOURCE."id","col" = DBT_INTERNAL_SOURCE."col"

when not matched then insert
    ("id", "col")
values
    ("id", "col")`

Expected Behavior

Incremental models with merge strategy should work

Steps To Reproduce

Run this example Model:

{{
config(
materialized="incremental", incremental_strategy="merge", unique_key="id"
)
}}

select 1 AS id, 1 AS col
UNION
select 2 AS id, 2 AS col
UNION
select 3 AS id, 2 AS col
{% if is_incremental() %}
UNION select 4 as id, 4 as col
{% endif %}

the first time (non-incremental run) it works fine. On the second run, it will fail with error Column 'id' is ambiguous

Environment

- OS: macOS Sonoma
- dbt-dremio: 1.7.0
- Dremio Software: N/A
- Dremio Cloud: Yes

Relevant log output

09:06:25  Running with dbt=1.7.0
09:06:25  Registered adapter: dremio=1.7.0
09:06:25  Found 10 models, 1 test, 14 sources, 0 exposures, 0 metrics, 570 macros, 0 groups, 0 semantic models
09:06:25  
09:06:31  Concurrency: 2 threads (target='prod')
09:06:31  
09:06:31  1 of 1 START sql incremental model refined.address_changes ..................... [RUN]
09:06:41  1 of 1 ERROR creating sql incremental model refined.address_changes ............ [ERROR in 9.50s]
09:06:41  
09:06:41  Finished running 1 incremental model in 0 hours 0 minutes and 15.89 seconds (15.89s).
09:06:41  
09:06:41  Completed with 1 error and 0 warnings:
09:06:41  
09:06:41    Runtime Error in model address_changes (models/refined/address_changes.sql)
  ERROR: Column 'id' is ambiguous
09:06:41  
09:06:41  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
@PilotKresse PilotKresse added the bug Something isn't working label Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

1 participant