How does "Use MERGE for De-duplication of final tables" work in Snowflake destination? #47219
hongbo-miao
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am trying to understand how "Use MERGE for De-duplication of final tables" works in Snowflake desitination.
If source data before is
and changed to
And this stream is using "Incremental| Append + Deduped".
Here is my understnading:
When "Use MERGE for De-duplication of final tables" is disabled, the Snowflake destination will change to
When "Use MERGE for De-duplication of final tables" is enabled, will the Snowflake destination will change to
Am I understanding correctly? Thanks!
UPDATE 1:
Hmm I may be wrong. I saw source code.
When "Use MERGE for De-duplication of final tables" disabled, it calls
insertRecordsDedupe
. InsideinsertRecordsDedupe
, it callscdcDeletes
. So I assume it also deletes data.Hmm so now I am not fully sure about their difference.
Beta Was this translation helpful? Give feedback.
All reactions