Skip to content

Commit

Permalink
add cases
Browse files Browse the repository at this point in the history
- test_f_2pc_schema_change
- test_f_conflict_be_restart
- test_f_delete_publish_skip_read
- test_f_partial_update_complex_type_schema_change
- test_flexible_partial_update_restricts
- test_f_schema_change
- test_flexible_partial_update_all_types
- test_partial_update_delete_sign
- test_flexible_partial_update_auto_inc

modify case test_flexible_partial_update_delete_sign, test_flexible_partial_update_seq_col

forbid variant case

tmp add cases

add case test_flexible_partial_update_auto_inc

fix case
  • Loading branch information
bobhan1 committed Sep 11, 2024
1 parent c2df466 commit 9067bc7
Show file tree
Hide file tree
Showing 70 changed files with 5,495 additions and 11 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -158,12 +158,12 @@ public TPipelineFragmentParams plan(TUniqueId loadId, int fragmentInstanceIdInde

if (uniquekeyUpdateMode == TUniqueKeyUpdateMode.UPDATE_FLEXIBLE_COLUMNS && !destTable.hasSkipBitmapColumn()) {
throw new UserException("Flexible partial update can only support table with skip bitmap hidden column."
+ "But table " + destTable.getName() + "doesn't have it");
+ " But table " + destTable.getName() + " doesn't have it");
}
if (uniquekeyUpdateMode == TUniqueKeyUpdateMode.UPDATE_FLEXIBLE_COLUMNS
&& !destTable.getEnableLightSchemaChange()) {
throw new UserException("Flexible partial update can only support table with light_schema_change enabled."
+ "But table " + destTable.getName() + "'s property light_schema_change is false");
+ " But table " + destTable.getName() + "'s property light_schema_change is false");
}
if (uniquekeyUpdateMode == TUniqueKeyUpdateMode.UPDATE_FLEXIBLE_COLUMNS
&& destTable.hasVariantColumns()) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"k1":1,"c1":999,"c2":999}
{"k1":2,"c1":888,"c2":888}
{"k1":3,"c1":777,"c2":777}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
-- This file is automatically generated. You should know what you did if you want to edit this
-- !sql --
1 1 1 1 1
2 2 2 2 2
3 3 3 3 3

-- !sql --
1 999 999 1 1 0 3
3 777 777 3 3 0 3

-- !sql --
1 1 1 1 1 0 2
1 999 999 1 1 0 3
2 \N \N \N \N 1 4
2 2 2 2 2 0 2
2 888 888 2 2 0 3
3 3 3 3 3 0 2
3 777 777 3 3 0 3

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{"k": 1000, "v1": 10}
{"k": 2000, "v2": 20, "v5": 25}
{"k": 3000, "v3": 30}
{"v4": 20, "v1": 43, "v3": 99}
{"v3": 666, "v5": null}
{"v1": 999, "v3": 777}
{"v4": 222, "v3": 888}
{"v2": 111, "v3": 111}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{"k": 1, "v1": 10}
{"k": 2, "v2": 20, "v5": 25, "v3": 20}
{"k": 3, "v3": 30}
{"k": 8, "v2": 88}
{"k": 9, "v2": 99, "v3": 77}
{"k": 10, "v4": 666, "v5": 555}
7 changes: 7 additions & 0 deletions regression-test/data/unique_with_mow_p0/flexible/delete1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{"k": 1, "__DORIS_DELETE_SIGN__": 1}
{"k": 2, "v2": 20, "v5": 25}
{"k": 3, "__DORIS_DELETE_SIGN__": 1}
{"k": 4, "v4": 20, "v1": 43, "v3": 99}
{"k": 5, "v5": null}
{"k": 6, "v1": 999, "v3": 777}
{"k": 7, "__DORIS_DELETE_SIGN__": 1}
2 changes: 2 additions & 0 deletions regression-test/data/unique_with_mow_p0/flexible/delete2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"k": 1, "__DORIS_DELETE_SIGN__": 1}
{"k": 7, "__DORIS_DELETE_SIGN__": 1}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"id":2,"c_array":[1,2,3,5,8,13]}
{"id":1,"c_array":[1,3,5,7,9,11]}
{"id":4,"c_array":[1,2,3,4,5]}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"id":2,"c_jsonb":{"name": "tom", "age": 35}}
{"id":1,"c_jsonb":{"name": "micheal", "age": 28}}
{"id":3,"c_jsonb":{"name": "kk", "age": 42}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"id":2,"c_map":{"count": 100}}
{"id":1,"c_map":{"num": 9}}
{"id":5,"c_map":{"answer": 42}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"id":2,"c_struct":{"a":100,"b":1000}}
{"id":1,"c_struct":{"a":1, "b":10}}
{"id":6,"c_struct":{"a":1000000, "b":10000000}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"id":1,"c_varchar":"apache doris"}
{"id":2,"c_varchar":"apache doris 2.0"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{"k1": 0, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 1, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 2, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 3, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 4, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 5, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 6, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 7, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 8, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 9, "v2": "aaaaaaaaaaaaaaaaaaaa"}
{"k1": 10, "v2": "aaaaaaaaaaaaaaaaaaaa"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{"k1": 0, "v1": "aaaaaaaaaa"}
{"k1": 1, "v1": "aaaaaaaaaa"}
{"k1": 2, "v1": "aaaaaaaaaa"}
{"k1": 3, "v1": "aaaaaaaaaa"}
{"k1": 4, "v1": "aaaaaaaaaa"}
{"k1": 5, "v1": "aaaaaaaaaa"}
{"k1": 6, "v1": "aaaaaaaaaa"}
{"k1": 7, "v1": "aaaaaaaaaa"}
{"k1": 8, "v1": "aaaaaaaaaa"}
{"k1": 9, "v1": "aaaaaaaaaa"}
{"k1": 10, "v1": "aaaaaaaaaa"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{"k1": 0}
{"k1": 1}
{"k1": 2}
{"k1": 3}
{"k1": 4}
{"k1": 5}
{"k1": 6}
{"k1": 7}
{"k1": 8}
{"k1": 9}
{"k1": 10}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"k1": 1, "c1": 10, "c2": 10}
{"k1": 3, "c1": 30, "c2": 30}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":12,"c1":120,"rename_c5":120,"c6":120,"add_k1":null}
{"c0":9,"c2":99.99,"rename_c5":999,"c7":999,"add_k1":null}
{"c0":10,"c2":1010.1010,"rename_c5":1010,"c4":1010,"add_k1":null}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":13,"c1":130,"c5":130,"c6":120,"c2":130,"c10":130,"add_k1":null}
{"c0":11,"c6":111111,"c7":111111,"c3":111111,"add_k1":null}
{"c0":12,"c2":1212.1212,"c9":1212,"c7":1212,"add_k1":null}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"k1":null,"c0":14,"c7":1414,"c9":1414,"c3":1414}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"add_k1":null,"c0":5,"c6":1515,"c5":1515,"c2":15.1515}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":5, "c1": 50, "c4":50,"c2":50,"c6":50}
{"c0":1,"c1":111}
{"c0":2,"c3":222}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":6, "c1": 60, "c6":60,"c3":60,"c5":60,"c7":60,"c8":60}
{"c0":3,"c1":333}
{"c0":4,"c3":444}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":7, "c2":70.7,"c3":70,"c5":70,"c7":70}
{"c0":5,"c1":555}
{"c0":6,"c3":666}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":8,"add_k1":80,"c1":80,"c10":80,"c7":80,"c3":80}
{"c0":1,"add_k1":null, "c2":111.111}
{"c0":3,"add_k1":null,"c2":333.333}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":9,"add_k1":null,"c1":90,"c10":90,"c2":90,"c7":90,"c3":90}
{"c0":2,"add_k1":null,"c2":222.222}
{"c0":4,"add_k1":null,"c2":444.444}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":10,"add_k1":null,"c1":100,"c9":100,"c2":100,"c5":100,"c6":100}
{"c0":5,"c2":555.55,"c4":555,"add_k1":null}
{"c0":7,"c2":777.777,"add_k1":null}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"c0":11,"c1":110,"c3":110,"c2":100,"c7":110,"c6":110,"add_k1":null}
{"c0":6,"c2":66.66,"c4":666,"c9":666,"add_k1":null}
{"c0":8,"c2":888.8,"c7":888,"add_k1":null}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"c0":3, "c3": 30, "c1": 30, "c6": 30,"c10":30}
{"c0":4, "c10": 40, "c4":40,"c8":40,"c4":40}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
{"c0":1, "c1": 10, "c2": 10, "c5": 10}
{"c0":2, "c1": 20, "c3":20,"c7":20,"c4":20}
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
-- This file is automatically generated. You should know what you did if you want to edit this
-- !sql --
0 \N \N \N \N \N
1 \N \N \N \N \N
10 \N \N \N \N \N
2 \N \N \N \N \N
3 \N \N \N \N \N
4 \N \N \N \N \N
5 \N \N \N \N \N
6 \N \N \N \N \N
7 \N \N \N \N \N
8 \N \N \N \N \N
9 \N \N \N \N \N

-- !sql --
test_f_2pc_schema_change UNIQUE_KEYS k1 varchar(10) varchar(10) No true \N true
v1 varchar(10) varchar(10) Yes false \N NONE true
v2 varchar(20) varchar(20) Yes false \N NONE true
renamed_v4 varchar(10) varchar(10) Yes false \N NONE true
v5 varchar(10) varchar(10) Yes false \N NONE true
v6 varchar(50) varchar(50) Yes false \N NONE true

-- !sql --
0 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
1 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
10 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
2 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
3 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
4 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
5 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
6 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
7 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
8 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
9 \N aaaaaaaaaaaaaaaaaaaa \N \N \N

-- !sql --
0 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
1 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
10 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
2 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
3 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
4 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
5 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
6 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
7 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
8 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
9 \N aaaaaaaaaaaaaaaaaaaa \N \N \N

-- !sql --
0 \N \N \N \N \N
1 \N \N \N \N \N
10 \N \N \N \N \N
2 \N \N \N \N \N
3 \N \N \N \N \N
4 \N \N \N \N \N
5 \N \N \N \N \N
6 \N \N \N \N \N
7 \N \N \N \N \N
8 \N \N \N \N \N
9 \N \N \N \N \N

-- !sql --
test_f_2pc_schema_change UNIQUE_KEYS k1 varchar(10) varchar(10) No true \N true
v1 varchar(10) varchar(10) Yes false \N NONE true
v2 varchar(20) varchar(20) Yes false \N NONE true
renamed_v4 varchar(10) varchar(10) Yes false \N NONE true
v5 varchar(10) varchar(10) Yes false \N NONE true
v6 varchar(50) varchar(50) Yes false \N NONE true

-- !sql --
0 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
1 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
10 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
2 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
3 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
4 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
5 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
6 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
7 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
8 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
9 \N aaaaaaaaaaaaaaaaaaaa \N \N \N

-- !sql --
0 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
1 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
10 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
2 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
3 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
4 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
5 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
6 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
7 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
8 \N aaaaaaaaaaaaaaaaaaaa \N \N \N
9 \N aaaaaaaaaaaaaaaaaaaa \N \N \N

Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
-- This file is automatically generated. You should know what you did if you want to edit this
-- !update_varchar --
1 apache doris \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444}

-- !update_jsonb --
1 apache doris {"name":"micheal","age":28}
2 apache doris 2.0 {"name":"tom","age":35}
3 \N {"name":"kk","age":42}

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400]
3 \N {"name":"kk","age":42} \N

-- !update_array --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11]
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [1, 2, 3, 5, 8, 13]
3 \N {"name":"kk","age":42} \N
4 \N \N [1, 2, 3, 4, 5]

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100}
3 \N {"name":"kk","age":42} \N \N
4 \N \N [1, 2, 3, 4, 5] \N

-- !update_map --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9}
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"count":100}
3 \N {"name":"kk","age":42} \N \N
4 \N \N [1, 2, 3, 4, 5] \N
5 \N \N \N {"answer":42}

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9} \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100} {"a":3, "b":4}
3 \N {"name":"kk","age":42} \N \N \N
4 \N \N [1, 2, 3, 4, 5] \N \N
5 \N \N \N {"answer":42} \N

-- !update_struct --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9} {"a":1, "b":10}
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100} {"a":100, "b":1000}
3 \N {"name":"kk","age":42} \N \N \N
4 \N \N [1, 2, 3, 4, 5] \N \N
5 \N \N \N {"answer":42} \N
6 \N \N \N \N {"a":1000000, "b":10000000}

-- !update_varchar --
1 apache doris \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444}

-- !update_jsonb --
1 apache doris {"name":"micheal","age":28}
2 apache doris 2.0 {"name":"tom","age":35}
3 \N {"name":"kk","age":42}

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400]
3 \N {"name":"kk","age":42} \N

-- !update_array --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11]
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [1, 2, 3, 5, 8, 13]
3 \N {"name":"kk","age":42} \N
4 \N \N [1, 2, 3, 4, 5]

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100}
3 \N {"name":"kk","age":42} \N \N
4 \N \N [1, 2, 3, 4, 5] \N

-- !update_map --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9}
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"count":100}
3 \N {"name":"kk","age":42} \N \N
4 \N \N [1, 2, 3, 4, 5] \N
5 \N \N \N {"answer":42}

-- !update_varchar --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9} \N
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100} {"a":3, "b":4}
3 \N {"name":"kk","age":42} \N \N \N
4 \N \N [1, 2, 3, 4, 5] \N \N
5 \N \N \N {"answer":42} \N

-- !update_struct --
1 apache doris {"name":"micheal","age":28} [1, 3, 5, 7, 9, 11] {"num":9} {"a":1, "b":10}
2 apache doris 2.0 {"jsonk3":333,"jsonk4":444} [300, 400] {"a":100} {"a":100, "b":1000}
3 \N {"name":"kk","age":42} \N \N \N
4 \N \N [1, 2, 3, 4, 5] \N \N
5 \N \N \N {"answer":42} \N
6 \N \N \N \N {"a":1000000, "b":10000000}

Loading

0 comments on commit 9067bc7

Please sign in to comment.