-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[chore] remove duplicate code in schema scanner #40456
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 38571 ms
|
TPC-DS: Total hot run time: 187488 ms
|
ClickBench: Total hot run time: 31.53 s
|
TeamCity be ut coverage result: |
…#40153) (#40636) ## Proposed changes #40568 #40455 #40456 #40153 Test: 2024-09-10 14:46:14.206 INFO [suite-thread-1] (SuiteContext.groovy:299) - Create new connection for user 'partitions_user' 2024-09-10 14:46:14.214 INFO [suite-thread-1] (Suite.groovy:1162) - Execute tag: select_check_5, sql: select TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME from information_schema.partitions where table_schema="test_partitions_schema_db" order by TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME 2024-09-10 14:46:14.245 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-10 14:46:14.246 INFO [suite-thread-1] (ScriptContext.groovy:120) - Run test_partitions_schema in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy succeed 2024-09-10 14:46:14.252 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-10 14:47:03.655 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy: group=default,p0, name=test_partitions_schema 2024-09-10 14:47:03.656 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-10 14:47:03.656 INFO [main] (RegressionTest.groovy:410) - Test 1 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-10 14:49:48.239 INFO [suite-thread-2] (Suite.groovy:1162) - Execute tag: sql, sql: select * from triggers 2024-09-10 14:49:48.261 INFO [suite-thread-2] (Suite.groovy:1162) - Execute tag: sql, sql: select * from parameters 2024-09-10 14:49:48.280 INFO [suite-thread-2] (Suite.groovy:1162) - Execute tag: sql, sql: select * from profiling 2024-09-10 14:49:48.294 INFO [suite-thread-2] (ScriptContext.groovy:120) - Run test_query_sys_tables in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_query_sys_tables.groovy succeed 2024-09-10 14:49:58.800 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-10 14:50:48.173 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/nereids_p0/system/test_query_sys_tables.groovy: group=query,p0, name=test_query_sys_tables /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_query_sys_tables.groovy: group=query,p0, name=test_query_sys_tables 2024-09-10 14:50:48.174 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-10 14:50:48.175 INFO [main] (RegressionTest.groovy:410) - Test 2 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-10 14:50:48.175 INFO [main] (RegressionTest.groovy:119) - Test finished --------- Co-authored-by: Mingyu Chen <[email protected]>
#40687) backport #40568 #40455 #40456 #40153 #34384 Test result: 2024-09-11 11:00:45.618 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:00:45.619 INFO [suite-thread-1] (Suite.groovy:359) - Execute sql: REVOKE SELECT_PRIV ON test_partitions_schema_db.duplicate_table FROM partitions_user 2024-09-11 11:00:45.625 INFO [suite-thread-1] (SuiteContext.groovy:299) - Create new connection for user 'partitions_user' 2024-09-11 11:00:45.632 INFO [suite-thread-1] (Suite.groovy:1162) - Execute tag: select_check_5, sql: select TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME from information_schema.partitions where table_schema="test_partitions_schema_db" order by TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_ORDINAL_POSITION,SUBPARTITION_ORDINAL_POSITION,PARTITION_METHOD,SUBPARTITION_METHOD,PARTITION_EXPRESSION,SUBPARTITION_EXPRESSION,PARTITION_DESCRIPTION,TABLE_ROWS,AVG_ROW_LENGTH,DATA_LENGTH,MAX_DATA_LENGTH,INDEX_LENGTH,DATA_FREE,CHECKSUM,PARTITION_COMMENT,NODEGROUP,TABLESPACE_NAME 2024-09-11 11:00:45.644 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:00:45.645 INFO [suite-thread-1] (ScriptContext.groovy:120) - Run test_partitions_schema in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy succeed 2024-09-11 11:00:45.652 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-11 11:01:10.321 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_partitions_schema.groovy: group=default,p0, name=test_partitions_schema 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:410) - Test 1 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-11 11:01:10.322 INFO [main] (RegressionTest.groovy:119) - Test finished 2024-09-11 11:03:00.712 INFO [suite-thread-1] (Suite.groovy:1162) - Execute tag: select_check_5, sql: select * from information_schema.table_options ORDER BY TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,TABLE_MODEL,TABLE_MODEL_KEY,DISTRIBUTE_KEY,DISTRIBUTE_TYPE,BUCKETS_NUM,PARTITION_NUM; 2024-09-11 11:03:00.729 INFO [suite-thread-1] (SuiteContext.groovy:309) - Recover original connection 2024-09-11 11:03:00.731 INFO [suite-thread-1] (ScriptContext.groovy:120) - Run test_table_options in /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_table_options.groovy succeed 2024-09-11 11:03:04.817 INFO [main] (RegressionTest.groovy:259) - Start to run single scripts 2024-09-11 11:03:28.741 INFO [main] (RegressionTest.groovy:380) - Success suites: /root/doris/workspace/doris/regression-test/suites/query_p0/system/test_table_options.groovy: group=default,p0, name=test_table_options 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:459) - All suites success. ____ _ ____ ____ _____ ____ | _ \ / \ / ___/ ___|| ____| _ \ | |_) / _ \ \___ \___ \| _| | | | | | __/ ___ \ ___) |__) | |___| |_| | |_| /_/ \_\____/____/|_____|____/ 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:410) - Test 1 suites, failed 0 suites, fatal 0 scripts, skipped 0 scripts 2024-09-11 11:03:28.742 INFO [main] (RegressionTest.groovy:119) - Test finished *************************** 7. row *************************** PartitionId: 18035 PartitionName: p100 VisibleVersion: 2 VisibleVersionTime: 2024-09-11 10:59:28 State: NORMAL PartitionKey: col_1 Range: [types: [INT]; keys: [83647]; ..types: [INT]; keys: [2147483647]; ) DistributionKey: pk Buckets: 10 ReplicationNum: 1 StorageMedium: HDD CooldownTime: 9999-12-31 15:59:59 RemoteStoragePolicy: LastConsistencyCheckTime: NULL DataSize: 2.872 KB IsInMemory: false ReplicaAllocation: tag.location.default: 1 IsMutable: true SyncWithBaseTables: true UnsyncTables: NULL CommittedVersion: 2 RowCount: 4 7 rows in set (0.01 sec) --------- Co-authored-by: Mingyu Chen <[email protected]>
Proposed changes
Issue Number: close #xxx
remove duplicate code in schema scanner