Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CH] Ignore unstabe uts and add more message when failed. #7821

Merged
merged 3 commits into from
Nov 5, 2024

Conversation

baibaichen
Copy link
Contributor

What changes were proposed in this pull request?

  1. read data from orc file format failed to load date32, it looks like related Update ORC to the current HEAD ClickHouse/ClickHouse#69473 upgading orc.
  2. 'Missmatch result columns size' => dump query plan and substrait plan for further investigation.

How was this patch tested?

using existed UTs

@baibaichen baibaichen marked this pull request as ready for review November 5, 2024 10:03
Copy link

github-actions bot commented Nov 5, 2024

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Nov 5, 2024

Run Gluten Clickhouse CI on x86

1 similar comment
Copy link

github-actions bot commented Nov 5, 2024

Run Gluten Clickhouse CI on x86

@@ -144,13 +135,15 @@ void adjustOutput(const DB::QueryPlanPtr & query_plan, const substrait::PlanRel
const auto & original_cols = original_header.getColumnsWithTypeAndName();
if (static_cast<size_t>(output_schema.types_size()) != original_cols.size())
{
auto * logger = &Poco::Logger::get("SerializedPlanParser");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auto logger = getLogger("SerialziedPlanParser");

Copy link

github-actions bot commented Nov 5, 2024

Run Gluten Clickhouse CI on x86

@baibaichen baibaichen merged commit f498fe7 into apache:main Nov 5, 2024
8 checks passed
@baibaichen baibaichen deleted the feature/ignore_unstable_ut branch November 5, 2024 13:11
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_11_05_2024_time.csv log/native_master_11_04_2024_5dfbe5258b_time.csv difference percentage
q1 14.03 13.94 -0.089 99.36%
q2 15.87 15.27 -0.595 96.25%
q3 5.36 5.17 -0.194 96.38%
q4 72.58 72.22 -0.353 99.51%
q5 8.84 9.58 0.740 108.37%
q6 2.36 3.89 1.525 164.61%
q7 6.19 6.30 0.108 101.74%
q8 3.23 3.37 0.139 104.28%
q9 24.93 26.51 1.572 106.30%
q10 9.81 9.07 -0.740 92.46%
q11 39.19 37.83 -1.358 96.53%
q12 1.48 1.33 -0.156 89.48%
q13 6.64 6.28 -0.364 94.53%
q14a 50.06 47.78 -2.273 95.46%
q14b 43.16 42.85 -0.304 99.30%
q15 2.31 2.60 0.297 112.89%
q16 49.39 49.08 -0.317 99.36%
q17 4.80 5.65 0.856 117.85%
q18 6.92 6.91 -0.013 99.81%
q19 2.44 2.21 -0.234 90.41%
q20 1.55 1.51 -0.039 97.49%
q21 1.15 1.21 0.062 105.40%
q22 8.03 8.15 0.123 101.53%
q23a 106.37 108.70 2.336 102.20%
q23b 128.46 129.45 0.993 100.77%
q24a 115.11 107.07 -8.043 93.01%
q24b 107.42 101.02 -6.396 94.05%
q25 5.00 3.98 -1.019 79.62%
q26 4.29 3.63 -0.661 84.59%
q27 4.96 4.13 -0.826 83.35%
q28 31.80 33.95 2.152 106.77%
q29 10.88 11.34 0.454 104.17%
q30 4.69 4.64 -0.051 98.91%
q31 7.45 7.23 -0.221 97.03%
q32 1.29 1.29 -0.005 99.64%
q33 4.46 4.64 0.178 103.99%
q34 3.93 5.11 1.182 130.10%
q35 10.21 9.59 -0.613 93.99%
q36 5.27 6.04 0.761 114.43%
q37 5.00 4.98 -0.026 99.47%
q38 14.73 15.35 0.627 104.26%
q39a 3.15 3.60 0.447 114.16%
q39b 3.27 3.09 -0.185 94.35%
q40 5.33 4.10 -1.234 76.85%
q41 0.68 0.63 -0.046 93.27%
q42 1.03 0.95 -0.078 92.45%
q43 4.79 4.71 -0.080 98.33%
q44 9.20 10.66 1.465 115.93%
q45 3.25 3.38 0.131 104.03%
q46 3.88 3.89 0.011 100.29%
q47 18.20 17.90 -0.294 98.39%
q48 5.13 5.14 0.008 100.15%
q49 7.08 6.89 -0.191 97.30%
q50 22.53 21.41 -1.120 95.03%
q51 9.62 9.36 -0.256 97.34%
q52 1.06 1.09 0.034 103.23%
q53 2.46 2.28 -0.186 92.46%
q54 3.64 3.79 0.149 104.11%
q55 1.11 0.99 -0.113 89.78%
q56 4.19 4.10 -0.090 97.85%
q57 10.51 10.39 -0.126 98.81%
q58 2.42 2.49 0.067 102.76%
q59 11.58 10.76 -0.822 92.90%
q60 4.11 4.05 -0.063 98.48%
q61 4.25 4.20 -0.044 98.96%
q62 4.65 4.96 0.317 106.83%
q63 2.41 2.36 -0.043 98.21%
q64 66.24 59.74 -6.497 90.19%
q65 19.07 17.51 -1.558 91.83%
q66 4.36 4.83 0.464 110.64%
q67 419.02 430.70 11.681 102.79%
q68 3.48 3.82 0.333 109.57%
q69 5.13 5.62 0.489 109.52%
q70 10.65 10.72 0.078 100.73%
q71 2.50 2.30 -0.201 91.94%
q72 218.14 217.08 -1.053 99.52%
q73 2.37 2.45 0.074 103.10%
q74 22.83 23.02 0.193 100.85%
q75 26.34 25.27 -1.064 95.96%
q76 12.16 11.47 -0.689 94.33%
q77 2.19 1.95 -0.237 89.16%
q78 49.00 53.21 4.213 108.60%
q79 3.93 3.93 0.004 100.09%
q80 11.72 12.10 0.371 103.17%
q81 4.61 4.81 0.205 104.45%
q82 7.55 7.87 0.316 104.19%
q83 1.70 1.56 -0.142 91.66%
q84 2.89 2.98 0.093 103.21%
q85 7.95 6.99 -0.953 88.01%
q86 4.35 5.46 1.111 125.55%
q87 13.67 13.67 -0.003 99.98%
q88 18.24 18.55 0.305 101.67%
q89 3.39 3.09 -0.296 91.25%
q90 3.05 3.00 -0.054 98.22%
q91 2.22 2.22 0.006 100.26%
q92 1.24 1.24 0.002 100.12%
q93 39.96 40.24 0.282 100.71%
q94 26.66 27.40 0.738 102.77%
q9 90.63 89.73 -0.900 99.01%
q5 2.56 2.64 0.073 102.87%
q96 17.55 17.79 0.235 101.34%
q97 1.88 2.06 0.178 109.44%
q98 9.52 11.36 1.839 119.32%
q99 9.52 11.36 1.839 119.32%
total 2239.91 2236.42 -3.493 99.84%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_11_05_2024_time.csv log/native_master_11_05_2024_71b7645001_time.csv difference percentage
q1 52.66 52.41 -0.247 99.53%
q2 30.81 31.37 0.557 101.81%
q3 54.32 54.42 0.100 100.18%
q4 43.00 43.58 0.585 101.36%
q5 105.60 104.27 -1.328 98.74%
q6 11.63 13.06 1.435 112.34%
q7 112.71 111.13 -1.580 98.60%
q8 117.44 115.24 -2.202 98.12%
q9 171.62 170.17 -1.449 99.16%
q10 65.74 66.86 1.121 101.70%
q11 27.03 26.91 -0.123 99.54%
q12 31.95 33.76 1.811 105.67%
q13 52.46 52.53 0.073 100.14%
q14 23.50 25.31 1.813 107.71%
q15 49.89 51.09 1.194 102.39%
q16 18.24 17.24 -1.008 94.48%
q17 125.01 125.66 0.652 100.52%
q18 196.79 198.47 1.680 100.85%
q19 27.60 30.01 2.406 108.72%
q20 41.21 44.58 3.374 108.19%
q21 332.16 332.16 -0.005 100.00%
q22 16.58 15.95 -0.627 96.22%
total 1707.95 1716.19 8.231 100.48%

@baibaichen
Copy link
Contributor Author

ClickHouse/ClickHouse#71500 fixed 24be715

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants