Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-3704][CH]Enable orc/parquet subcolumn pruning #3707

Merged
merged 2 commits into from
Nov 23, 2023

Conversation

taiyang-li
Copy link
Contributor

@taiyang-li taiyang-li commented Nov 14, 2023

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

(Fixes: #3704)

related prs:

Copy link

#3704

Copy link

Run Gluten Clickhouse CI

@taiyang-li taiyang-li marked this pull request as draft November 14, 2023 08:53
@taiyang-li taiyang-li marked this pull request as ready for review November 15, 2023 09:35
Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

@baibaichen
Copy link
Contributor

which ut will use this functionality? how to verify it?

@taiyang-li
Copy link
Contributor Author

which ut will use this functionality? how to verify it?

Existed tests: test("test orc/parquet table with null complex type values")

Copy link
Contributor

@baibaichen baibaichen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@binmahone binmahone merged commit df3eb37 into apache:main Nov 23, 2023
7 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_11_23_2023_time.csv log/native_master_11_22_2023_16abf3966_time.csv difference percentage
q1 34.09 34.52 0.432 101.27%
q2 24.43 24.61 0.185 100.76%
q3 37.11 37.51 0.402 101.08%
q4 35.49 36.72 1.234 103.48%
q5 70.74 70.57 -0.175 99.75%
q6 6.77 7.86 1.095 116.18%
q7 85.18 84.69 -0.499 99.41%
q8 84.73 85.82 1.086 101.28%
q9 118.73 124.99 6.260 105.27%
q10 43.59 43.50 -0.094 99.79%
q11 19.61 19.62 0.010 100.05%
q12 23.77 25.22 1.457 106.13%
q13 45.78 45.75 -0.029 99.94%
q14 17.52 19.48 1.957 111.17%
q15 28.14 27.08 -1.058 96.24%
q16 15.14 15.52 0.378 102.50%
q17 100.38 100.26 -0.124 99.88%
q18 147.08 147.25 0.177 100.12%
q19 12.95 13.16 0.203 101.56%
q20 28.07 26.97 -1.106 96.06%
q21 218.99 221.08 2.085 100.95%
q22 12.93 12.89 -0.040 99.69%
total 1211.23 1225.07 13.836 101.14%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CH] support tuple subcolumn pruning for orc/parquet
4 participants