Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Remove unnecessary case match in getAttrForAggregateExpr #3629

Merged
merged 2 commits into from
Nov 7, 2023

Conversation

liujiayi771
Copy link
Contributor

What changes were proposed in this pull request?

There are many processes in the getAttrForAggregateExprs method that construct aggregateAttr and index for specific aggFuncs. In fact, the patterns of these processes are consistent. They iterate over inputAggBufferAttributes of aggFunc, transform them, and insert them into aggregateAttr. Then, they increase the count in index for each insertion. The logic for validating the supported modes of different aggFuncs is the only specific part, which can be extracted separately.

How was this patch tested?

Exists CI

Copy link

github-actions bot commented Nov 6, 2023

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

github-actions bot commented Nov 6, 2023

Run Gluten Clickhouse CI

2 similar comments
Copy link

github-actions bot commented Nov 6, 2023

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Nov 6, 2023

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Nov 6, 2023

Run Gluten Clickhouse CI

Copy link

github-actions bot commented Nov 6, 2023

Run Gluten Clickhouse CI

@liujiayi771 liujiayi771 marked this pull request as ready for review November 6, 2023 14:05
Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks nice. Can be merged when our internal test passes.

@rui-mo rui-mo merged commit 7308fdb into apache:main Nov 7, 2023
17 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_3629_time.csv log/native_master_11_06_2023_d57fdf26f_time.csv difference percentage
q1 33.90 33.34 -0.552 98.37%
q2 24.90 24.84 -0.059 99.76%
q3 37.81 38.31 0.492 101.30%
q4 37.06 37.40 0.345 100.93%
q5 70.83 70.83 0.000 100.00%
q6 7.93 7.74 -0.187 97.65%
q7 85.37 85.97 0.601 100.70%
q8 86.42 84.68 -1.734 97.99%
q9 122.52 124.39 1.871 101.53%
q10 52.92 54.67 1.743 103.29%
q11 20.30 19.96 -0.341 98.32%
q12 27.03 24.52 -2.516 90.69%
q13 48.93 50.37 1.441 102.95%
q14 17.30 18.50 1.198 106.93%
q15 33.06 32.09 -0.969 97.07%
q16 16.50 16.51 0.005 100.03%
q17 103.39 100.90 -2.496 97.59%
q18 147.83 148.27 0.442 100.30%
q19 14.98 14.74 -0.238 98.41%
q20 30.18 30.95 0.768 102.55%
q21 227.56 225.06 -2.502 98.90%
q22 13.18 13.61 0.423 103.21%
total 1259.91 1257.65 -2.261 99.82%

Comment on lines -422 to -432
if ExpressionMappings.expressionExtensionTransformer.extensionExpressionsMapping.contains(
extendedAggFunc.getClass) =>
// get attributes from the extended aggregate functions
ExpressionMappings.expressionExtensionTransformer
.getAttrsForExtensionAggregateExpr(
aggregateFunc,
mode,
exp,
aggregateAttributeList,
aggregateAttr,
index)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can not delete this part for the extension, when there are some custom agg functions which have the specified logic to implement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that CustomSum should also follow this pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CustomSum is only a simple custom ut case, in out internal product, we need to implement some custom logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation. I see. Sorry for removing that portion of the logic.

@zzcclp
Copy link
Contributor

zzcclp commented Nov 7, 2023

I will raise a pr to fix the extension problem.

zzcclp added a commit to zzcclp/gluten that referenced this pull request Nov 8, 2023
…nctions

In PR apache#3629, it removes the logic to support the custom aggregate functions, must be reverted.

Close apache#3644.
zzcclp added a commit to zzcclp/gluten that referenced this pull request Nov 8, 2023
…nctions

In PR apache#3629, it removes the logic to support the custom aggregate functions, must be reverted.

Close apache#3644.
zzcclp added a commit that referenced this pull request Nov 8, 2023
…nctions (#3648)

In PR #3629, it removes the logic to support the custom aggregate functions, must be reverted.

Close #3644.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants