Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: projection merge and projection filter transpose rule #162

Merged
merged 72 commits into from
May 1, 2024
Merged

feat: projection merge and projection filter transpose rule #162

merged 72 commits into from
May 1, 2024

Conversation

Sweetsuro
Copy link
Contributor

@Sweetsuro Sweetsuro commented Apr 17, 2024 •
edited
Loading

This PR implements a part of the projection transpose series of rules. It also includes a fair amount of refactoring.

Projection Merge Rule

  • This rule matches on two projection nodes and combines the two nodes into one.
  • It is added to the heuristic optimizer pass before the cascades optimizer. In the future, it should also be added to a pass after the cascades optimizer.

Projection Filter Transpose Rule

  • This rule matches pushes a projection node passed a filter node. If the filter node contains columns that are not in this projection node, the top most projection node is also kept.
  • It is added as a cascades rule.

Refactoring

Relevant functions for projection transpose rules can be found in project_transpose_common.rs. Rules are implemented in separate files as a part of the projection_transpose module rather than in all one file. Similarly, FilterProjectTransposeRule and ProjectionPullUpJoin were moved into this module.

Testing

Unit tests using the dummy heuristic optimizer were implemented.

jurplel and others added 30 commits February 23, 2024 17:09
@jurplel
Add recursive logic, and add filter/sort transpose logic
8f09e0e
@jurplel
change a couple notes
e84a576
@jurplel
Modularize projection rewriting code from ProjectionPullUpJoin rule (…
097029a 
…and fix compiler warnings
@jurplel
Hardcoded filter/sort test
b5e881f
@jurplel
push past projection w/ broken test
67e9632
@jurplel
Implement filter merge
330cdac
@jurplel
More prog on join
64d5572
@jurplel
merge
6020a31
@jurplel
Publicize flattened_nested_logical
bc100ed
@jurplel
Fix filter merge
abbd6b7
@jurplel
Join progress
f78f37b
@jurplel
Add dummy cost model
9c9c515
@jurplel
dummmy catalog + dummy optimizer method
f4a8506
@jurplel
Join test start
e7d4c35
@jurplel
Filter pushdown is beginning to work
609233d
@jurplel
Working testing infra + all tests working!
bb3d1a2
@jurplel
Fix projection mistake
9cb0894
@jurplel
restructure predicate traversal
038db19
@jurplel
Push filter past agg
5023c2a
@jurplel
Merge w/ main
dacc205
@jurplel
Remove print
924649b
@AveryQi115
two stage heuristic
16650a3 
Signed-off-by: AveryQi115 <[email protected]>
@jurplel
Merge remote-tracking branch 'origin/main' into bowad/filter_pushdown
72e1484
@jurplel
merge w/ two-stage
7d1768e
@jurplel
Revise testing infra and port filter pushdown to it
5d71f67
@jurplel
Make categorize_conds_helper more flexible
927717f
@jurplel
Fix crashes related to List
2e906c8
@jurplel
Improve generation of singleton expressions
88397f4
@jurplel
Update tpc-h sqlplannertest with filter pushdown run
7f2f942
@jurplel
non-merging merge
52f9408
jurplel and others added 20 commits March 29, 2024 21:06
@jurplel
Address most comments
5e9b482
@AveryQi115
fix for heuristic rule wrapper
f04e4cb 
Signed-off-by: AveryQi115 <[email protected]>
@Sweetsuro
working project filter transpose
c057b1e
@Sweetsuro
Merge remote-tracking branch 'origin/bowad/filter_pushdown' into swee…
3d142b3 
…tsuro/project-remove
@AveryQi115
heuristic rule wrapper fix
7fe1fd0 
Signed-off-by: AveryQi115 <[email protected]>
@AveryQi115
still merge group
487f1e2 
Signed-off-by: AveryQi115 <[email protected]>
@AveryQi115
Merge branch 'main' into heuristic_wrapper_fix
0c6d314
@AveryQi115
fmt
1d4cd5b 
Signed-off-by: AveryQi115 <[email protected]>
@AveryQi115
Merge branch 'main' into heuristic_wrapper_fix
9c45e69
@AveryQi115
tpc-h result changed due to search space limit after the fix
1814b84 
Signed-off-by: AveryQi115 <[email protected]>
@Sweetsuro
massive refactor
8b10c94
@Sweetsuro
Merge branch 'main' into sweetsuro/project-remove
82c18f0
@Sweetsuro
Merge branch 'heuristic_wrapper_fix' into sweetsuro/project-remove
2532254
@Sweetsuro
project merge as default hueristic
fc18944
@Sweetsuro
move filter_project test cases
e92271d
@Sweetsuro
project filter unit tests
37bfc37
@Sweetsuro
project merge tests
804ef96
@Sweetsuro
move filter project into project filter file
facd914
@Sweetsuro
fix fmt and clippy
42bb85b
@Sweetsuro
Merge branch 'main' into sweetsuro/project-remove
3c6ccb3
@Sweetsuro Sweetsuro marked this pull request as ready for review April 17, 2024 20:32
skyzh
skyzh approved these changes Apr 18, 2024
View reviewed changes
Copy link
Member

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM, good work :)

jurplel
jurplel approved these changes May 1, 2024
View reviewed changes
Copy link
Member

@jurplel jurplel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work on testing. Sorry for the delay.

@Sweetsuro Sweetsuro merged commit 9ef6339 into main May 1, 2024
1 check passed
@Sweetsuro Sweetsuro deleted the sweetsuro/project-remove branch May 1, 2024 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skyzh skyzh skyzh approved these changes

@jurplel jurplel jurplel approved these changes

@AveryQi115 AveryQi115 Awaiting requested review from AveryQi115

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Sweetsuro @skyzh @jurplel @AveryQi115