-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor!: isolate arrow code #266
base: main
Are you sure you want to change the base?
Conversation
@JayWhite2357 Have a review in your free time. |
@varshith257 Really thanks for your PR! Actually it would be better if you bump Arrow to 53.1.0 in a separate PR. Or I can do it. |
@iajoiner I will bump to its latest version in follow-up PR if needee |
844f684
to
4af2e8b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks pretty good so far. The main concern I have is around cyclic dependencies.
The resulting code must not introduce cyclic dependencies and should resolve any existing ones with arrow related code. This is one of the primary reasons for this cleanup: in order to more easily identify and resolve these.
In particular, nothing in base
should depend on anything outside of base
.
I've added comments to this effect. If you resolve those along with @iajoiner's concerns, we should be good on this PR.
crates/proof-of-sql/src/base/arrow/provable_query_result_test.rs
Outdated
Show resolved
Hide resolved
crates/proof-of-sql/src/base/arrow/final_round_builder_arrow_tests.rs
Outdated
Show resolved
Hide resolved
crates/proof-of-sql/src/base/arrow/final_round_builder_arrow_tests.rs
Outdated
Show resolved
Hide resolved
@JayWhite2357 Will bump with the fixes |
…of-sql into refactor/arrow
…roof-of-sql into refactor/arrow
@JayWhite2357 @iajoiner Can have a look at it here? I am not sure what it making to fail |
I believe It's because of being unable to find the implementations in |
…of-sql into refactor/arrow
replace RecordBatch::try_from(query_result).unwrap() with RecordBatch::try_from(query_result.table).unwrap() This is caused because I asked you to remove this code: impl<S: Scalar> TryFrom<QueryData<S>> for RecordBatch {
type Error = ArrowError;
fn try_from(value: QueryData<S>) -> Result<Self, Self::Error> {
Self::try_from(value.table)
}
} |
This is a final fix and all CI will pass though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@varshith257 Really thanks for your great work!
Please be sure to look over the pull request guidelines here: https://github.com/spaceandtimelabs/sxt-proof-of-sql/blob/main/CONTRIBUTING.md#submit-pr.
Please go through the following checklist
!
is used if and only if at least one breaking change has been introduced.source scripts/run_ci_checks.sh
.Rationale for this change
Currently, arrow is one of the elements that must be disabled to enable no_std support. There are many instances of #[cfg(feature = "arrow")] throughout the codebase, which leaves the code cluttered. To make code readable and manageable , arrow module is introduced.
Closes #237
/claim #237
What changes are included in this PR?
Are these changes tested?
Yes