Skip to content

Commit

Permalink
[DOCS] Add docs on to_arrow and as_arrow
Browse files Browse the repository at this point in the history
  • Loading branch information
samster25 committed Sep 27, 2024
1 parent 44c09de commit 1140a72
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 1 deletion.
5 changes: 4 additions & 1 deletion src/daft-core/src/array/ops/as_arrow.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ use crate::{
pub trait AsArrow {
type Output;

// Retrieve the underlying concrete Arrow2 array.
/// Retrieve the underlying internal Arrow2 array.
/// This does not correct for the logical types and will just yield the physical type of the array.
/// For example, a TimestampArray will yield an arrow Int64Array rather than a arrow Timestamp Array.
/// To get a corrected arrow type, see `.to_arrow()`.
fn as_arrow(&self) -> &Self::Output;
}

Expand Down
6 changes: 6 additions & 0 deletions src/daft-core/src/series/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ impl PartialEq for Series {
}

impl Series {
/// Exports this Series into an Arrow arrow that is corrected for the Arrow type system.
/// For example, Daft's TimestampArray is a logical type that is backed by an Int64Array Physical array.
/// If we were to call `.as_arrow()` or `.physical`on the TimestampArray, we would get an Int64Array that represented the time units.
/// However if we want to export our Timestamp array to another arrow system like arrow2 kernels or python, duckdb or more.

Check warning on line 44 in src/daft-core/src/series/mod.rs

View check run for this annotation

Codecov / codecov/patch

src/daft-core/src/series/mod.rs#L41-L44

Added lines #L41 - L44 were not covered by tests
/// We should convert it back to the canonical arrow dtype of Timestamp rather than Int64.
/// To get the internal physical type without conversion, see `as_arrow()`.

Check warning on line 46 in src/daft-core/src/series/mod.rs

View check run for this annotation

Codecov / codecov/patch

src/daft-core/src/series/mod.rs#L46

Added line #L46 was not covered by tests
pub fn to_arrow(&self) -> Box<dyn arrow2::array::Array> {
self.inner.to_arrow()
}
Expand Down

0 comments on commit 1140a72

Please sign in to comment.