Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] New DataType: Duration (without arithmetic) #1051

Merged
merged 22 commits into from
Jun 21, 2023
Merged

Conversation

xcharleslin
Copy link
Contributor

@xcharleslin xcharleslin commented Jun 14, 2023

Adds support for an Arrow-like Duration datatype in Daft, parameterized by TimeUnit.

Includes: conversion to/from Arrow, Python; casting (using Arrow logical->logical casts); blackbox kernels.

Does not include temporal arithmetic, which requires refactoring the type system (next PR).

Drivebys
Converted various Array ops to macros.

@codecov
Copy link

codecov bot commented Jun 15, 2023

Codecov Report

Merging #1051 (01e45b6) into main (6d1feb7) will increase coverage by 0.01%.
The diff coverage is 75.00%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1051      +/-   ##
==========================================
+ Coverage   88.65%   88.66%   +0.01%     
==========================================
  Files          54       54              
  Lines        5375     5382       +7     
==========================================
+ Hits         4765     4772       +7     
  Misses        610      610              
Impacted Files Coverage Δ
daft/expressions/expressions.py 91.81% <0.00%> (-0.28%) ⬇️
daft/datatype.py 92.57% <100.00%> (+0.17%) ⬆️

... and 1 file with indirect coverage changes

@xcharleslin xcharleslin changed the title Charles/duration [FEAT] Charles/duration Jun 15, 2023
@github-actions github-actions bot added the enhancement New feature or request label Jun 15, 2023
@xcharleslin xcharleslin changed the title [FEAT] Charles/duration [FEAT] New DataType: Duration (without arithmetic) Jun 15, 2023
@xcharleslin xcharleslin marked this pull request as ready for review June 15, 2023 23:00
Copy link
Member

@samster25 samster25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

let new_array = self.physical.filter(mask)?;
Ok(Self::new(self.field.clone(), new_array))
}
macro_rules! impl_logical_array_filter {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since all of these are the same, what do you think of putting it into the logicalarray impl https://github.com/Eventual-Inc/Daft/blob/main/src/datatypes/logical.rs#L15

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, good idea

let new_array = self.physical.if_else(&other.physical, predicate)?;
Ok(Self::new(self.field.clone(), new_array))
}
macro_rules! impl_logicalarray_if_else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing as filter: thoughts on moving this to logicalarray impl: https://github.com/Eventual-Inc/Daft/blob/main/src/datatypes/logical.rs#L15

@xcharleslin xcharleslin enabled auto-merge (squash) June 21, 2023 18:10
@xcharleslin xcharleslin merged commit 1907413 into main Jun 21, 2023
@xcharleslin xcharleslin deleted the charles/duration branch June 21, 2023 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants