-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable querying cumulative metrics with agg_time_dimension #999
Conversation
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
a0865b7
to
945c5bf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks for splitting this up into such bite-sized chunks for review!
There are a few boundary cases I'd like to test here and read through before approving.
- The time range constraint SQL rendering - the query runs, but I worry this is because the optimizer collapses the metric_time references into direct column name references, and it's not clear to me what the raw rendering will look like. I see this is tested for result correctness in the integration test (which is great) but there is no SQL to look at right now.
- Do we render correctly for metrics that take multiple measures with the same agg_time_dimension?
- Do we throw the right error for metrics that take multiple measures with different agg_time_dimensions?
cumulative_metric_constrained_node: Optional[ConstrainTimeRangeNode] = None | ||
if ( | ||
cumulative_metric_adjusted_time_constraint is not None | ||
and time_range_constraint is not None | ||
and queried_linkable_specs.contains_metric_time | ||
and queried_agg_time_dimension_specs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Careful.... the ConstrainTimeRangeNode is hard-coded against metric time here:
https://github.com/dbt-labs/metricflow/blob/main/metricflow/plan_conversion/dataflow_to_sql.py#L991
If you follow that through the underlying property accessor it's just a set of TimeDimensionInstances manufactured out of thin air with the value of the METRIC_TIME_NAME enum entry.
It's possible this will still work due to magic, but we may need to change things up a bit to allow an override, otherwise time range constraints might not work properly for cases where people use the underlying agg_time_dimension name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep I'm aware! But good point that I actually I should revert the change on this line so it only applies the time constraint with metric_time
- forgot to do that.
I had actually put up a separate issue for this since it's not directly related to the feature: #1008
Since the time constraint param isn't exposed in any of our APIs, only in the MF CLI, it seems less important to fix that issue before releasing this feature.
Note: in the integration test you noted, I used a where filter instead of a time constraint.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just put up a commit to raise a more appropriate error if metric_time
is not used here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the time constraint param isn't exposed in any of our APIs, only in the MF CLI, it seems less important to fix that issue before releasing this feature.
Right, makes sense, although we still don't want result divergence for people who do local dev with the CLI and then issue where filters in prod cloud accounts.
If I'm honest, though, I'm mostly looking ahead to predicate pushdown implementation concerns. :)
I'll take a look at the update!
date_part=None, | ||
) | ||
self._metric_time_specs = tuple( | ||
TimeDimensionSpec.generate_possible_specs_for_time_dimension( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice.
@tlento I realized this isn't relevant to this PR since cumulative metrics can't have multiple input measures. So I'll move dealing with this feedback to the time offset PR! |
556a458
to
f8bae3a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
Resolves #1000
Description
Enable querying cumulative metrics with their
agg_time_dimension
. Should have the same behavior as when queried withmetric_time
.