Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SL-1631] Ensure predicate rendering is compatible with partition pruning directives across relevant engines #1014

Open
Tracked by #1010
tlento opened this issue Jan 29, 2024 · 0 comments
Assignees
Labels
Medium priority Created by Linear-GitHub Sync Metricflow Created by Linear-GitHub Sync

Comments

@tlento
Copy link
Contributor

tlento commented Jan 29, 2024

After all the other tasks associated with the overall predicate pushdown issue are complete, there will still be a number of cases where we will need to improve pruning triggers for certain engine types. Scenarios we may need to address with follow-ups include:

  1. Cases where the partition filter granularity is different from the dimension itself and the query spec in such a way that our granularity adjustments bypass pruning - i.e., if we do date_trunc then partition pruning may not apply.
  2. Cases where the partition filter column is tied to the agg_time_dimension but is not the agg_time_dimension itself (e.g., a PARTITION_LOAD_DATE might be mapped on to a daily granularity time field, and we'd want to render a separate filter expression)
  3. Cases where there's a categorical partition that can be mapped from some other partition, like maybe a table is partitioned by REGION_CODE so any filter on COUNTRY can easily be made more efficient
  4. Cases where a partition filter must always be provided, even if it is a partition IS NOT NULL constraint (like 3 above but where users want to group by country instead of pruning out input)

Some of these can be handled with more robust documentation around existing syntax and use of the is_partition flag on dimension columns, but others will likely require expanded support for partition specification in the semantic models themselves.

From SyncLinear.com | SL-1631

@tlento tlento changed the title Ensure predicate rendering is compatible with partition pruning directives across relevant engines [SL-1631] Ensure predicate rendering is compatible with partition pruning directives across relevant engines Jan 29, 2024
@tlento tlento added the Metricflow Gap Created by Linear-GitHub Sync label Jan 30, 2024
@tlento tlento self-assigned this Jan 30, 2024
@tlento tlento added the Medium priority Created by Linear-GitHub Sync label Jan 30, 2024
@tlento tlento added Metricflow Created by Linear-GitHub Sync and removed Metricflow Gap Created by Linear-GitHub Sync labels Feb 8, 2024
@tlento tlento added this to the Predicate Pushdown milestone Mar 11, 2024
@tlento tlento removed this from the Predicate Pushdown milestone Sep 12, 2024
@tlento tlento added Low priority Created by Linear-GitHub Sync Medium priority Created by Linear-GitHub Sync and removed Medium priority Created by Linear-GitHub Sync Low priority Created by Linear-GitHub Sync labels Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Medium priority Created by Linear-GitHub Sync Metricflow Created by Linear-GitHub Sync
Projects
None yet
Development

No branches or pull requests

1 participant