Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-44362: Fix unit test for spatial overlaps #1032

Merged
merged 2 commits into from
Jul 10, 2024
Merged

Conversation

andy-slac
Copy link
Contributor

@andy-slac andy-slac commented Jun 30, 2024

Using non-common skypix is going to be deprecated in favor of a
direct use of regions. New query system raises NotImplementedError
for non-common skypix, this patch replaces that exception with
a new exception class to support exception propagation from remote
butler to client.

test_skypix_constraint_queries unit test is updated to handle
exceptions from the new query system. When direct butler switches
to the new query system, that test can be removed or changed to
always expect the exceptions.

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes
  • (if changing dimensions.yaml) make a copy of dimensions.yaml in configs/old_dimensions

@andy-slac
Copy link
Contributor Author

@TallJimbo, I tried few different ideas and ended up with a special visitor that rewrites predicates which use non-common skypix. A complicated thing is that it also needs to override dimension group in the tree to replace skypix there too. It fixes that particular query that was broken in the unit test with htm11 constraint, but that unit test is still failing on a subsequent query. Before I try to fix the next thing, could you check that this new visitor makes sense or maybe I need something different?

Copy link

codecov bot commented Jun 30, 2024

Codecov Report

Attention: Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Project coverage is 89.36%. Comparing base (eb93ffb) to head (61684c3).

Files Patch % Lines
...thon/lsst/daf/butler/registry/dimensions/static.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1032      +/-   ##
==========================================
- Coverage   89.37%   89.36%   -0.01%     
==========================================
  Files         359      359              
  Lines       45506    45511       +5     
  Branches     9352     9354       +2     
==========================================
+ Hits        40670    40673       +3     
- Misses       3535     3536       +1     
- Partials     1301     1302       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TallJimbo
Copy link
Member

We talked about this at the Tuesday standup (while you were on vacation), and given the new complexity that seems to be required, I think we should really consider deprecating this functionality instead, since I think the primary reason it's important in the old query system is as a workaround for general geometry overlap queries, and we can do that better in the new query system if we do it more directly.

@andy-slac
Copy link
Contributor Author

I'm ok with deprecation, but does that mean that we still need to support until next release? Or is it fine if it's broken for remote butler only?

@TallJimbo
Copy link
Member

I think it's fine for it to be broken for RemoteButler only for now; the deprecation period will constrain the timeline for having the DirectButler's old query methods delegate to the new system, but there are other differences we plan to deprecate in that category already. Those are being tracked on DM-43587, and I've added this issue there as well.

@andy-slac andy-slac force-pushed the tickets/DM-44362 branch 2 times, most recently from fd3d28a to 26a7ad9 Compare July 9, 2024 20:34
Using non-common skypix is going to be deprecated in favor of a
direct use of regions. New query system raises NotImplementedError
for non-common skypix, this patch replaces that exception with
a new exception class to support exception propagation from remote
butler to client.

`test_skypix_constraint_queries` unit test is updated to handle
exceptions from the new query system. When direct butler switches
to the new query system, that test can be removed or changed to
always expect the exceptions.
@andy-slac andy-slac changed the title DM-44362: Improve support for non-common skypix DM-44362: Fix unit test for spatial overlaps Jul 9, 2024
@andy-slac andy-slac marked this pull request as ready for review July 9, 2024 20:38
@andy-slac andy-slac merged commit 11c1998 into main Jul 10, 2024
18 checks passed
@andy-slac andy-slac deleted the tickets/DM-44362 branch July 10, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants