Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-45556: Switch command-line tools to new query interface #1045

Merged
merged 18 commits into from
Aug 7, 2024
Merged

Conversation

timj
Copy link
Member

@timj timj commented Aug 1, 2024

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes
  • (if changing dimensions.yaml) make a copy of dimensions.yaml in configs/old_dimensions

Copy link

codecov bot commented Aug 2, 2024

Codecov Report

Attention: Patch coverage is 67.54967% with 49 lines in your changes missing coverage. Please review.

Project coverage is 89.37%. Comparing base (faad881) to head (9d749eb).
Report is 2 commits behind head on main.

Files Patch % Lines
python/lsst/daf/butler/script/exportCalibs.py 14.28% 12 Missing ⚠️
...on/lsst/daf/butler/script/queryDimensionRecords.py 57.14% 4 Missing and 5 partials ⚠️
python/lsst/daf/butler/script/transferDatasets.py 10.00% 9 Missing ⚠️
python/lsst/daf/butler/script/queryDataIds.py 71.42% 4 Missing and 4 partials ⚠️
...thon/lsst/daf/butler/script/certifyCalibrations.py 0.00% 6 Missing ⚠️
python/lsst/daf/butler/cli/cmd/commands.py 0.00% 1 Missing and 1 partial ⚠️
python/lsst/daf/butler/queries/_query.py 33.33% 1 Missing and 1 partial ⚠️
python/lsst/daf/butler/cli/utils.py 96.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1045      +/-   ##
==========================================
- Coverage   89.39%   89.37%   -0.02%     
==========================================
  Files         360      360              
  Lines       45683    45750      +67     
  Branches     9369     9390      +21     
==========================================
+ Hits        40837    40888      +51     
- Misses       3520     3531      +11     
- Partials     1326     1331       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@timj timj force-pushed the tickets/DM-45556 branch 5 times, most recently from a254a59 to d479ffc Compare August 6, 2024 18:04
@timj timj force-pushed the tickets/DM-45556 branch 2 times, most recently from d83e624 to 6c9a3e4 Compare August 6, 2024 19:39
tests/test_cliCmdQueryDataIds.py Show resolved Hide resolved
python/lsst/daf/butler/script/exportCalibs.py Show resolved Hide resolved
python/lsst/daf/butler/script/exportCalibs.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/script/exportCalibs.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/script/queryDatasets.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/script/removeRuns.py Outdated Show resolved Hide resolved
if not refs:
raise RuntimeError(f"No inputs found for dataset {dataset_type_name} in {input_collection}.")
with butler._query() as query:
results = query.datasets(dataset_type_name, collections=input_collection)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default for find-first has changed, so this is a behavior change. But I think it's an unambiguous improvement, because in the past, if the query had returned multiple results for the same data ID, the certify call later would have choked on them.

timj added 18 commits August 7, 2024 12:46
This allows it to be printed using repr()
Previously the exception that ended up with click was ExitStatus
which was an empty string. This made it impossible to work out
why a test failed. Now run tests in a mode where the exception
is converted to ClickException which does now give feedback
on test failure.
This is also needed because the pruneDatasets command line test
tries to mock out the getDatasets method but doesn't try to mock
out the constructor. This meant that the queryCollections code
was triggering immediately but there are no collections to query.
Deferring allowed the mocks to work.
It is no longer possible to use components.
The new system does not support no_check.
Otherwise they come from a set in random order each time.
@timj timj merged commit 427f673 into main Aug 7, 2024
16 checks passed
@timj timj deleted the tickets/DM-45556 branch August 7, 2024 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants