Skip to content

Commit

Permalink
Allow non-expanded datasetRefs in Butler.transfer_from.
Browse files Browse the repository at this point in the history
Previously, transfer_from did not impose requirements on dataset refs,
but set a flag on _importDatasets that (per its docs) should only have
been set if the caller could guarantee expanded datasetRefs.

This commit keeps the open-ended precondition on transfer_from, and
allows internal expansion instead. This should be a no-op if the input
datasetRefs are already expanded. The expand flag is left at its
default instead of being explicitly set to True to give _importDatasets
ultimate control over expansion (e.g., in case it no longer requires
expanded IDs in the future).
  • Loading branch information
kfindeisen committed Oct 17, 2023
1 parent 9166e75 commit f10f94d
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 2 deletions.
2 changes: 2 additions & 0 deletions doc/changes/DM-41165.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Butler.transfer_from no longer requires expanded dataset refs under certain circumstances.
However, providing expanded refs in advance is still recommended for efficiency.
5 changes: 3 additions & 2 deletions python/lsst/daf/butler/_butler.py
Original file line number Diff line number Diff line change
Expand Up @@ -2296,7 +2296,8 @@ def transfer_from(
`Butler` whose registry will be used to expand data IDs.
source_refs : iterable of `DatasetRef`
Datasets defined in the source butler that should be transferred to
this butler.
this butler. In most circumstances, ``transfer_from`` is faster if
the dataset refs are expanded.
transfer : `str`, optional
Transfer mode passed to `~lsst.daf.butler.Datastore.transfer_from`.
skip_missing : `bool`
Expand Down Expand Up @@ -2466,7 +2467,7 @@ def transfer_from(

# Assume we are using UUIDs and the source refs will match
# those imported.
imported_refs = self._registry._importDatasets(refs_to_import, expand=False)
imported_refs = self._registry._importDatasets(refs_to_import)
assert set(imported_refs) == set(refs_to_import)
n_imported += len(imported_refs)

Expand Down

0 comments on commit f10f94d

Please sign in to comment.