Ophys dandiset has duplicate object ID

[rly]

File sub-R6_ses-20200206T210000_behavior+ophys in dandiset 000054, which we use for testing in dandiset 000717 and staging dandiset 213889 has duplicate object ID for paths:

processing/ophys/Fluorescence/Deconvolved/rois
processing/ophys/Fluorescence/Neuropil/rois
processing/ophys/Fluorescence/RoiResponseSeries/rois

This causes an error in nwbfile.objects which builds a dictionary based on object ID and assumes uniqueness. These object IDs should be unique within a file. As a result, our helper function get_object_by_name raises an error.

action items (1 and 2 are outside the purview of this benchmarks repo, but noting here anyway):

1. the pynwb validator should check for uniqueness of all object IDs
2. we should scan through all dandisets for these issues (we can use jeremy's cached lindi files for fast scanning), and, when found, suggest replacing the duplicate IDs with new ones
3. we should manually replace the duplicate IDs in this ophys dandiset, upload it to dandiset 000717, re-generate a lindi file, and upload that to staging dandiset 213889

[CodyCBakerPhD]

we should manually replace the duplicate IDs in this ophys dandiset, upload it to dandiset 000717, re-generate a lindi file, and upload that to staging dandiset 213889

I'm doing that first part now

Then I'll ping you do regenerate lindi and re-upload

[CodyCBakerPhD]

To clarify my understanding - would this have been expected behavior if these fields were links to a single common (re-used) DynamicTableRegion? And the problem is that they were written as standalone datasets instead of links?

[CodyCBakerPhD]

Wait, no... all 3 roi fields in the file have the same Object Ref, so they are indeed pointing to the same object

I also don't encounter any issue with nwbfile.objects on this file

Please advise @rly

[rly]

Ah, I see now. I was looking at the LINDI file which does not appear to differentiate between linked datasets and datasets. Looking at the NWB file as an h5 file, I see that they are the same object. This is a bug in LINDI. I'll file a ticket there.

Thanks for looking into this!