heudiconv crashes when converting a derived ADC map from Siemens scanner #633

neurorepro · 2023-02-08T15:15:15Z

Summary

heudiconv crashes when converting a derived ADC map from Siemens scanner with the following trace:

INFO: Running heudiconv version 0.11.6 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bug_dwi/bids_mysubj/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
Traceback (most recent call last):
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/mnt/projects/bids/golestani_data/test_numpy_error/test_heudiconv.py", line 20, in <module>
    runner(args)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 202, in group_dicoms_into_seqinfos
    mwinfo = validate_dicom(filename, dcmfilter)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 106, in validate_dicom
    del mw.series_signature[sig]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 642, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 516, in image_shape
    first_frame = self.frames[0]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/pydicom/multival.py", line 102, in __getitem__
    return self._list[index]
IndexError: list index out of range

This is generated by frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) in the if-block:

if stackid_tag in dim_seq:
    stackid_dim_idx = dim_seq.index(stackid_tag)
    frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
    dim_seq.pop(stackid_dim_idx)

Looks like this is because frame_indices is empty (so no second axis), which should be expected for a derived map though, since earlier there is:

frame_indices = np.array(
    [frame.FrameContentSequence[0].DimensionIndexValues
     for frame in self.frames])

with

self.frames = pydicom.Sequence(
    frame for frame in self.frames if
    frame.MRDiffusionSequence[0].DiffusionDirectionality
    != 'ISOTROPIC'
    )

So for a derived map self.frames will become empty, so frame_indices will be so as well, and so the if-block will bring hell.

The thing i don't understand is why we are running the if-block in this case, and why the condition evaluates to true in my case where stackid_tag=2134102 and dim_seq[0] = (0020, 9056). But i have to admit i didn't look into the __int__ function of the dim_seq items.

Just in case this is useful, here is the metadata for this single problematic dcm file:

{
	"Modality": "MR",
	"MagneticFieldStrength": 3,
	"ImagingFrequency": 123.226,
	"Manufacturer": "Siemens",
	"ManufacturersModelName": "MAGNETOM Prisma",
	"InstitutionName": "Neurocenter",
	"InstitutionAddress": "Brain avenue",
	"DeviceSerialNumber": "42",
	"StationName": "AWP42",
	"BodyPartExamined": "BRAIN",
	"PatientPosition": "HFS",
	"ProcedureStepDescription": "Not trying to crash",
	"SoftwareVersions": "syngo MR XA30",
	"MRAcquisitionType": "2D",
	"SeriesDescription": "dwi_acq-B2850x1D5MMISO_afid-beta_ADC",
	"ProtocolName": "dwi_acq-B2850x1D5MMISO_afid-beta",
	"ScanningSequence": "EP",
	"SequenceVariant": "SK\\SP",
	"ScanOptions": "PFP\\FS",
	"PulseSequenceName": "*epse2d1_150",
	"ImageType": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC"],
	"ImageTypeText": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC", "ND"],
	"NonlinearGradientCorrection": false,
	"RawImage": false,
	"SeriesNumber": 51,
	"AcquisitionTime": "17:11:9.695000",
	"SliceThickness": 1.5,
	"SpacingBetweenSlices": 1.5,
	"EchoTime": 0.074,
	"RepetitionTime": 6.7,
	"MTState": false,
	"FlipAngle": 90,
	"PartialFourier": 0.875,
	"BaseResolution": 150,
	"ShimSetting": [
		-1273,
		-11693,
		6264,
		288,
		261,
		-78,
		-194,
		57	],
	"TxRefAmp": 233.778,
	"PhaseResolution": 1,
	"ReceiveCoilName": "HeadNeck_64",
	"ReceiveCoilActiveElements": "HC1-6",
	"CoilString": "HeadNeck_64",
	"PulseSequenceDetails": "%SiemensSeq%\\ep2d_diff",
	"RefLinesPE": 45,
	"CoilCombinationMethod": "Adaptive Combine",
	"MultibandAccelerationFactor": 2,
	"PercentPhaseFOV": 100,
	"PercentSampling": 100,
	"EchoTrainLength": 44,
	"PartialFourierDirection": "PHASE",
	"AcquisitionMatrixPE": 150,
	"ReconMatrixPE": 150,
	"BandwidthPerPixelPhaseEncode": 26.316,
	"ParallelReductionFactorInPlane": 3,
	"ParallelAcquisitionTechnique": "SMS",
	"EffectiveEchoSpacing": 0.000253331,
	"DerivedVendorReportedEchoSpacing": 0.000759994,
	"TotalReadoutTime": 0.0377464,
	"PixelBandwidth": 1515,
	"DwellTime": 2.2e-06,
	"PhaseEncodingDirection": "j-",
	"SliceTiming": [
		6.6225,
		0,
		6.7625,
		0.14,
		6.9025,
		0.2825,
		7.045,
		0.4225,
		7.185,
		0.5625,
		7.325,
		0.705,
		7.4675,
		0.845,
		7.6075,
		0.9875,
		7.7475,
		1.1275,
		7.89,
		1.2675,
		8.03,
		1.41,
		8.17,
		1.55,
		8.3125,
		1.69,
		8.4525,
		1.8325,
		8.595,
		1.9725,
		8.735,
		2.1125,
		8.875,
		2.255,
		9.0175,
		2.395,
		9.1575,
		2.535,
		9.2975,
		2.6775,
		9.44,
		2.8175,
		9.58,
		2.9575,
		9.72,
		3.1,
		9.8625,
		3.24,
		10.0025,
		3.3825,
		10.1425,
		3.5225,
		10.285,
		3.6625,
		10.425,
		3.805,
		10.565,
		3.945,
		10.7075,
		4.085,
		10.8475,
		4.2275,
		10.99,
		4.3675,
		11.13,
		4.5075,
		11.27,
		4.65,
		11.4125,
		4.79,
		11.5525,
		4.93,
		11.6925,
		5.0725,
		11.835,
		5.2125,
		11.975,
		5.3525,
		12.115,
		5.495,
		12.2575,
		5.635,
		12.3975,
		5.7775,
		12.5375,
		5.9175,
		12.68,
		6.0575,
		12.82,
		6.2,
		12.96,
		6.34,
		13.1025,
		6.48	],
	"ImageOrientationPatientDICOM": [
		1,
		0,
		0,
		0,
		1,
		0	],
	"InPlanePhaseEncodingDirectionDICOM": "COL",
	"ConversionSoftware": "dcm2niix",
	"ConversionSoftwareVersion": "v1.0.20220720"
}

Platform details:

Choose one:

Local environment

Container

Heudiconv version:

Running heudiconv version 0.11.6 latest 0.11.6

The text was updated successfully, but these errors were encountered:

Closes nipy#633 Somewhat of a blind fix without looking to deep why we get IndexError instead of KeyError

neurorepro · 2023-02-08T16:58:51Z

@yarikoptic I got a new error with #634 . This definitely counts as progress:

INFO: Running heudiconv version 0.11.6.post31+g24f4d88 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bids_dwi_min_newheudi/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py:525: UserWarning: Derived images found and removed
  warnings.warn('Derived images found and removed')
Traceback (most recent call last):
  File "/home/michael/.conda/envs/heudiconv310_dwi/bin/heudiconv", line 8, in <module>
    sys.exit(main())
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/dicoms.py", line 232, in group_dicoms_into_seqinfos
    if mw.is_same_series(mwgroup[idx]):
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 357, in is_same_series
    my_sig = self.series_signature
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 625, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 505, in image_shape
    first_frame = self.frames[0]
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/pydicom/multival.py", line 104, in __getitem__
    return self._list[index]
IndexError: list index out of range

neurorepro · 2023-02-08T17:40:34Z

@yarikoptic looking at the conversion of other metrics with version 0.11.6 of heudiconv (not the PR), i also got the error for TRACEW, but not for FA and ColFA (which stands for something like colored FA).

Serendipitously i noticed that the third dimension of ColFA is not properly retrieved by heudiconv which shows dim3 to be 1 instead of 94. This may be due to the image datatype (RGB24) and Siemens colored FA volumes are probably not on top of heudiconv backlog.

yarikoptic · 2023-02-08T19:49:08Z

given that we get through nibabel/nicom/dicomwrappers.py", line 357, in is_same_series -- the issue might even be in nibabel , and worth checking/filing there.

yarikoptic · 2023-08-09T19:47:13Z

also happens upon the same del mw.series_signature[sig] for

numpy.AxisError: axis 1 is out of bounds for array of dimension 1 #670
which was reported to numpy.AxisError: axis 1 is out of bounds for array of dimension 1 nibabel#1245 . @neurorepro - any way to get access to such a file?

yarikoptic · 2024-02-28T15:57:10Z

@neurorepro could you try with current fresh nibabel? current heudiconv image has it: #670 (comment)

yarikoptic added a commit to dbic/heudiconv that referenced this issue Feb 8, 2023

Also tollerate IndexError while trying to delete a field in mw

24f4d88

Closes nipy#633 Somewhat of a blind fix without looking to deep why we get IndexError instead of KeyError

yarikoptic linked a pull request Feb 8, 2023 that will close this issue

Also tolerate IndexError while trying to delete a field in mw #634

Open

yarikoptic mentioned this issue Aug 9, 2023

Unable to get attribute from Philips DICOM: Calculated shape does not match number of frames. nipy/nibabel#1211

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

neurorepro commented Feb 8, 2023 •

edited

Loading

neurorepro commented Feb 8, 2023 •

edited

Loading

neurorepro commented Feb 8, 2023

yarikoptic commented Feb 8, 2023

yarikoptic commented Aug 9, 2023

yarikoptic commented Feb 28, 2024

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

Comments

neurorepro commented Feb 8, 2023 • edited Loading

Summary

Platform details:

neurorepro commented Feb 8, 2023 • edited Loading

neurorepro commented Feb 8, 2023

yarikoptic commented Feb 8, 2023

yarikoptic commented Aug 9, 2023

yarikoptic commented Feb 28, 2024

neurorepro commented Feb 8, 2023 •

edited

Loading

neurorepro commented Feb 8, 2023 •

edited

Loading