You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This maybe a bug or something the processor needs to figure out before running process_mth5 but here's a scenario that happens.
Typical instruments have multiple sample rates and different ways of saving those data to disk. Occasionally, there is a hiccup in saving and maybe one channel gets corrupt for that run. So you may have run 1 with all 5 channels, run 2 has 4 channels and run 3 has all 5 channels again.
If you try to process these 3 runs together an error will occur because the spectrograms will be different shapes for the run with the missing channel. That is because drop_nans wipes out the missing channel and all the data that in the other channels for the missing data. So if you have the missing channel is an output channel like ex then ey and hz will also be wiped out over the missing time period of ex. Then when you go to estimate weights there is a mismatch in shapes between the input and output channels because output will contain info for all 3 runs.
Because nans are taken care of in the loop on line 252 in pipelines\transfer_function_helpers.py | process_transfer_function call in line 244 pipelines\transfer_function_helpers.py | process_transfer_function | X, Y, RR = drop_nans(X, Y, RR) might be superfluous. When I comment out that line you can process the 3 runs, now you aren't exactly processing the 3 runs you are processing the 2 that have all the same channels and skipping the odd ball. There probably isn't a better way of handling this programmatically unless because drop_nans will drop all channels for which there are nans in the xr.Dataset.
Maybe there could be a check in RunSummary.check_runs_are_valid that identifies runs that are different from the rest. I guess you need at least 3 runs to do the check to find the odd one out. And just provide a warning to the user? I'll create a PR for this.
The text was updated successfully, but these errors were encountered:
This maybe a bug or something the processor needs to figure out before running
process_mth5
but here's a scenario that happens.Typical instruments have multiple sample rates and different ways of saving those data to disk. Occasionally, there is a hiccup in saving and maybe one channel gets corrupt for that run. So you may have run 1 with all 5 channels, run 2 has 4 channels and run 3 has all 5 channels again.
If you try to process these 3 runs together an error will occur because the spectrograms will be different shapes for the run with the missing channel. That is because
drop_nans
wipes out the missing channel and all the data that in the other channels for the missing data. So if you have the missing channel is anoutput
channel likeex
theney
andhz
will also be wiped out over the missing time period ofex
. Then when you go to estimate weights there is a mismatch in shapes between theinput
andoutput
channels becauseoutput
will contain info for all 3 runs.Because
nan
s are taken care of in the loop on line 252 inpipelines\transfer_function_helpers.py | process_transfer_function
call in line 244pipelines\transfer_function_helpers.py | process_transfer_function | X, Y, RR = drop_nans(X, Y, RR)
might be superfluous. When I comment out that line you can process the 3 runs, now you aren't exactly processing the 3 runs you are processing the 2 that have all the same channels and skipping the odd ball. There probably isn't a better way of handling this programmatically unless becausedrop_nans
will drop all channels for which there are nans in the xr.Dataset.Maybe there could be a check in
RunSummary.check_runs_are_valid
that identifies runs that are different from the rest. I guess you need at least 3 runs to do the check to find the odd one out. And just provide a warning to the user? I'll create a PR for this.The text was updated successfully, but these errors were encountered: