Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process runs with different number of channels #326

Open
kujaku11 opened this issue May 29, 2024 · 0 comments
Open

Process runs with different number of channels #326

kujaku11 opened this issue May 29, 2024 · 0 comments

Comments

@kujaku11
Copy link
Collaborator

This maybe a bug or something the processor needs to figure out before running process_mth5 but here's a scenario that happens.

Typical instruments have multiple sample rates and different ways of saving those data to disk. Occasionally, there is a hiccup in saving and maybe one channel gets corrupt for that run. So you may have run 1 with all 5 channels, run 2 has 4 channels and run 3 has all 5 channels again.

If you try to process these 3 runs together an error will occur because the spectrograms will be different shapes for the run with the missing channel. That is because drop_nans wipes out the missing channel and all the data that in the other channels for the missing data. So if you have the missing channel is an output channel like ex then ey and hz will also be wiped out over the missing time period of ex. Then when you go to estimate weights there is a mismatch in shapes between the input and output channels because output will contain info for all 3 runs.

Because nans are taken care of in the loop on line 252 in pipelines\transfer_function_helpers.py | process_transfer_function call in line 244 pipelines\transfer_function_helpers.py | process_transfer_function | X, Y, RR = drop_nans(X, Y, RR) might be superfluous. When I comment out that line you can process the 3 runs, now you aren't exactly processing the 3 runs you are processing the 2 that have all the same channels and skipping the odd ball. There probably isn't a better way of handling this programmatically unless because drop_nans will drop all channels for which there are nans in the xr.Dataset.

Maybe there could be a check in RunSummary.check_runs_are_valid that identifies runs that are different from the rest. I guess you need at least 3 runs to do the check to find the odd one out. And just provide a warning to the user? I'll create a PR for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant