multi_fast5 fails schema

[callumparr]

I was interested to see fast5 files before and after VBZ compression would pass h5 validator.

I first tested on output basecalled fast5 from MinKNOW from December 2019. It reported 20,000 errors from on multi_fast5. Is this because the schema hasn't been update for a while?

[fbrennen]

Hi @callumparr -- thanks for the report. Could you show us a few of the errors? It has been a while since we updated the schema so a few changes may have crept in.

It's worth noting that the validator will likely not work on VBZ-compressed files unless the plugin is installed separately. We should call this out in the documentation and we haven't.

[callumparr]

Hi sorry I forgot to reply to this. This is from a sample multi_fast5 data set for testing master of pores pipeline but I got same error with my own data set.

Sample here:

https://github.com/biocorecrg/master_of_pores/tree/master/data/multifast

I have the ONT VBZ plugin installed and export the plugin library variable thing.

What does the validator check? Just that it is formatted correctly for single or multi or also things like if squiggles are written or just raw?

`
h5_validate ont_h5_validator/h5_validator/schemas/multi_read_fast5.yaml master_of_pores/data/multifast/multifast5_1.fast5 -v
/home/minion/miniconda3/lib/python3.6/site-packages/ont_h5_validator-2.0.1-py3.6.egg/h5_validator/schema.py:43: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
Validation encountered 100 errors in master_of_pores/data/multifast/multifast5_1.fast5

Error at /read_00666bd7-121a-4194-9e6c-a6556286138c/PreviousReadInfo:
Failed to match /read_00666bd7-121a-4194-9e6c-a6556286138c/PreviousReadInfo to item in schema
Error at /read_089a34bb-8c02-4fed-8214-790868062e9f/PreviousReadInfo:
Failed to match /read_089a34bb-8c02-4fed-8214-790868062e9f/PreviousReadInfo to item in schema
Error at /read_094f1315-ed28-4659-9aa9-f1a0b05735d8/PreviousReadInfo:
Failed to match /read_094f1315-ed28-4659-9aa9-f1a0b05735d8/PreviousReadInfo to item in schema
Error at /read_0a1d4b39-e15a-4b3e-90f7-2de7ecbd2638/PreviousReadInfo:
Failed to match /read_0a1d4b39-e15a-4b3e-90f7-2de7ecbd2638/PreviousReadInfo to item in schema
Error at /read_0d752c10-05c6-4a72-a0f3-6c65b56f776f/PreviousReadInfo:
Failed to match /read_0d752c10-05c6-4a72-a0f3-6c65b56f776f/PreviousReadInfo to item in schema
Error at /read_0e46b21e-c95f-4253-b03c-ee90b658e99a/PreviousReadInfo:
Failed to match /read_0e46b21e-c95f-4253-b03c-ee90b658e99a/PreviousReadInfo to item in schema
Error at /read_10f54f83-5c8f-4d8e-a493-26653ea4ea50/PreviousReadInfo:
Failed to match /read_10f54f83-5c8f-4d8e-a493-26653ea4ea50/PreviousReadInfo to item in schema
Error at /read_1198182b-e7b9-498e-a54f-be8b78385deb/PreviousReadInfo:
Failed to match /read_1198182b-e7b9-498e-a54f-be8b78385deb/PreviousReadInfo to item in schema
Error at /read_124b76d8-ebc0-4069-a027-1d01240c840e/PreviousReadInfo:
Failed to match /read_124b76d8-ebc0-4069-a027-1d01240c840e/PreviousReadInfo to item in schema
Error at /read_194e05c0-3dc5-4b08-b97b-77e7406386bd/PreviousReadInfo:
Failed to match /read_194e05c0-3dc5-4b08-b97b-77e7406386bd/PreviousReadInfo to item in schema
Error at /read_1c5f8f50-83f1-41ab-9d0f-77fed47f05b3/PreviousReadInfo:
Failed to match /read_1c5f8f50-83f1-41ab-9d0f-77fed47f05b3/PreviousReadInfo to item in schema
Error at /read_1db10d73-2486-45b9-a26b-ee192d6e6f40/PreviousReadInfo:
Failed to match /read_1db10d73-2486-45b9-a26b-ee192d6e6f40/PreviousReadInfo to item in schem
`