Fix problematic header read/write for *.weights file #842
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
In the latest version 1.8.0, Darknet.save_darknet_weights() generates a problematic *.weights file, in which "minor" field is always 0, however the length of "seen" field is 8 bytes.
This may cause abnormal offset calculation when try to use official Darknet or other third-party Darknet-compatible repository (e.g. rknn-toolkit by Rockchip) to load the exported *.weights file, corrupting the whole weights and biases behind the header, then leading to totally wrong inference!
Reference to the read and write of *.weights header, from Darknet's official implementation:
void load_weights_upto(network *net, char *filename, int start, int cutoff)
(https://github.com/pjreddie/darknet/blob/master/src/parser.c#L1218)
void save_weights_upto(network *net, char *filename, int cutoff
(https://github.com/pjreddie/darknet/blob/master/src/parser.c#L1007)
Related issues
Resolves #292, and all other problems likewise caused by wrong exported *.weights are totally resolved.
However, saved module state_dict (*.pth file) does not contain the value of "seen" class member, which is 0 by default.
Convert a *.pth to *.weights will fill the "seen" field zero, losing the original "seen" from any pretrained *.weights, which is just a little problem and not related to inference BTW.
Necessary checks