Lazy-load compression filters in H5WasmProvider
#1522
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks to the new h5wasm-plugins package (kudos @bmaranville!!), I'm able to load compression plugins on demand in
H5WasmProvider
. 🎉I was hoping that everything could be taken care of by
H5WasmProvider
under the hood, but since@h5web/h5wasm
is a library, it's not that simple. The plugin files have to be loaded into Emscripten's file system as array buffers, which means they first have to be fetched lazily somehow ... and only the consumer application knows how to do that:fetch
altogether (e.g. VS Code webview).The solution I came up with is to expose a
getPlugin
prop. If the prop is implemented by the consumer app, then it will be called with the name of a known plugin when the time comes to load it into Emscripten. It is then up to the consumer application to return an array buffer (orundefined
if the plugin is not supported).H5WasmProvider
holds a mapping of known plugins against their corresponding HDF5 filter IDs. For now, this list will need to be kept up to date as more plugins are added toh5wasm-plugins
. In the future, we could add a prop to let consumers declare additional plugins without having to wait for H5Web updates.This PR brings support for the following 7 compression filters in the h5wasm demo: Blosc, bzip2, LZ4, LZF, SZ, ZFP and Zstandard. You can try it with this file, which comes from the
h5wasm-plugins
repo: https://www.silx.org/pub/h5web/filters.h5