Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't convert boolean enums unless JSON compatibility is requested #76

Merged
merged 1 commit into from
Aug 6, 2024

Conversation

axelboc
Copy link
Collaborator

@axelboc axelboc commented Aug 2, 2024

H5Web supports plotting boolean datasets:

image

Currently, h5wasm converts boolean enum arrays, which are read as numeric typed arrays by the HDF5 library, to plain JS boolean arrays. This means that in H5Web, I have to convert those boolean arrays back to numeric arrays before I can plot them.

In this PR, I'm proposing to no longer convert boolean enum arrays when json_compatible is false - i.e. to convert them only when JSON-compatible data is requested.

@axelboc axelboc changed the title Keep boolean enums as unit8 unless JSON compatibility is requested Keep boolean enums as uint8 unless JSON compatibility is requested Aug 2, 2024
@axelboc axelboc changed the title Keep boolean enums as uint8 unless JSON compatibility is requested Keep boolean enums as unt8 unless JSON compatibility is requested Aug 2, 2024
@axelboc axelboc changed the title Keep boolean enums as unt8 unless JSON compatibility is requested Keep boolean enums as int8 unless JSON compatibility is requested Aug 2, 2024
@axelboc axelboc changed the title Keep boolean enums as int8 unless JSON compatibility is requested Don't convert boolean enums unless JSON compatibility is requested Aug 2, 2024
@axelboc
Copy link
Collaborator Author

axelboc commented Aug 2, 2024

Sorry for the spam, I'm struggling to run the tests locally. Also, the typed array returned by HDF5 lib depends on the enum type's underlying base type, which seems to be int8 and not uint8 for files written with h5py.

@bmaranville
Copy link
Member

bmaranville commented Aug 2, 2024

To run tests locally, you can use a conda environment (specifically using conda-forge):

mamba create -n h5wasm -c conda-forge --override-channels nodejs emsdk make cmake
mamba activate h5wasm
emsdk install 3.1.46
emsdk activate 3.1.46
#  follow instructions that are printed from the "activate" command to add emscripten paths, e.g.
#  source "/Users/bbm/mambaforge/envs/h5wasm/lib/python3.12/site-packages/emsdk/emsdk_env.sh"
make
npm i
npm run build
npm test

the emscripten compile will be a little slow the first time, because it's compiling and caching a bunch of system libraries.

@bmaranville
Copy link
Member

For your changes in this PR, you will only need to run make once, since you're only touching the .ts files.

@axelboc
Copy link
Collaborator Author

axelboc commented Aug 2, 2024

Thanks, what's weird is that I already have a working environment that allows me to run make and npm run build without issue. It's when I run npm test (from the root of the repo) that things go AWOL. HDF5 complains that it can't open any of the test files:

 #007: /__w/libhdf5-wasm/libhdf5-wasm/build/1.14.2/_deps/hdf5-src/src/H5FDsec2.c line 326 in H5FD__sec2_open(): unable to open file: name = 'test/empty.h5', errno = 44, error message = 'No such file or directory', flags = 0, o_flags = 0

The files do exist, obviously, so not sure what's going on.

@axelboc
Copy link
Collaborator Author

axelboc commented Aug 2, 2024

Ah all good, I had made changes to CMakeList.txt a while back when investigating something (can't remember what). Anyway, reverting and running make && npm run build fixed it. Sorry for the bother.

@bmaranville
Copy link
Member

Is this one ready for review?

@axelboc
Copy link
Collaborator Author

axelboc commented Aug 5, 2024

Yep, sorry :)

@bmaranville bmaranville self-requested a review August 6, 2024 14:28
Copy link
Member

@bmaranville bmaranville left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I think your team requested the conversion to boolean in the first place, I'm happy to make this subtle change; if you wouldn't mind adding a note to the CHANGELOG.md indicating the new behavior I'd appreciate it. (feel free to add a section for v0.7.6 - we can change the version number later if breaking metadata changes are included into something like v0.8.0)

@axelboc
Copy link
Collaborator Author

axelboc commented Aug 6, 2024

No worries, done!

@bmaranville bmaranville merged commit dc3f592 into usnistgov:main Aug 6, 2024
1 check passed
@axelboc axelboc deleted the no-bool-cast branch August 29, 2024 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants