Incompatibility with NumPy 2.0

[Tobias-Fischer]

Error:

loris_extension.cpp:241:42: error: no member named 'fields' in '_PyArray_Descr'

See conda-forge/loris-feedstock#8

[biphasic]

thank you Tobias but I'm afraid neither Alex nor me have the time to maintain this project at the moment. Alex wrote a new version of a parser library that works with the .es format https://github.com/neuromorphicsystems/event_stream
and also the aedat format https://github.com/neuromorphicsystems/aedat for info

[aMarcireau]

@Tobias-Fischer, @biphasic,

The problem comes from the use of direct accessors on numpy structures, since they are replaced with functions in numpy 2.0.

loris/source/cpp/loris_extension.cpp

Line 241 in f21c819

auto fields = PyArray_DESCR(events)->fields;

This is only a problem in event_stream_write since it needs to validate the input array's type. The fix recommended by numpy is pretty straightforward and I readily implemented it in event_stream (neuromorphicsystems/event_stream@960bd63 and neuromorphicsystems/event_stream@e3248b3), which uses code that is very similar to Loris's.

The C++ extension used by Loris is there to support .dat and .es. I think that this leaves us with two options to support numpy 2:

• Strip event_stream_read and event_stream_write from Loris and make it depend on https://github.com/neuromorphicsystems/event_stream. We can leave .dat functions as-is since they do not support writing.
• Port the changes made to event_stream to Loris.

Which option do you prefer?

[aMarcireau]

Another (potentially more problematic) issue for numpy 2 support is that numpy 1 and numpy 2 are not binary-compatible. Whilst we can make sure that event-stream / aedat / loris can be built with both, wheels built with GitHub Actions and shipped via PyPi or Conda need to be compiled with one or the other. The user will need to use the same version of numpy at runtime to avoid issues. We could simply mark these libraries as depending on numpy 2, but this could be an issue if users need other libraries that have not been upgraded and need numpy 1 in the same project.

I am tempted to continue compiling with numpy 1 (at least for another month or so). Users that need numpy 2 will need to compile from source with the sdist in the mean time.

[Tobias-Fischer]

Hi @aMarcireau - thanks a lot for looking into this!
Re strip vs port: I don't have a preference - it seems like a relatively minor change, so porting is probably the easier way to go?
Re binary compatibility: For conda-forge, this is being taken care of. The migrator in conda-forge/loris-feedstock#8 once merged will add a dependency on numpy 2, and thus the package going forward will not install in numpy 1 environments.

[Tobias-Fischer]

For a port see #12