Enhanced provenance tracking?

[johnomotani]

I added some extra outputs to STORM that track the git diff, CMake cache, loaded modules (on systems with modules), and state of BOUT-configs/STORM-configs (if used). Shouldn't be much work to copy over to Hermes-3.

The system was based on a Python script
https://github.com/boutproject/STORM/blob/master/build_tools/save_git_version.py
that generated a storm_version.hxx header file with constexpr values that could be added to the dump files
https://github.com/boutproject/STORM/blob/b85c514cb582f8ce152f5fb2828aff2132279cf7/storm3d/storm.cxx#L40-L47
CMake would use that Python script as part of the build process
https://github.com/boutproject/STORM/blob/b85c514cb582f8ce152f5fb2828aff2132279cf7/CMakeLists.txt#L17-L38

[bendudson]

Hey @johnomotani This would be great! Is there any way it can be added to BOUT++ in a way that makes this easy to add to multiple models? It would be good to have this capability everywhere.

There is tracking of the git version https://github.com/bendudson/hermes-3/blob/master/CMakeLists.txt#L36
using CMake script https://github.com/bendudson/hermes-3/blob/master/cmake/GetGitRevisionDescription.cmake

[johnomotani]

I've noticed that, at least with the way I'd set it up, you need the right Python packages (mostly gitpython if I remember right) installed at configure time. So it does increase the complexity, which is not great, especially as part of BOUT++ - maybe if it was an opt-in with some CMake option, whose documentation could tell you to make sure Python is set up correctly?
The stuff with configuration repos, as far as I can see, has to make fairly strong assumptions about what your installation looks like, so again might be tricky to generalise enough for BOUT++.
In general though, I agree all this stuff should go as far upstream as possible...

PS I have no time to do this, it came to mind as I was writing a Marconi proposal where we're transitioning from STORM to Hermes-3, and the old proposal had mentioned provenance tracking. The links are here to help anyone who wants to volunteer to pick it up!

[bendudson]

Thanks @johnomotani ! Understandable that you have no time; thanks very much for the links.