[Core] Project-based multistage

[rubenzorrilla]

📝 Description
This PR adds the upgraded version of our current multistage capabilities. The enhancements can be summarized into refactoring the code to be much more flexible/extensible and adding checkpointing (i.e. restarting) between stages.

The main changes are the following

• Creation of the (deliberately) pure python Project container. This new object is conceived to hold all the simulation data, that is to say the Model, the settings, the active stages dictionary and the stages output data dictionary (see [FastPR][Core] Add GetAnalysisStageFinalData to AnalysisStage #11335). Note that it is also in charge of doing the Save and Load of the multistage.
• The multistage_analysis.py is now the multistage_orchestrator.py. We conceived the orchestrators as the class that knows the multistage simulation workflow. Note that this implies knowing at which points we are calling the Project save and load in order to do so.
• Creation of a derived sequential_multistage_orchestrator.py that does the standard one-after-each-other (note that this emulates our current multistage_analysis.py. On top of that, we added some features to it that we believe could help usability such as the output of the project parameters to be used together with each restarting file.

There are also some minor changes that we require in order to make this work. There are

• Registering the orchestrators as well as the analysis stage. I note that we still support the creation of unregistered stages but throwing a warning.
• Adding a Save and Load method to current analysis_stage.py. This will remain empty by default as the current SequentialMultistageOrchestrator does the checkpointing between stages or, in other words, no stage instances are pickled. Nonetheless, this might not be the case in the future or in some forks, so we are providing the interface to achieving so.

As a final note I'd like to add that these changes require some minor modifications in our current multistage MainKratos.py and I/O. Having said this, I'll update current examples accordingly once this PR is merged.

[philbucher]

I have a couple of comments, but only minor, will address them in the next days

Looking good overall, happy to see that this makes progress!

[loumalouomega]

Okay from my side. It would be nice to have a test or at least an example in the Example repo

[rubenzorrilla]

@philbucher @pooyan-dadvand I went through all your requirements / suggestions. Besides I added a test running a "fake" multistage workflow.

Last but not least I'm sharing a working "real" example in here so you can see how the MainKratos.py will look like. You can execute it with python3 MainKratos.py ProjectParametersCustom.json. I'll update the examples repository and documentation accordingly once this is merged.

1x1_square_multistage_project.zip

[pooyan-dadvand]

Just to add that, going through your example, adding the stage setting make it much more readable than without!

[rubenzorrilla]

Just to add that, going through your example, adding the stage setting make it much more readable than without!

Agree. Besides, it makes easier the inclusion of already existing json files.

[loumalouomega]

abc?

[loumalouomega]

I see https://docs.python.org/3/library/abc.html

[loumalouomega]

Removing my block

[pooyan-dadvand]

Thanks @rubenzorrilla for considering the suggestions.

I am unblocking as the pickleable problem would not affect the current implementation.