Future release (1.4.x or 1.5.0 or 2.0)

[shuds13]

Target date: TBC

Question: Should gen_on_manager becomes default (what about non-persistent gens). This is very breaking hence might be for 2.0.

• Support sub-classed generators with an ask/tell interface Interface with Generators that expose an .ask/.tell interface #1240

• Make sure persistent gens return None as first parameter unless want to return something.

• Resolve Better capture dependencies for 'extra' tests. #1403 and Pydantic warnings #1404

• Improve coverage of core libE methods (if reasonably easy to do so). Files to consider

  • libE.py
    • TCP worker-side content.
  • Should all of worker.py be covered?
  • comms.py
  • mpi_resources.py exception handling
  • genfunc init where I think much might be removable.
  • gpu_detect but not if that's difficult to test/mock. Same for tcp_mgr
  • Slots in worker_resources.py

Canceled / Put on hold:

• Support generator running on the manager Support in-place gen on manager #1142 (in-place)
• Allow num workers to be set by detected GPUs Allows workers and rsets to be set by detected GPUs. #939
• Add notebook with simple optimization example - inc. simple persistent generator (similar to paper example).

Questions:

• Should use_persis_return_sim be deprecated - if so now or next release. Requires ensuring all functions update to returning None (including in docs) - which should do anyway.
• Should H0 be an argument to ensemble.run()?

[jlnav]

Make sure persistent gens return None as first parameter unless want to return something.

Out of curiosity, I think functions that return "nothing" already have their output parameter captured as None, e.g.

>>> out = print("HELLO")
>>> type(out)
<class 'NoneType'>

Within the worker, current logic, probably can be dramatically improved:

            if out:
                if len(out) >= 3:  # Out, persis_info, calc_status
                    calc_status = out[2]
                    return out
                elif len(out) == 2:  # Out, persis_info OR Out, calc_status
                    if isinstance(out[1], (int, str)):  # got Out, calc_status
                        calc_status = out[1]
                        return out[0], Work["persis_info"], calc_status
                    return *out, calc_status  # got Out, persis_info
                else:
                    return out, Work["persis_info"], calc_status
            else:
                return None, Work["persis_info"], calc_status

Looks like if a gen returns None or otherwise, the intention may be captured anyway? I dunno