Updates to setup.py

CHANGELOG.rst

@@ -8,6 +8,26 @@ GitHub issues are referenced, and can be viewed with hyperlinks on the `github r
 
 .. _`github releases page`: https://github.com/Libensemble/libensemble/releases
 
+Release 0.10.2
+--------------
+
+:Date: July 20, 2023
+
+* Fix issues with workflow dirs:
+  * Ensure relative paths are interpreted from where calling script is run. #1020
+  * Create intermediate directories for workflow paths. #1017
+
+* Fixes issue where libEnsemble pre-initialized a shared multiprocssing queue. #1026
+
+:Note:
+
+* Tested platforms include Linux, MacOS, Windows and major systems including Frontier (OLCF), Polaris (ALCF), Perlmutter (NERSC), Theta (ALCF) and Bebop. The major system tests ran heterogeneous workflows.
+
+:Known issues:
+
+* On systems using SLURM 23.02, some issues have been experienced when using ``mpi4py`` comms.
+* See the known issues section in the documentation for more information (https://libensemble.readthedocs.io/en/main/known_issues.html).
+
 Release 0.10.1
 --------------
 
@@ -36,16 +56,16 @@ Release 0.10.0
 New capabilities:
 
 * Enhance portability and simplify the assignment of procs/GPUs to worker resources #928 / #983
- * Auto-detect GPUs across systems (inc. Nvidia, AMD, and Intel GPUs).
- * Auto-determination of GPU assignment method by MPI runner or provided platform.
- * Portable `auto_assign_gpus` / `match_procs_to_gpus` and `num_gpus` arguments added to the MPI executor submit.
- * Add `set_to_gpus` function (similar to `set_to_slots`).
- * Allow users to specify known systems via option or environment variable.
- * Allow users to specify their own system configurations.
- * These changes remove a number of tweaks that were needed for particular platforms.
+  * Auto-detect GPUs across systems (inc. Nvidia, AMD, and Intel GPUs).
+  * Auto-determination of GPU assignment method by MPI runner or provided platform.
+  * Portable `auto_assign_gpus` / `match_procs_to_gpus` and `num_gpus` arguments added to the MPI executor submit.
+  * Add `set_to_gpus` function (similar to `set_to_slots`).
+  * Allow users to specify known systems via option or environment variable.
+  * Allow users to specify their own system configurations.
+  * These changes remove a number of tweaks that were needed for particular platforms.
 
 *  Resource management supports GPU and non-GPU simulations in the same ensemble. #993
- * User's can specify `num_procs` and `num_gpus` in the generator for each evaluation.
+  * User's can specify `num_procs` and `num_gpus` in the generator for each evaluation.
 
 * Pydantic models are used for validating major libE input (input can be provided as classes or dictionaries). #878
 * Added option to store output and ensemble directories in a workflow directory. #982
@@ -71,10 +91,10 @@ Documentation:
 Tests and Examples:
 
 * Updated forces_gpu tutorial example. #956
- * Source code edit is not required for the GPU version.
- * Reports whether running on device or host.
- * Increases problem size.
- * Added versions with persistent generator and multi-task (GPU v non-GPU).
+  * Source code edit is not required for the GPU version.
+  * Reports whether running on device or host.
+  * Increases problem size.
+  * Added versions with persistent generator and multi-task (GPU v non-GPU).
 * Moved multiple tests, generators, and simulators to the community repo.
 * Added ytopt example. And updated heFFTe example. #943
 * Support Python 3.11 #922

README.rst

@@ -1,4 +1,4 @@
-.. image:: docs/images/libEnsemble_Logo.svg
+.. image:: https://raw.githubusercontent.com/Libensemble/libensemble/main/docs/images/libE_logo.png
    :align: center
    :alt: libEnsemble
 
@@ -7,6 +7,14 @@
 .. image:: https://img.shields.io/pypi/v/libensemble.svg?color=blue
    :target: https://pypi.org/project/libensemble
 
+.. image:: https://img.shields.io/conda/v/conda-forge/libensemble?color=blue
+   :target: https://anaconda.org/conda-forge/libensemble
+
+.. image:: https://img.shields.io/spack/v/py-libensemble?color=blue
+   :target: https://spack.readthedocs.io/en/latest/package_list.html#py-libensemble
+
+|
+
 .. image:: https://github.com/Libensemble/libensemble/workflows/libEnsemble-CI/badge.svg?branch=main
    :target: https://github.com/Libensemble/libensemble/actions
 

docs/advanced_installation.rst

@@ -9,8 +9,9 @@ automatically installed alongside libEnsemble:
 * Python_ 3.8 or above
 * NumPy_
 * psutil_
-* setuptools_
 * pydantic_
+* pyyaml_
+* tomli_
 
 In view of libEnsemble's compiled dependencies, the following installation
 methods each offer a trade-off between convenience and the ability
@@ -178,7 +179,8 @@ often include the system MPI library.
 .. _`Open MPI`: https://www.open-mpi.org/
 .. _psutil: https://pypi.org/project/psutil/
 .. _pydantic: https://pydantic-docs.helpmanual.io/
+.. _pyyaml: https://github.com/yaml/pyyaml
 .. _Python: http://www.python.org
-.. _setuptools: https://setuptools.pypa.io/en/latest/
 .. _Spack: https://spack.readthedocs.io/en/latest
 .. _spack_libe: https://github.com/Libensemble/spack_libe
+.. _ tomli: https://github.com/hukkin/tomli

docs/known_issues.rst

@@ -4,9 +4,15 @@ Known Issues
 The following selection describes known bugs, errors, or other difficulties that
 may occur when using libEnsemble.
 
-* As of 10/13/2022, on Perlmutter there was an issue running concurrent applications
-  on a node, following a recent system update. This also affects previous versions
-  of libEnsemble, and is being investigated.
+* Platforms using SLURM version 23.02 experience a `pickle error`_ when using
+  ``mpi4py`` comms. Disabling matching probes via the environment variable
+  ``export MPI4PY_RC_RECV_MPROBE=0`` or adding ``mpi4py.rc.recv_mprobe = False``
+  at the top of the calling script should resolve this error. If using the MPI
+  executor and multiple workers per node, some users may experience failed
+  applications with the message
+  ``srun: error: CPU binding outside of job step allocation, allocated`` in
+  the application's standard error. This is being investigated. If this happens
+  we recommend using ``local`` comms in place of ``mpi4py``.
 * When using the Executor: OpenMPI does not work with direct MPI task
   submissions in mpi4py comms mode, since OpenMPI does not support nested MPI
   executions. Use either ``local`` mode or the Balsam Executor instead.
@@ -23,3 +29,5 @@ may occur when using libEnsemble.
   :doc:`FAQ<FAQ>` for more information.
 * We currently recommended running in Central mode on Bridges as distributed
   runs are experiencing hangs.
+
+.. _pickle error: https://docs.nersc.gov/development/languages/python/using-python-perlmutter/#missing-support-for-matched-proberecv

docs/platforms/polaris.rst

@@ -42,12 +42,15 @@ for installing libEnsemble, including using Spack.
 Ensuring use of mpiexec
 -----------------------
 
-If using the :doc:`MPIExecutor<../executor/mpi_executor>` it is recommended to
-ensure you are using ``mpiexec`` instead of ``aprun``. When setting up the executor use::
+Prior to libE v 0.10.0, when using the :doc:`MPIExecutor<../executor/mpi_executor>` it
+is necessary to manually tell libEnsemble to use``mpiexec`` instead of ``aprun``.
+When setting up the executor use::
 
     from libensemble.executors.mpi_executor import MPIExecutor
     exctr = MPIExecutor(custom_info={'mpi_runner':'mpich', 'runner_name':'mpiexec'})
 
+From version 0.10.0, this is not necessary.
+
 Job Submission
 --------------
 

docs/tutorials/executor_forces_tutorial.rst

@@ -2,31 +2,23 @@
 Executor with Electrostatic Forces
 ==================================
 
-This tutorial highlights libEnsemble's capability to execute
+This tutorial highlights libEnsemble's capability to portably execute
 and monitor external scripts or user applications within simulation or generator
-functions using the :doc:`executor<../executor/overview>`. In this tutorial,
-our calling script registers a compiled executable that simulates
+functions using the :doc:`executor<../executor/overview>`.
+
+In this tutorial, our calling script registers a compiled executable that simulates
 electrostatic forces between a collection of particles. The simulator function
 launches instances of this executable and reads output files to determine
 if the run was successful.
 
-It is possible to use ``subprocess`` calls from Python to issue
-commands such as ``jsrun`` or ``aprun`` to run applications. Unfortunately,
-hard-coding such commands within user scripts isn't portable.
-Furthermore, many systems like Argonne's :doc:`Theta<../platforms/theta>` do not
-allow libEnsemble to submit additional tasks from the compute nodes. On these
-systems, a proxy launch mechanism (such as Balsam) is required.
-libEnsemble's Executors were developed to directly address such issues.
-
-In particular, we'll be experimenting with
-libEnsemble's :doc:`MPI Executor<../executor/mpi_executor>`, since it can automatically
-detect available MPI runners and resources, and by default divide them equally among workers.
+This tutorial uses libEnsemble's :doc:`MPI Executor<../executor/mpi_executor>`,
+since it can automatically detect available MPI runners and resources.
 
 Getting Started
 ---------------
 
 The simulation source code ``forces.c`` can be obtained directly from the
-libEnsemble repository here_.
+libEnsemble repository in the forces_app_ directory.
 
 Assuming MPI and its C compiler ``mpicc`` are available, compile
 ``forces.c`` into an executable (``forces.x``) with:
@@ -35,9 +27,14 @@ Assuming MPI and its C compiler ``mpicc`` are available, compile
 
     $ mpicc -O3 -o forces.x forces.c -lm
 
+Alternative build lines for difference platforms can be found in the ``build_forces.sh``
+file in the same directory.
+
 Calling Script
 --------------
 
+Complete scripts for this example can be found in the forces_simple_ directory.
+
 Let's begin by writing our calling script to parameterize our simulation and
 generation functions and call libEnsemble. Create a Python file called `run_libe_forces.py`
 containing:
@@ -66,13 +63,11 @@ containing:
     sim_app = os.path.join(os.getcwd(), "../forces_app/forces.x")
     exctr.register_app(full_path=sim_app, app_name="forces")
 
-On line 15, we instantiate our :doc:`MPI Executor<../executor/mpi_executor>` class instance,
-which can optionally be customized by specifying alternative MPI runners. The
-auto-detected default should be sufficient.
+On line 15, we instantiate our :doc:`MPI Executor<../executor/mpi_executor>`.
 
 Registering an application is as easy as providing the full file-path and giving
-it a memorable name. This Executor instance will later be retrieved within our
-simulation function to launch the registered app.
+it a memorable name. This Executor will later be retrieved within our simulation
+function to launch the registered app.
 
 Next define the :ref:`sim_specs<datastruct-sim-specs>` and
 :ref:`gen_specs<datastruct-gen-specs>` data structures. Recall that these
@@ -225,12 +220,10 @@ for starters:
 
 We retrieve the generated number of particles from ``H`` and construct
 an argument string for our launched application. The particle count doubles up
-as a random number seed here. Note a fourth argument can be added to forces
-that gives forces a chance of a "bad run" (a float between 0 and 1), but
-for now that will default to zero.
+as a random number seed here.
 
-We then retrieve our previously instantiated Executor instance from the
-class definition, where it was automatically stored as an attribute.
+We then retrieve our previously instantiated Executor from the class definition,
+where it was automatically stored as an attribute.
 
 After submitting the "forces" app for execution,
 a :ref:`Task<task_tag>` object is returned that correlates with the launched app.
@@ -283,10 +276,10 @@ This completes our calling script and simulation function. Run libEnsemble with:
 
     $ python run_libe_forces.py --comms local --nworkers [nworkers]
 
-This may take up to a minute to complete. Output files---including ``forces.stat``
-and files containing ``stdout`` and ``stderr`` content for each task---should
-appear in the current working directory. Overall workflow information
-should appear in ``libE_stats.txt`` and ``ensemble.log`` as usual.
+Output files---including ``forces.stat`` and files containing ``stdout`` and
+``stderr`` content for each task---should appear in the current working
+directory. Overall workflow information should appear in ``libE_stats.txt``
+and ``ensemble.log`` as usual.
 
 For example, my ``libE_stats.txt`` resembled::
 
@@ -338,6 +331,8 @@ Each of these example files can be found in the repository in `examples/tutorial
 For further experimentation, we recommend trying out this libEnsemble tutorial
 workflow on a cluster or multi-node system, since libEnsemble can also manage
 those resources and is developed to coordinate computations at huge scales.
+See ref:`HPC platform guides<platform-index>` for more information.
+
 Please feel free to contact us or open an issue on GitHub_ if this tutorial
 workflow doesn't work properly on your cluster or other compute resource.
 
@@ -386,6 +381,7 @@ These may require additional browsing of the documentation to complete.
 
         ...
 
-.. _here: https://raw.githubusercontent.com/Libensemble/libensemble/main/libensemble/tests/scaling_tests/forces/forces.c
+.. _forces_app: https://github.com/Libensemble/libensemble/tree/main/libensemble/tests/scaling_tests/forces/forces_app
+.. _forces_simple: https://github.com/Libensemble/libensemble/tree/main/libensemble/tests/scaling_tests/forces/forces_simple
 .. _examples/tutorials/forces_with_executor: https://github.com/Libensemble/libensemble/tree/develop/examples/tutorials/forces_with_executor
 .. _GitHub: https://github.com/Libensemble/libensemble/issues

libensemble/tests/scaling_tests/forces/forces_gpu/run_libe_forces.py

@@ -27,14 +27,9 @@
 from libensemble.libE import libE
 from libensemble.tools import add_unique_random_streams, parse_args
 
-# Uncomment for var resources
-# from libensemble.gen_funcs.sampling import uniform_random_sample_with_variable_resources as gen_f
-
-
 # Uncomment for var resources (checksum will change due to rng differences)
 # from libensemble.gen_funcs.sampling import uniform_random_sample_with_variable_resources as gen_f
 
-
 # Parse number of workers, comms type, etc. from arguments
 nworkers, is_manager, libE_specs, _ = parse_args()
 

libensemble/version.py

@@ -1 +1 @@
-__version__ = "0.10.1+dev"
+__version__ = "0.10.2"

setup.py

@@ -14,11 +14,10 @@
 
 """
 
+from pathlib import Path
 from setuptools import setup
 from setuptools.command.test import test as TestCommand
 
-DOCLINES = (__doc__ or "").split("\n")
-
 exec(open("libensemble/version.py").read())
 
 
@@ -49,7 +48,7 @@ def run_tests(self):
     name="libensemble",
     version=__version__,
     description="Library to coordinate the concurrent evaluation of dynamic ensembles of calculations",
-    long_description="\n".join(DOCLINES[2:]),
+    long_description=Path("README.rst").read_text(encoding="utf-8"),
     url="https://github.com/Libensemble/libensemble",
     author="Jeffrey Larson, Stephen Hudson, Stefan M. Wild, David Bindel and John-Luke Navarro",
     author_email="[email protected]",
@@ -69,7 +68,7 @@ def run_tests(self):
         "libensemble.tests.unit_tests",
         "libensemble.tests.regression_tests",
     ],
-    install_requires=["numpy", "psutil", "setuptools", "pydantic<2", "tomli", "pyyaml"],
+    install_requires=["numpy", "psutil", "pydantic<2", "tomli", "pyyaml"],
     # If run tests through setup.py - downloads these but does not install
     tests_require=[
         "pytest>=3.1",