Merge pull request #270 from spc-group/toml

Refactored instrument loading into an Instrument() class

docs/conf.py

@@ -37,7 +37,7 @@
     "autoapi.sphinx",
     "sphinx.ext.napoleon",
     "sphinx.ext.todo",
-    "nbsphinx",
+    # "nbsphinx",
 ]
 
 autoapi_modules = {"haven": None}
@@ -48,17 +48,17 @@
 napoleon_numpy_docstring = True
 
 # Add custom pre-amble to the rendered jupyter notebooks
-nbsphinx_prolog = r"""
+# nbsphinx_prolog = r"""
 
-This file is also available as an interactive jupyter notebook.
+# This file is also available as an interactive jupyter notebook.
 
-`Download this file as a notebook`_ (hint: right-click -> "save as").
+# `Download this file as a notebook`_ (hint: right-click -> "save as").
 
-.. _Download this file as a notebook: {{ env.doc2path(env.docname) }}
+# .. _Download this file as a notebook: {{ env.doc2path(env.docname, base=None)|string }}
 
-"""
+# """
 
-nbsphinx_epilog = nbsphinx_prolog
+# nbsphinx_epilog = nbsphinx_prolog
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]

docs/topic_guides/area_detectors.rst

@@ -4,13 +4,13 @@ Area Detectors and Cameras
 
 Area detectors are all largely the same but with small variations from
 device-to-device. All the device definitions for area detectors are in
-the :py:mod:`haven.instrument.area_detector` module.
+the :py:mod:`haven.devices.area_detector` module.
 
 Currently supported detectors:
 
-- Eiger 500K (:py:class:`~haven.instrument.area_detector.Eiger500K`)
-- Lambda (:py:class:`~haven.instrument.area_detector.Lambda250K`)
-- Simulated detector (:py:class:`~haven.instrument.area_detector.SimDetector`)
+- Eiger 500K (:py:class:`~haven.devices.area_detector.Eiger500K`)
+- Lambda (:py:class:`~haven.devices.area_detector.Lambda250K`)
+- Simulated detector (:py:class:`~haven.devices.area_detector.SimDetector`)
 
 EPICS and Ophyd do not make a distinction between area detectors and
 cameras. After all, a camera is just an area detector for visible
@@ -39,7 +39,7 @@ the device later from the instrument registry.
 The key "prefix" should list the IOC prefix, minus the trailing
 ":". The key "device_class" should point to a subclass of ophyd's
 :py:class:`~ophyd.areadetector.detectors.DetectorBase` class that is
-defined in :py:mod:`haven.instrument.area_detector`.
+defined in :py:mod:`haven.devices.area_detector`.
 
 
 .. code-block:: toml

docs/topic_guides/beamline_configuration.rst

@@ -1,105 +1,141 @@
-###################
-Configuration Files
-###################
+#########################
+Instrument Configuration
+#########################
 
 .. contents:: Table of Contents
     :depth: 3
 
+This page describes the procedure for defining the beamline
+configuration. Haven contains definitions for many Ophyd and
+Ophyd-async devices, however **Haven needs a beamline configuration
+file** to know which specific devices are needed for each beamline.
+
+These files should be **listed in the environmental variable**
+``HAVEN_CONFIG_FILES`` as a semi-colon separated list (e.g. ``export
+HAVEN_CONFIG_FILES=$HOME/bluesky/iconfig.toml:/local/bluesky/iconfig_extra.toml``).
+
+Then the devices defined in these files can be loaded in python:
+
+.. code-block:: python
+
+    from haven import beamline
+    await beamline.load()
+
+Once the beamline has been loaded, the devices are available using an
+Ophyd registry attached to the beamline object. For example,
+``beamline.registry["austin"]`` would return an Ophyd device instance
+named *"austin"*, and ``beamline.registry.findall("ion_chambers")``
+would return all devices with the "ion_chambers" Ophyd label.
+
+
 Motivation
 ----------
 
 Haven's goal is to **provide support for all of the spectroscopy
 beamlines**. However, each beamline is different, and these
 differences are **managed by a set of configuration files**, similar
-to the .ini files used in the old LabView solution. To keep the
+to the .ini files used in the old LabView applications. To keep the
 complexity of these configuration files manageable, Haven gets much of
 the needed information from the IOCs directly.
 
+The job of processing the configuration files is handled by the
+:py:class:`~haven.instrument.Instrument` class. This class keeps track
+of the configuration file schema, as well as the resulting devices.
+
 Haven/Firefly should always load without a specific configuration
 file, but will probably not do anything useful.
 
-Checking Configuration
-----------------------
 
-If Haven is installed with pip, the command ``haven_config`` can be
-used to read configuration variables as they will be seen by Haven:
+Device Definitions
+------------------
 
-.. code:: bash
+The beamline instrument loader can either instantiate ophyd devices
+directly, or using factory functions.
 
-	  $ haven_config beamline
-	  {'is_connected': False, 'name': 'SPC Beamline (sector unknown)'}
-	  $ haven_config beamline.is_connected
-	  False
 
+Simple Devices
+^^^^^^^^^^^^^^
 
-Configuration File Priority
----------------------------
+Each device class has an entry in the
+:py:object:`~haven.instrument.beamline` loader. To create a new
+device, add a table to the configuration file for each device instance
+to create. The keys in the table should correspond to arguments passed
+to the device's ``__init__()`` method.
 
-There are several sources of configuration files, described in detail
-below. They are loaded in the following order, with lower numbers
-taking precedence over higher numbers.
+Typically, the key for the table is the joined-lower version of the
+class name. For example, an instance of the
+:py:class:`~haven.devices.mirrors.HighHeatLoadMirror` device class
+would be added to the configuration file as:
 
-1. Files listed in the ``$HAVEN_CONFIG_FILES``
-2. ``~/bluesky/instrument/iconfig.toml`` (for backwards compatibility)
-3. ``~/bluesky/iconfig.toml`` (best place)
-4. ``iconfig_default.toml`` packaged with Haven
+.. code-block:: toml
 
-Unless there's a good reason to do otherwise, **most beamline
-configuration belongs in ~/bluesky/iconfig.toml**.
+   [[ high_heat_load_mirror ]]
+   name = "ORM1"
+   prefix = "255ida:ORM1:"
+   bendable = false
 
-For example, to enable support for our Universal Robotics robot
-*Austin* to 25-ID-C, open the file ``~/bluesky/iconfig.toml`` and add
-the following:
+The instrument loader will then create a new device as
+``HighHeatLoadMirror(name="ORM1", prefix="255ida:", bendable=False)``.
 
-.. code:: toml
-   
-   [robot.Austin]
-   prefix = "25idAustin"
+The resulting device can then be retrieved from the beamline
+instrument registry: ``beamline.registry["ORM1"]``.
 
 .. note::
 
-   The prevent accidental changes, the bluesky configuration files may
-   not be writable by the user accounts at the beamline. For example,
-   at 25-ID, the user account does not have permission to write to
-   ``~/bluesky/iconfig.toml`` so **changes must be made as the staff
-   account**.
-
-``HAVEN_CONFIG_FILES`` Environment
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    The Ophyd registry allows looking up devices by Ophyd
+    label. E.g. ``beamline.registry.findall("ion_chambers")`` will
+    retrieve all devices with *"ion_chambers"* in its labels.
+
+    The instrument loader itself does not handle labels. In most
+    cases, reasonable defaults should be set by the device's
+    ``__init__()`` methods, however for more control the device table
+    could also contain the *labels* key, with the beamline then being
+    responsible for ensuring these labels are correct.
+
+    For example, the following device would be accesible by
+    ``registry['I0']`` and ``registry.findall("detectors")``, but not
+    by ``registry.findall(["ion_chambers"])``
 
-If the environmental variable ``HAVEN_CONFIG_FILES`` is set to a
-*comma-separated* list of file path, then these files will take
-priority, with later entries superseding earlier entries.
+    .. code-block:: toml
 
+        [[ ion_chamber ]]
+	name = "I0"
+	...
+	labels = ["detectors"]
 
-``~/bluesky/iconfig.toml``
-^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The file ``~/bluesky/iconfig.toml`` will be read if it is
-present. **This is the best place to put beamline-specific
-configuration.**
+Factory Functions
+^^^^^^^^^^^^^^^^^
 
-The file ``~/bluesky/instrument/iconfig.toml`` is also read for
-backwards compatibility. It should not be used for new deployments,
-and support for it may be removed without warning.
+Devices can be created using functions instead of
+:py:class:`~ophyd.Device` classes. The general idea is the same. For
+each factory function, the instrument loader will look for tables with
+arguments to this function, typically derived from the joined-lower
+name for the factory. For example, the function:
 
-``iconfig_default.toml``
-^^^^^^^^^^^^^^^^^^^^^^^^
+.. code-block:: python
 
-Haven includes an set of default configuration values in
-``src/haven/iconfig_default.toml``. This is mainly so that Haven and
-Firefly can still run during development without a dedicated
-configuration file. It also serves as a starting point for deploying
-Haven to a new beamline. See the section on testing below for
-suggestions on how to add default configuration.
+    def make_area_detector(name: str, prefix: str, ad_version: str = "4.3") -> Device:
+         ...
+
+could have an entry in the configuration file:
+
+.. code-block:: toml
+
+    [[ area_detector ]]
+    name = "sim_det"
+
+These factory functions should return either a **new Device**, or a
+**iterable of new devices**.
+
 
 Development and Testing
 -----------------------
 
 While adding features and tests to Haven, it is often necessary to
 read a configuration file, for example when testing functions that
 load devices through
-:py:func:`~haven.instrument.load_instrument.load_instrument()`. However,
+:py:func:`~haven.load_instrument.load_instrument()`. However,
 the configuration that is loaded should not come from a real beamline
 configuration or else there is a risk of controlling real hardware
 while running tests.
@@ -129,15 +165,27 @@ beamline-specific configuration, it can be added in one of two places.
   fluorescence detectors. This configuration should not point to real
   hardware.
 
+
+Checking Configuration
+----------------------
+
+If Haven is installed with pip, the command ``haven_config`` can be
+used to read configuration variables as they will be seen by Haven:
+
+.. code:: bash
+
+	  $ haven_config beamline
+	  {'hardware_is_present': False, 'name': 'SPC Beamline (sector unknown)'}
+	  $ haven_config beamline.hardware_is_present
+	  False
+  
+
 Example Configuration
 ---------------------
 
-Below are some examples of configuration that can be re-used for new
-devices support or beamline setup.
+Below is an example of a configuration that can be re-used for new
+device support or beamline setup.
 
-.. literalinclude:: ../../src/haven/iconfig_default.toml
-   :caption: iconfig_default.toml
-   :language: toml
 
 .. literalinclude:: ../../src/haven/iconfig_testing.toml
    :caption: iconfig_testing.toml

docs/topic_guides/changing_haven.rst

@@ -61,9 +61,9 @@ properly setup, the tests can be run using:
 xfailed, and warnings are expected.
 
 While running the tests, devices created using
-:py:func:`~haven.instrument.device.make_device()` will be replaced
+:py:func:`~haven.devices.device.make_device()` will be replaced
 with simulated devices using Ophyd's *sim* module. This means that
-:py:func:`~haven.instrument.load_instrument()` can be called without
+:py:func:`~haven.devices.load_instrument()` can be called without
 hardware being present, and the corresponding fake devices can be
 found in the :py:obj:`haven.registry`.