CHANGES.rst

Change History

The project milestones describe the future plans.

1.6.15

released 2023-03-27

Maintenance

• add log_path=None kwarg to apstools.utils.stream_log_handler()

Fixes

• resolved AttributeError for certain logging configurations

1.6.14

released 2023-03-12

Breaking Changes

• listobjects() printing keyword argument is deprecated.
• ListRuns.to_dataframe()' method is deprecated.
• ListRuns.to_table() method is deprecated.
• listruns() printing keyword argument is deprecated.
• listruns() tablefmt keyword argument is deprecated. Use table_style instead.
• listruns() now returns pyRestTable.Table or pandas.DataFrame

New Features

• MeasCompTc32: Measurement Computing TC-32 Thermocouple reader

Enhancements

• Add guide What are the objects to control?

Fixes

• labels_to_streams(): do not plot items in the labeled stream
• labels_to_streams(): failed when no motor label assigned

Maintenance

• add more unit tests for apstools.plans
• addDeviceDataAsStream() renamed to write_stream()
• FileWriterCallbackBase.file_name now a property, uses pathlib
• listobjects() now returns pyRestTable (by default)
• listdevice() now returns pyRestTable (by default)
• listplans() now returns pyRestTable (by default)
• listruns() now returns pyRestTable (by default)
• NXWriter: unit tests added for proper NXdata@axes structure.

1.6.13

released 2023-02-16

Maintenance

• Release process updated on the wiki.

1.6.12

released 2023-02-16

Maintenance

• NXWriter: add wait_writer_plan_stub() method for use in a plan.
• Release process documented on the wiki.

1.6.11

released 2023-02-15

New Features

• Support to record all motor (or other ophyd-labeled devices) positions at start (or end) of run.
• SpecWriterCallback: write #O and #P lines from motor positions recorded at start of run.

Fixes

• NXWriter: scans with area detectors did not have image data. Fixed. Clients must wait for writer to finish.
• PVPositionerSoftDone: computation of 'done' signal and 'inposition' deconvoluted.
• Struck3820: changed (typo) do_readl_all to do_read_all
• Tests involving PVPositionerSoftDone call '.cb_readback()' method to upate 'done' signal.

Maintenance

• Added github/super-linter workflow, adds mypy, isort, and other tests.
• Apply isort code style.

Known Problems

• Observing random occurrence of unit test failures involving PVPositionerSoftDone.

1.6.10

released 2023-01-11

New Features

• Add apstools.utils.plotxy() utility function.
• Add tag-based documentation selection via PyData switcher.

Enhancements

• In listruns(), also search the hints for keys.

Fixes

• CI failed to build documentation when creating pip installation.
• ScalerMotorFlyer default fly_time_pad increased to 10 s.
• ScalerMotorFlyer now sets scaler to "OneShot" mode.
• ScalerMotorFlyer in-position comparison now based on motor's precision.
• Version number was not reported correctly in certain situations (issue #771).

Maintenance

• "Custom HDF5 File Name" example updated to latest advice.
• Pin Sphinx to <6 due to problems with PyData and Sphinx v6.
• Conda channel "nsls2forge" no longer needed.
• Remove nsls2forge channel from conda environment.

New Contributors

• @jwkim-anl (Jong Woo Kim, ANL) for the plotxy() idea.

1.6.9

released 2022-11-30

New Features

• Add ScalerMotorFlyer() device.
• Add functions to support reporting of logging messages.
• Add :meth:`restorable_stage_sigs` decorator.
• Add support for Python 3.11.
• Add utils.analysis_1D() & utils.analysis_2D() functions for peaks statistics.

Enhancements

• Add example notebook Fly Scans with EPICS motor and scaler.
• Add guide How to Search in Databroker.
• Add guide How to setup logging.

Maintenance

• Add convenience import:from apstools.devices import AD_EpicsFileNameMixin.
• Enable: from apstools.devices import AD_EpicsFileNameMixin.
• Resolve intermittent, random CI failures.
• Resolve problems reporting unit test coverage statistics.
• Unit tests now support Python version 3.8, 3.9, 3.10, & 3.11.

1.6.8

released 2022-10-16

Fixes

• [again] Make sure that YML (and other) files are packaged for pip and conda.

1.6.7

skipped

1.6.6

released 2022-10-13

New Features

• Add request_input() plan stub, per user request.
• Add fly scan for scaler v. continuous motor.

Maintenance

• Rename: devices.make_dict_device() is now devices.dict_device_factory().

Fixes

• Make sure that YML (and other) files are packaged for pip and conda.
• PVPositioner got stuck if target position was the same as current position.

Deprecations

• Removed all snapshot support.

1.6.5

released 2022-10-04

Maintenance

• Pip requirements updated.

1.6.4

released 2022-10-03

New Features and/or Enhancements

• Example of imaging with AreaDetector in Single mode writing HDF5 files.
• lineup(): user can choose which feature (max, min, cen, com), additional API changes.
• devices.make_dict_device() to make a recordable Device from a dictionary.
• Only publish documentation on demand by executing GitHub workflow.
• Switch HTML documentation to use PyData theme.

Maintenance

• Re-arranged the documentation. Some examples renamed. Content unchanged.
• Update the APS cycle dates file through 2023-04-30.

1.6.3

released 2022-08-15

New Features and/or Enhancements

• Add devices.CamMixin_V3_1_1
• Add devices.CamMixin_V34
• Add devices.SingleTrigger_V34
• Add EpicsScanIdSignal (scan_id from EPICS PV).
• Add run_blocking_function() plan to run blocking functions in the RunEngine.
• Published on conda-forge conda install -c conda-forge apstools.
• replay() can now take a run, [run], header, or [header]
• Switch HTML documentation to use furo theme.

Maintenance

• Combine install steps, workflows in CI
• Describe how to use the NXWriter callback.
• Refactor AD test with EPICS-controlled image file names.
• Refactor test_move_to_zero().
• Unit tests now support Python version 3.8, 3.9, & 3.10.

Deprecations

• bluesky_snapshot_viewer and underlying GUI code will be dropped by 2022-12-31.
• Drop support for Python 3.7 per NEP29.
• Stop publishing on channel -c aps-anl-tag (use -c conda-forge instead).

1.6.2

released 2022-07-06

Notice

• Confirmed: databroker finds HDF5 image files with custom names.
• Still not ready for databroker 2.0+.

New Features

• Documentation website: https://bcda-aps.github.io/apstools/
• New example: user-controlled HDF5 image file names.
• Using Jupyter notebooks directly (via nbsphinx extension).
• User-controlled HDF5, JPEG, TIFF image file names.

Fixes

• Fix timeout problem in utils.connect_pvlist().
• Fix unexpected key in datum kwargs.
• Fix utils.listdevice() mixing dot and underline name separators.

Maintenance

• Applied custom project badge for APS software License.
• Use micromamba in testing workflows.

Deprecations

• Drop LGTM.com static code analysis service.
• Drop RTD (readthedocs) documentation publishing service.

Contributors

• Harry Zhou

1.6.1

released 2022-01-26

Fixes

• Move enable Component out from synApps Record devices.
• Renew the unit tests for PVPositionerSoftDoneWithStop.

1.6.0

released 2022-01-20

Breaking Changes

• Moved apsbss support to new apsbss package (install with either pip or conda). See https://bcda-aps.github.io/apsbss/ for details.
• Can use Python 3.7 - 3.9. Cannot use Python 3.10 yet due to upstream limitation from databroker and intake packages.
• Moved command_list_as_table() from utils into plans/command_list.
• Removed BusyStatus from apstools.synApps.busy
• callbacks/: DocumentCollectorCallback, document_contents_callback, and SnapshotReport moved into callbacks/.
• devices/: Reorganized all devices, including synApps/, into devices/ subpackage.
• devices/: SynPseudoVoigt() moved from signals/ to devices/.
• plans/: Reorganized plans.py and _plans/ into plans/ subpackage.
• snapshot/: Moved snapshot application and related files to a subdirectory.
• utils/: Reorganized utils.py and _utils/ into utils/ subpackage.

New Features and/or Enhancements

• Add support for Eurotherm 2216e temperature controller
• Add support for Lakeshore 336 temperature controller
• Add support for Lakeshore 340 temperature controller
• Add support for synApps calc scalcout record.
• Add support for synApps calc sseq record.
• Add support for EPICS base sub record.
• Add support for synApps calc userAve database.
• Add support for synApps calc userStringSeq database.
• Add support for synApps calc userStringCalc database.
• Add support for synApps optics 2slit database.

Fixes

• Convert None to "null" when saving PeakStats to stream.

Maintenance

Now testing with Python versions 3.7 - 3.9. (Can't use with Py3.10 yet due to upstream requirements.)

Update notebooks:

• demo_specfile_example
• demo_tuneaxis

Remove notebooks:

• demo_specfile_databroker

Deprecations

• Applications
  • apstools_plan_catalog application and related support.
• Devices
  • ApsCycleComputedRO
  • move_energy() method in KohzuSeqCtl_Monochromator class
  • ProcessController
• Utilities
  • device_read2table
  • json_export
  • json_import
  • listdevice_1_5_2
  • listruns_v1_4
  • object_explorer

Contributors

• Gilberto Fabbris
• Jan Ilavsky
• Qingteng Zhang

1.5.4

released 2021-11-25

NOTE: The apsbss component will be moved out of apstools into its own package with the next release (1.6.0, ~Feb 2022) of apstools.

Notice

The Python version is limited to 3.7 due to aps-dm-api package. Expect this limitation to be relaxed, allowing any Python 3.7 and higher with the 1.6.0 release.

Fixes

• Added table of APS run cycle dates. Use that if aps-dm-api not available.
• Restricted python version to 3.7 due to upstream aps_dm_api package.
• Rename name uid to token to avoid LGTM security false alert.

Deprecations

This support was marked as deprecated in release 1.5.4:

• apstools.devices.ApsCycleComputedRO

1.5.3

released 2021-10-15

Notice

The apstools.beamtime module and related content (includes apsbss) will be moved to a new repository for release 1.6.0. This will remove the requirement that the APS data management tools (package aps-dm, which only works on the APS computing network) be included. With this change, users will be able to conda install apstools -c aps-anl-tag on computers outside of the APS computing network.

Breaking Changes

• apstools.utils.listdevice has a new API (old version renamed to listdevice_1_5_2)

New Features and/or Enhancements

• Kohzu monochromator energy, wavelength, and theta each are now a PVPositioner (subclass).
• Linkam temperature controller CI94
• Linkam temperature controller T96
• Stanford Research Systems 570 current preamplifier
• Stanford Research Systems PTC10 temperature controller
• XIA PF4 filter now supports multiple PF4 units.
• Generalize that amplifiers will have a gain Component attribute.
• Generalize that temperature controllers will have a temperature Component attribute that is a positioner (subclass of ophyd.PVPositioner).
• Enhanced positioners for EPICS Devices: * apstools.devices.PVPositionerSoftDone * apstools.devices.PVPositionerSoftDoneWithStop

Fixes

• Fixed bug in devices.ApsCycleComputedRO and devices.ApsCycleDM involving datetime.

Maintenance

• Moved all device support into individual modules under apstools._devices because apstools.devices module was getting too big. Will refactor all with release 1.6.0.
• Add unit tests for devices.ApsCycle* Devices.
• Add EPICS IOCs (ADSimDetector and synApps xxx) to continuous integration for use in unit testing.
• Unit tests now use pytest package.
• Suppress certain warnings during unit testing.

Deprecations

This support will be removed in release 1.6.0:

• apstools.beamtime module and related content (includes apsbss) will be moved to a new repository
• apstools.devices.ProcessController
• apstools.utils.device_read2table
• apstools.utils.listdevice_1_5_2
• apstools.utils.object_explorer

Contributors

• Fanny Rodolakis
• Gilberto Fabbris
• Jan Ilavsky
• Qingteng Zhang
• 4-ID-C Polar
• 8-ID-I XPCS
• 9-ID-C USAXS

1.5.2 (and previous)

See this table for release change histories, highlighted by version control reference (pull request or issue):

1.5.2:	released 2021-09-29 Drop Codacy (https://app.codacy.com/gh/BCDA-APS/apstools) as no longer needed. #540 Add apstools.utils.listplans() function. #534 Add apstools.utils.OverrideParameters class. Hoisted from APS USAXS instrument. #537 Enhancements to apstools.utils.listruns(): Add search by list of scan_id or uid values. Optimize search speed. #534 Add apstools.plans.documentation_run() plan. Hoisted from APS USAXS instrument. #528 Add kind= kwarg to synApps Devices. #539 Break devices into submodule _devices.
1.5.1:	released 2021-07-22 #522 Deprecate apstools.devices.ProcessController. Suggest ophyd.PVPositioner instead. #521 Enhancement: new functions: getRunData(), getRunDataValue(), getStreamValues() & listRunKeys() #518 Bug fixed: TypeError from summary() of CalcoutRecord #517 Added support for python 3.9. #514 Refactor 'SIGNAL.value' to 'SIGNAL.get()'
1.5.0:	released 2021-04-02 #504 comment Dropped support for python 3.6. #495 Dropped diffractometer support code. #511 & #497 Add utils.findbyname() and utils.findbypv() functions. #506 spec2ophyd can now read SPEC config files from APS 17BM #504 Overhaul of listruns() using pandas. Previous code renamed to listruns_v1_4(). #503 Unit tests with data now used msgpack-backed databroker. #495 remove hklpy requirement since all diffractometer support code will be moved to [hklpy](https://github.com/bluesky/hklpy) package.
1.4.1:	released: 2021-01-23 add Area Detector configuration examples: Pilatus & Perkin-Elmer, both writing image to HDF5 file #488 use first trigger_mode when priming AD plugin #487 ensure spec2ophyd code is packaged
1.4.0:	released: 2021-01-15 #483 Python code style must pass flake8 test. #482 specwriter: Fix bug when plan_args structure includes a numpy ndarray. #474 :func:`apstools.utils.listruns()` now defaults to the current catalog in use. New functions: :func:`apstools.utils.getDatabase` :func:`apstools.utils.getDefaultDatabase` #472 Respond to changes in upstream packages. package requirements auto-detection of command list format (Excel or text) use openpyxl [1] instead of xlrd [2] and pandas [3] to read Microsoft Excel .xlsx spreadsheet files [1] https://openpyxl.readthedocs.io [2] https://xlrd.readthedocs.io [3] https://pandas.pydata.org #470 Area Detector plugin preparation & detection. :func:`apstools.devices.AD_plugin_primed()` re-written completely :func:`apstools.devices.AD_prime_plugin()` replaced by :func:`apstools.devices.AD_prime_plugin2()` #463 Remove deprecated features. apstools.suspenders.SuspendWhenChanged() apstools.utils.plot_prune_fifo() apstools.utils.show_ophyd_symbols() apstools.synapps.asyn.AsynRecord.binary_output_maxlength() apstools.devices.AD_warmed_up() #451 Undulator and Kohzu monochromator functionalities :class:`apstools.devices.ApsUndulator()` Adds some Signal components (such as setting kind kwarg) that are helpful in moving the undulator
1.3.9:	released 2020-11-30 #459 apsbss: list ESAFs & proposals from other cycles #457 :func:`apstools.utils.rss_mem()`: show memory used by this process
1.3.8:	released: 2020-10-23 #449 diffractometer wh() shows extra positioners #446 utils: device_read2table() renamed to listdevice() #445 synApps: add Device for iocStats #437 diffractometer add pa() report #426 diffractometer add simulated diffractometers #425 BUG fixed: listruns() when no stop document #423 BUG fixed: apsbss IOC starter script
1.3.7:	released: 2020-09-18 #422 additional AD support from APS USAXS #421 wait for undulator when start_button pushed #418 apsbss: only update APS run cycle name after current cycle ends
1.3.6:	released 2020-09-04 #416 apsbss: allow iso8601 time strings to have option for fractional seconds #415 Get APS cycle name from official source
1.3.5:	released 2020-08-25 #406 replace plot_prune_fifo() with trim_plot() and trim_plot_by_name() #405 add Y1 & Z2 read-only signal to Kohzu Monochromator device #403 deprecate SuspendWhenChanged()
1.3.4:	released 2020-08-14 #400 resolve warnings and example documentation inconsistency #399 parse iso8601 date for py36 #398 DiffractometerMixin: add wh() method #396 provide spec2ophyd application #394 add utils.copy_filtered_catalog() #392 RTD make parameter lists clearer #390 improve formatting of parameter list in RTD #388 add utils.quantify_md_key_use() #385 spec2ophyd: make entry point
1.3.3:	released 2020-07-22 #384 apsbss: print, not log from commands #382 spec2ophyd analyses
1.3.2:	released 2020-07-20 #380 apsbss: fix object references
1.3.1:	released 2020-07-18 #378 apsbss_ioc.sh: add checkup (keep-alive feature for the IOC) #376 apsbss: example beam line-specific shell scripts #375 apsbss: add PVs for numbers of users #374 apsbss_ophyd: addDeviceDataAsStream() from USAXS #373 account for time zone when testing datetime-based file name #371 update & simplify the travis-ci setup #369 spec2ophyd: handle NONE in SPEC counters #368 spec2ophyd: config file as command-line argument #367 apsbss: move ophyd import from main #364 apsbss: add PVs for ioc_host and ioc_user #363 Handle when mailInFlag not provided #360 prefer logging to print
1.3.0:	release expected by 2020-07-15 add NeXus writer callback add apsbss : APS experiment metadata support #351 apsbss: put raw info into PV #350 apsbss: clarify meaning of reported dates #349 apsbss: add "next" subcommand #347 some apbss files not published #346 publish fails to push conda packages #344 listruns() uses databroker v2 API #343 review and update requirements #342 summarize runs in databroker by plan_name and frequency #341 tools to summarize activity #340 update copyright year #339 resolve Codacy code review issues #338 unit tests are leaving directories undeleted #337 Document new filewriter callbacks #336 add NeXus file writer from USAXS #335 update requirements #334 support APS proposal & ESAF systems to provide useful metadata #333 access APS proposal and ESAF information #332 listruns(): use databroker v2 API #329 add NeXus writer base class from USAXS
1.2.6:	released 2020-06-26 #331 listruns succeeds even when number of existing runs is less than requested #330 BUG: listruns: less than 20 runs in catalog #328 epid: add final_value (.VAL field) #327 epid: remove clock_ticks (.CT field) #326 BUG: epid failed to connect to .CT field #325 BUG: epid final_value signal not found #324 BUG: epid controlled_value signal name
1.2.5:	released 2020-06-05 #322 add py38 to travis config #320 multi-pass tune should use FWHM for next scan #318 AxisTunerMixin is now subclass of DeviceMixinBase #317 BUG: USAXS can't tune motors #316 BUG: Error in asyn object definition #315 BUG: AttributeError from db.hs
1.2.3:	released 2020-05-07 #314 fix ImportError about SignalRO #313 update packaging requirements
1.2.2:	released 2020-05-06 DEPRECATION #306 apstools.plans.show_ophyd_symbols() will be removed by 2020-07-01. Use apstools.plans.listobjects() instead. #311 adapt to databroker v1 #310 enhance listruns() search capabilities #308 manage diffractometer constraints #307 add diffractometer emhancements #306 rename show_ophyd_objects() as listobjects() #305 add utils.safe_ophyd_name() #299 set_lim() does not set low limit
1.2.1:	released 2020-02-18 - bug fix #297 fix import error
1.2.0:	released 2020-02-18 - remove deprecated functions #293 remove run_blocker_in_plan() #292 remove list_recent_scans() #291 remove unix_cmd() #288 add object_explorer() (from APS 8-ID-I)
1.1.19:	released 2020-02-15 #285 add EpicsMotorResolutionMixin #284 adjust ophyd.EpicsMotor when motor limits changed from other EPICS client #283 print_RE_md() now returns a pyRestTable.Table object
1.1.18:	released 2020-02-09 PyPI would not accept the 1.1.17 version: filename has already been used see release notes for 1.1.17
1.1.17:	released 2020-02-09 - hot fixes #277 replace .value with .get() #276 update ophyd metadata after motor set_lim() #274 APS user operations could be in mode 1 OR 2
1.1.16:	released 2019-12-05 #269 bug: shutter does not move when expected #268 add redefine_motor_position() plan #267 remove lineup() plan for now #266 bug fix for #265 #265 refactor of #264 #264 Limit number of traces shown on a plot - use a FIFO #263 device_read2table() should print unless optioned False #262 add lineup() plan (from APS 8-ID-I XPCS)
1.1.15:	released 2019-11-21 : bug fixes, adds asyn record support #259 resolve AssertionError from setup_lorentzian_swait #258 swait record does not units, some other fields #255 plans: resolve indentation error #254 add computed APS cycle as signal #252 synApps: add asyn record support
1.1.14:	released 2019-09-03 : bug fixes, more synApps support #246 synApps: shorten name from synApps_ophyd #245 swait & calcout: change from EpicsMotor to any EpicsSignal #240 swait: refactor swait record & userCalc support #239 transform: add support for transform record #238 calcout: add support for calcout record & userCalcOuts #237 epid: add support for epid record #234 utils: replicate the unix() command #230 signals: resolve TypeError
1.1.13:	released 2019-08-15 : enhancements, bug fix, rename #226 writer: unit tests for empty #O0 & P0 control lines #224 rename: list_recent_scans --> listscans #222 writer: add empty #O0 and #P0 lines #220 ProcessController: bug fix - raised TypeError
1.1.12:	released 2019-08-05 : bug fixes & updates #219 ProcessController: bug fixes #218 replay(): sort chronological by default #216 replay(): fails when not list
1.1.11:	released 2019-07-31 : updates & new utility #214 new: apstools.utils.APS_utils.replay() #213 list_recent_scans show exit_status #212 list_recent_scans show reconstructed scan command
1.1.10:	released 2019-07-30 : updates & bug fix #211 devices calls to superclass __init__() #209 devices call to superclass __init__() #207 show_ophyd_symbols also shows labels #206 new: apstools.utils.APS_utils.list_recent_scans() #205 show_ophyd_symbols uses ipython shell's namespace #202 add labels attribute to enable wa and ct magic commands
1.1.9:	released 2019-07-28 : updates & bug fix #203 SpecWriterCallback: #N is number of data columns #199 spec2ophyd handle CNTPAR:read_misc_1
1.1.8:	released 2019-07-25 : updates #196 spec2ophyd handle MOTPAR:read_misc_1 #194 new show_ophyd_symbols shows table of global ophyd Signal and Device instances #193 spec2ophyd ignore None items in SPEC config file #192 spec2ophyd handles VM_EPICS_PV in SPEC config file #191 spec2ophyd handles PSE_MAC_MOT in SPEC config file #190 spec2ophyd handles MOTPAR in SPEC config file
1.1.7:	released 2019-07-04 DEPRECATION apstools.plans.run_blocker_in_plan() will be removed by 2019-12-31. Do not write blocking code in bluesky plans. Dropped python 3.5 from supported versions #175 move plans.run_in_thread() to utils.run_in_thread() #168 new spec2ophyd migrates SPEC config file to ophyd setup #166 device_read2table(): format device.read() results in a pyRestTable.Table #161 addDeviceDataAsStream(): add Device as named document stream event #159 convert xlrd.XLRDError into apstools.utils.ExcelReadError #158 new run_command_file() runs a command list from text file or Excel spreadsheet
1.1.6:	released 2019-05-26 #156 add ProcessController Device #153 print dictionary contents as table #151 EpicsMotor support for enable/disable #148 more LGTM recommendations #146 LGTM code review recommendations #143 filewriter fails to raise IOError #141 ValueError during tune()
1.1.5:	released 2019-05-14 #135 add refresh button to snapshot GUI
1.1.4:	released 2019-05-14 #140 event-model needs at least v1.8.0 #139 ValueError in :func:`~apstools.plans.TuneAxis.tune._scan`
1.1.3:	released 2019-05-10 adds packaging dependence on event-model #137 adds utils.json_export() and utils.json_import()
1.1.1:	released 2019-05-09 adds packaging dependence on spec2nexus #136 get json document stream(s) #134 add build on travis-ci with py3.7 #130 fix conda recipe and pip dependencies (thanks to Maksim Rakitin!) #128 SpecWriterCallback.newfile() problem with scan_id = 0 #127 fixed: KeyError from SPEC filewriter #126 add uid to metadata #125 SPEC filewriter scan numbering when "new" data file exists #124 fixed: utils.trim_string_for_EPICS() trimmed string too long #100 fixed: SPEC file data columns in wrong places
1.1.0:	released 2019.04.16 change release numbering to Semantic Versioning (remove all previous tags and releases) batch scans using Excel spreadsheets bluesky_snapshot_viewer and bluesky_snapshot conda package available