Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Occasional pytest fails in CI, yet CI reported as OK #905

Closed
prjemian opened this issue Jan 8, 2024 · 2 comments · Fixed by #929
Closed

Occasional pytest fails in CI, yet CI reported as OK #905

prjemian opened this issue Jan 8, 2024 · 2 comments · Fixed by #929
Labels
bug
Milestone
1.6.19

Comments

@prjemian
Copy link
Contributor

prjemian commented Jan 8, 2024

CI failed in code not related to this PR. Coincident with such failure is dramatic reduction on code coverage. Here is the Py3.11 log with the fail

2024-01-08T21:23:21.2348097Z apstools/plans/tests/test_alignment.py::test_lineup2[signals4-mover4--1.2-1.2-11-max-2] FAILED [ 48%]
2024-01-08T21:23:21.2349107Z 
2024-01-08T21:23:21.2349402Z =================================== FAILURES ===================================
2024-01-08T21:23:21.2350418Z _______________ test_lineup2[signals4-mover4--1.2-1.2-11-max-2] ________________
2024-01-08T21:23:21.2352695Z 
2024-01-08T21:23:21.2354570Z signals = [SynPseudoVoigt(name='pvoigt', value=5, timestamp=1704748988.992658), EpicsSignalRO(read_pv='gp:userCalc1.VAL', name='..., 'channels.chan06.gate', 'count_mode', 'delay', 'auto_count_delay', 'freq', 'preset_time', 'auto_count_time', 'egu'])]
2024-01-08T21:23:21.2358614Z mover = EpicsSignal(read_pv='gp:gp:float1', name='axis', value=0.0, timestamp=1704748999.251151, tolerance=0.0001, auto_monitor=True, string=False, write_pv='gp:gp:float1', limits=False, put_complete=False)
2024-01-08T21:23:21.2360720Z start = -1.2, finish = 1.2, npts = 11, feature = 'max', nscans = 2
2024-01-08T21:23:21.2368328Z 
2024-01-08T21:23:21.2372382Z     @pytest.mark.parametrize(
2024-01-08T21:23:21.2377464Z         "signals, mover, start, finish, npts, feature, nscans",
2024-01-08T21:23:21.2378157Z         [
2024-01-08T21:23:21.2378841Z             [noisy, m1, -1.2, 1.2, 11, "max", 2],  # slower, is motor
2024-01-08T21:23:21.2379855Z             [pvoigt, axis, -1.2, 1.2, 11, "cen", 2],  # faster, is ao (float)
2024-01-08T21:23:21.2380802Z             [pvoigt, axis, -1.2, 1.2, 11, "max", 2],
2024-01-08T21:23:21.2381613Z             [[pvoigt], axis, -1.2, 1.2, 11, "max", 2],  # list
2024-01-08T21:23:21.2382714Z             [[pvoigt, noisy, scaler1], axis, -1.2, 1.2, 11, "max", 2],  # more than one detector
2024-01-08T21:23:21.2383817Z             [(pvoigt), axis, -1.2, 1.2, 11, "max", 2],  # tuple
2024-01-08T21:23:21.2388122Z             [pvoigt, axis, -1.2, 1.2, 11, "cen", 1],
2024-01-08T21:23:21.2388865Z             [pvoigt, axis, -1.2, 1.2, 11, "com", 1],
2024-01-08T21:23:21.2389598Z             [pvoigt, axis, -1.2, 1.2, 11, "max", 1],
2024-01-08T21:23:21.2390434Z             [pvoigt, axis, -1.2, 1.2, 11, "min", 1],  # pathological
2024-01-08T21:23:21.2391097Z         ],
2024-01-08T21:23:21.2391430Z     )
2024-01-08T21:23:21.2392046Z     def test_lineup2(signals, mover, start, finish, npts, feature, nscans):
2024-01-08T21:23:21.2392922Z         if isinstance(signals, SynPseudoVoigt):
2024-01-08T21:23:21.2393756Z             signals.randomize_parameters(scale=250_000, bkg=0.000_000_000_1)
2024-01-08T21:23:21.2394479Z         else:
2024-01-08T21:23:21.2394867Z             change_noisy_parameters()
2024-01-08T21:23:21.2395369Z     
2024-01-08T21:23:21.2395712Z         RE(bps.mv(mover, 0))
2024-01-08T21:23:21.2396229Z         assert get_position(mover) == 0.0
2024-01-08T21:23:21.2396759Z     
2024-01-08T21:23:21.2397506Z >       RE(alignment.lineup2(signals, mover, start, finish, npts, feature=feature, nscans=nscans))
2024-01-08T21:23:21.2398304Z 
2024-01-08T21:23:21.2398868Z /home/runner/work/apstools/apstools/apstools/plans/tests/test_alignment.py:196: 
2024-01-08T21:23:21.2399970Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-01-08T21:23:21.2401468Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:903: in __call__
2024-01-08T21:23:21.2402913Z     plan_return = self._resume_task(init_func=_build_task)
2024-01-08T21:23:21.2405237Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:1042: in _resume_task
2024-01-08T21:23:21.2406580Z     raise exc
2024-01-08T21:23:21.2408031Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:1672: in _run
2024-01-08T21:23:21.2409310Z     raise err
2024-01-08T21:23:21.2410614Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:1532: in _run
2024-01-08T21:23:21.2412016Z     msg = self._plan_stack[-1].send(resp)
2024-01-08T21:23:21.2412996Z /home/runner/work/apstools/apstools/apstools/plans/alignment.py:357: in lineup2
2024-01-08T21:23:21.2414016Z     yield from _inner()  # Run the scan.
2024-01-08T21:23:21.2415575Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:1203: in dec_inner
2024-01-08T21:23:21.2416933Z     return (yield from plan)
2024-01-08T21:23:21.2418436Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:408: in subs_wrapper
2024-01-08T21:23:21.2419940Z     return (yield from finalize_wrapper(_inner_plan(),
2024-01-08T21:23:21.2421899Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:535: in finalize_wrapper
2024-01-08T21:23:21.2423337Z     ret = yield from plan
2024-01-08T21:23:21.2424808Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:406: in _inner_plan
2024-01-08T21:23:21.2426182Z     return (yield from plan)
2024-01-08T21:23:21.2427062Z /home/runner/work/apstools/apstools/apstools/plans/alignment.py:354: in _inner
2024-01-08T21:23:21.2428324Z     yield from bp.rel_scan(detectors, mover, rel_start, rel_end, points, md=_md)
2024-01-08T21:23:21.2430085Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/plans.py:1410: in rel_scan
2024-01-08T21:23:21.2431417Z     return (yield from inner_rel_scan())
2024-01-08T21:23:21.2432977Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:1203: in dec_inner
2024-01-08T21:23:21.2434343Z     return (yield from plan)
2024-01-08T21:23:21.2435969Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:1163: in reset_positions_wrapper
2024-01-08T21:23:21.2437687Z     return (yield from finalize_wrapper(plan_mutator(plan, insert_reads),
2024-01-08T21:23:21.2439522Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:535: in finalize_wrapper
2024-01-08T21:23:21.2440942Z     ret = yield from plan
2024-01-08T21:23:21.2442410Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:173: in plan_mutator
2024-01-08T21:23:21.2443769Z     raise ex
2024-01-08T21:23:21.2445335Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:126: in plan_mutator
2024-01-08T21:23:21.2446816Z     msg = plan_stack[-1].send(ret)
2024-01-08T21:23:21.2448316Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:1203: in dec_inner
2024-01-08T21:23:21.2449681Z     return (yield from plan)
2024-01-08T21:23:21.2451265Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:1119: in relative_set_wrapper
2024-01-08T21:23:21.2452723Z     return (yield from plan)
2024-01-08T21:23:21.2454206Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:263: in msg_mutator
2024-01-08T21:23:21.2455567Z     msg = plan.send(_s)
2024-01-08T21:23:21.2457023Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:173: in plan_mutator
2024-01-08T21:23:21.2458733Z     raise ex
2024-01-08T21:23:21.2460146Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:126: in plan_mutator
2024-01-08T21:23:21.2461610Z     msg = plan_stack[-1].send(ret)
2024-01-08T21:23:21.2463069Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/plans.py:1407: in inner_rel_scan
2024-01-08T21:23:21.2464497Z     return (yield from scan(detectors, *args, num=num,
2024-01-08T21:23:21.2466018Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/plans.py:1120: in scan
2024-01-08T21:23:21.2467374Z     return (yield from scan_nd(detectors, full_cycler,
2024-01-08T21:23:21.2468914Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/plans.py:1015: in scan_nd
2024-01-08T21:23:21.2470208Z     return (yield from inner_scan_nd())
2024-01-08T21:23:21.2471751Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:1203: in dec_inner
2024-01-08T21:23:21.2473019Z     return (yield from plan)
2024-01-08T21:23:21.2474405Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:985: in stage_wrapper
2024-01-08T21:23:21.2475876Z     return (yield from finalize_wrapper(inner(), unstage_devices()))
2024-01-08T21:23:21.2477926Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:535: in finalize_wrapper
2024-01-08T21:23:21.2479360Z     ret = yield from plan
2024-01-08T21:23:21.2480852Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:983: in inner
2024-01-08T21:23:21.2482245Z     return (yield from plan)
2024-01-08T21:23:21.2483855Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:1203: in dec_inner
2024-01-08T21:23:21.2485453Z     return (yield from plan)
2024-01-08T21:23:21.2486990Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:351: in run_wrapper
2024-01-08T21:23:21.2488455Z     yield from contingency_wrapper(plan,
2024-01-08T21:23:21.2490263Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:625: in contingency_wrapper
2024-01-08T21:23:21.2491768Z     yield from else_plan()
2024-01-08T21:23:21.2493264Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/plan_stubs.py:984: in close_run
2024-01-08T21:23:21.2494973Z     return (yield Msg('close_run', exit_status=exit_status, reason=reason))
2024-01-08T21:23:21.2496770Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:198: in plan_mutator
2024-01-08T21:23:21.2498157Z     inner_ret = yield msg
2024-01-08T21:23:21.2499639Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:251: in msg_mutator
2024-01-08T21:23:21.2501014Z     _s = yield msg
2024-01-08T21:23:21.2502463Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/preprocessors.py:198: in plan_mutator
2024-01-08T21:23:21.2503852Z     inner_ret = yield msg
2024-01-08T21:23:21.2505241Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:1592: in _run
2024-01-08T21:23:21.2506553Z     new_response = await coro(msg)
2024-01-08T21:23:21.2508042Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:1809: in _close_run
2024-01-08T21:23:21.2509457Z     ret = (await current_run.close_run(msg))
2024-01-08T21:23:21.2510978Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/bundlers.py:131: in close_run
2024-01-08T21:23:21.2512370Z     await self.emit(DocumentNames.stop, doc)
2024-01-08T21:23:21.2514305Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:2529: in emit
2024-01-08T21:23:21.2515628Z     self.emit_sync(name, doc)
2024-01-08T21:23:21.2517071Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:2526: in emit_sync
2024-01-08T21:23:21.2518456Z     self.dispatcher.process(name, doc)
2024-01-08T21:23:21.2520000Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/run_engine.py:2549: in process
2024-01-08T21:23:21.2521541Z     exceptions = self.cb_registry.process(name, name.name, doc)
2024-01-08T21:23:21.2523268Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:385: in process
2024-01-08T21:23:21.2524614Z     func(*args, **kwargs)
2024-01-08T21:23:21.2526253Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/bluesky/utils/__init__.py:474: in __call__
2024-01-08T21:23:21.2527555Z     return mtd(*args, **kwargs)
2024-01-08T21:23:21.2528539Z /home/runner/work/apstools/apstools/apstools/callbacks/scan_signal_statistics.py:148: in receiver
2024-01-08T21:23:21.2529546Z     getattr(self, key)(document)
2024-01-08T21:23:21.2530462Z /home/runner/work/apstools/apstools/apstools/callbacks/scan_signal_statistics.py:199: in stop
2024-01-08T21:23:21.2531526Z     self.report()
2024-01-08T21:23:21.2532788Z /home/runner/work/apstools/apstools/apstools/callbacks/scan_signal_statistics.py:165: in report
2024-01-08T21:23:21.2533880Z     v = getattr(stats, k)
2024-01-08T21:23:21.2534463Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-01-08T21:23:21.2535018Z 
2024-01-08T21:23:21.2536762Z self = SummationRegisters(X=8.881784197001252e-16, XX=6.336, XXY=2475.2467216127584, XY=1.1368683772161603e-13, Y=4297.303336...946638440411, slope=-3.6820145648547125e-14, stddev_x=0.795989949685296, stddev_y=None, x_at_max_y=1.2, x_at_min_y=1.2)
2024-01-08T21:23:21.2538427Z 
2024-01-08T21:23:21.2538585Z     @property
2024-01-08T21:23:21.2538945Z     def stddev_y(self):
2024-01-08T21:23:21.2539329Z         r"""
2024-01-08T21:23:21.2539741Z         Standard deviation of :math:`y` values.
2024-01-08T21:23:21.2540269Z     
2024-01-08T21:23:21.2540885Z         .. math:: \sigma_y^2 = {{\sum{y^2} - \bar{y}\sum{y}} \over {n-1}}
2024-01-08T21:23:21.2541535Z         """
2024-01-08T21:23:21.2542165Z >       return math.sqrt((self.YY - self.mean_y * self.Y) / (self.n - 1))
2024-01-08T21:23:21.2542882Z E       ValueError: math domain error
2024-01-08T21:23:21.2543245Z 
2024-01-08T21:23:21.2544215Z /home/runner/micromamba/envs/anaconda-test-env-py-3.11/lib/python3.11/site-packages/pysumreg/sum_registers.py:192: ValueError
2024-01-08T21:23:21.2545624Z ----------------------------- Captured stdout call -----------------------------

Originally posted by @prjemian in #904 (comment)
@prjemian prjemian added the bug label Jan 8, 2024
@prjemian prjemian added this to the 1.6.19 milestone Jan 8, 2024
@prjemian prjemian mentioned this issue Jan 8, 2024
Closed
@prjemian
Copy link
Contributor Author

prjemian commented Jan 9, 2024

Look at this other workflow (for pipefail): https://github.com/NSLS-II/sirepo-bluesky/blob/39a7d09a3816f01396a4309149c0c2732606546a/.github/workflows/testing.yml#L72

@prjemian
Copy link
Contributor Author

prjemian commented Jan 9, 2024 

      - name: Copy databroker config file
        run: |
          set -vxeuo pipefail
          mkdir -v -p ~/.config/databroker/
          cp -v examples/local.yml ~/.config/databroker/

prjemian added a commit that referenced this issue Jan 9, 2024
@prjemian
CI #876 disable until #905
717b1c4
This was referenced Jan 22, 2024
Merged
Closed
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 add pipefail
017d2f9
@prjemian prjemian mentioned this issue Feb 26, 2024
Merged
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 syntax
1e4e74e
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 docs workflow, too
5f74014
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
DI #905 enable depndabot review
c7db7c2
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 cleanup/update/dispose workflows
a7a4e6d
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
DOC #905 reST syntax
f64da58
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 format them YAML files
ead6317
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 again
7e1a672
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
MNT #905 handle known exception
8a43ecb
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
DOC #905 update release notes
433c0fc
prjemian added a commit that referenced this issue Feb 26, 2024
@prjemian
CI #905 update
3f6b132
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
1.6.19
Development

Successfully merging a pull request may close this issue.

add pipefail to CI workflows BCDA-APS/apstools
1 participant
@prjemian