JP-2928: Update FGS guider test data, add image3 test

[melanieclarke]

Partially addresses JP-2928

Update test data for FGS guider regression tests to in-flight examples. Data is from PID 1029 obs 1, the same program as is used in the image2 test. The EXP_TYPEs tested are FGS_ACQ1, FGS_FINEGUIDE, FGS_ID-IMAGE, FGS_ID-STACK.

There was also an image3 test on non-guider data in the test_fgs_guider module that checked the resample step with input data that makes too large a mosaic. I moved this test to a new file (test_fgs_image3.py) and added a new test for image3 on FGS image data. I used cal files for PID 1029 obs 1 for both tests. To simulate the failure case on data that should succeed, I set the allowed memory environment variable to a small value.

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.datamodels.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.fits_generator.rst
• changes/<PR#>.set_telescope_pointing.rst
• changes/<PR#>.pipeline.rst

stage 1

• changes/<PR#>.group_scale.rst
• changes/<PR#>.dq_init.rst
• changes/<PR#>.emicorr.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.ipc.rst
• changes/<PR#>.firstframe.rst
• changes/<PR#>.lastframe.rst
• changes/<PR#>.reset.rst
• changes/<PR#>.superbias.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.rscd.rst
• changes/<PR#>.persistence.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.charge_migration.rst
• changes/<PR#>.jump.rst
• changes/<PR#>.clean_flicker_noise.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.gain_scale.rst

stage 2

• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.badpix_selfcal.rst
• changes/<PR#>.msaflagopen.rst
• changes/<PR#>.nsclean.rst
• changes/<PR#>.imprint.rst
• changes/<PR#>.background.rst
• changes/<PR#>.extract_2d.rst
• changes/<PR#>.master_background.rst
• changes/<PR#>.wavecorr.rst
• changes/<PR#>.srctype.rst
• changes/<PR#>.straylight.rst
• changes/<PR#>.wfss_contam.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.fringe.rst
• changes/<PR#>.pathloss.rst
• changes/<PR#>.barshadow.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.pixel_replace.rst
• changes/<PR#>.resample_spec.rst
• changes/<PR#>.residual_fringe.rst
• changes/<PR#>.cube_build.rst
• changes/<PR#>.extract_1d.rst
• changes/<PR#>.resample.rst

stage 3

• changes/<PR#>.assign_mtwcs.rst
• changes/<PR#>.mrs_imatch.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.exp_to_source.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.tso_photometry.rst
• changes/<PR#>.stack_refs.rst
• changes/<PR#>.align_refs.rst
• changes/<PR#>.klip.rst
• changes/<PR#>.spectral_leak.rst
• changes/<PR#>.source_catalog.rst
• changes/<PR#>.combine_1d.rst
• changes/<PR#>.ami.rst

other

• changes/<PR#>.wfs_combine.rst
• changes/<PR#>.white_light.rst
• changes/<PR#>.cube_skymatch.rst
• changes/<PR#>.engdb_tools.rst
• changes/<PR#>.guider_cds.rst

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.92%. Comparing base (8d2b64c) to head (1cb213a).
Report is 9 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8923   +/-   ##
=======================================
  Coverage   61.92%   61.92%           
=======================================
  Files         376      376           
  Lines       38691    38691           
=======================================
  Hits        23961    23961           
  Misses      14730    14730           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[melanieclarke]

Regression tests with locally generated truth files here:
https://github.com/spacetelescope/RegressionTests/actions/runs/11523062423

[melanieclarke]

The new regression test for image3 shows 1 element different in each of the catalog and segmented image, due to architectural differences. I will okify the Linux results, since it wouldn't help to threshold the segmented image differently.

Running again here, after okify:
https://github.com/spacetelescope/RegressionTests/actions/runs/11559125880

No failures.

[tapastro]

LGTM, though if we are okifying the Linux results we could probably also tighten the tolerances. But it's probably not worth iterating on.

[melanieclarke]

LGTM, though if we are okifying the Linux results we could probably also tighten the tolerances. But it's probably not worth iterating on.

For the i2d? I can tighten that, while we're here.

We do both, in various places in the regression tests. While working on these, I have generally kept looser tolerances on the resampled products, if that gets them through a cross-architecture check, because it is done for those products in most places across the regression tests. For things like slightly different edge pixels, loosening the tolerance doesn't help anyway because there are mismatched NaNs. It would be nice to have a more consistent strategy, though - maybe we can discuss that too, as part of JP-3789.

[melanieclarke]

Regression test here with just the test_fgs_image tests, showing the diff in the i2d product with the tighter tolerance:
https://github.com/spacetelescope/RegressionTests/actions/runs/11576974265

I have okified the Linux version and will go ahead and merge.