don't store empty columns in HDRTAB

[braingram]

This PR updates the model blender to skip storing columns full of nan for values not defined in any input (blended) model.

Regression tests: https://github.com/spacetelescope/RegressionTests/actions/runs/11824762170

As expected this causes many regression tests differences in the HDRTAB extension generated by the model blender like the following:

E            Data contains differences:
E              Tables have different number of columns:
E               a: 224
E               b: 417
E              Extra column GSC_VER of format D in b
E              Extra column HGA_STRT of format D in b
E              Extra column HGA_STOP of format D in b

To take one example jwst-pipeline/dev/truth/test_miri_cubebuild/jw01024-o001_t002_miri_ch1-short_s3d.fits the truth file currently has a HDRTAB with 417 columns. The GSC_VER column contains 2 nan values (neither of the 2 input files has that value defined). With this PR the HDRTAB has 224 columns and does not contain a column for GSC_VER.

There are also unrelated failures (that also appear on main) and appear to be due to a crds update:

E            Headers contain differences:
E              Keyword R_PTHLOS has different values:
E                 a> crds://jwst_nirspec_pathloss_0007.fits
E                  ?                                 ^
E                 b> crds://jwst_nirspec_pathloss_0004.fits
E                  ?

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

• changes/<PR#>.general.rst: infrastructure or miscellaneous change
• changes/<PR#>.docs.rst
• changes/<PR#>.stpipe.rst
• changes/<PR#>.datamodels.rst
• changes/<PR#>.scripts.rst
• changes/<PR#>.fits_generator.rst
• changes/<PR#>.set_telescope_pointing.rst
• changes/<PR#>.pipeline.rst

stage 1

• changes/<PR#>.group_scale.rst
• changes/<PR#>.dq_init.rst
• changes/<PR#>.emicorr.rst
• changes/<PR#>.saturation.rst
• changes/<PR#>.ipc.rst
• changes/<PR#>.firstframe.rst
• changes/<PR#>.lastframe.rst
• changes/<PR#>.reset.rst
• changes/<PR#>.superbias.rst
• changes/<PR#>.refpix.rst
• changes/<PR#>.linearity.rst
• changes/<PR#>.rscd.rst
• changes/<PR#>.persistence.rst
• changes/<PR#>.dark_current.rst
• changes/<PR#>.charge_migration.rst
• changes/<PR#>.jump.rst
• changes/<PR#>.clean_flicker_noise.rst
• changes/<PR#>.ramp_fitting.rst
• changes/<PR#>.gain_scale.rst

stage 2

• changes/<PR#>.assign_wcs.rst
• changes/<PR#>.badpix_selfcal.rst
• changes/<PR#>.msaflagopen.rst
• changes/<PR#>.nsclean.rst
• changes/<PR#>.imprint.rst
• changes/<PR#>.background.rst
• changes/<PR#>.extract_2d.rst
• changes/<PR#>.master_background.rst
• changes/<PR#>.wavecorr.rst
• changes/<PR#>.srctype.rst
• changes/<PR#>.straylight.rst
• changes/<PR#>.wfss_contam.rst
• changes/<PR#>.flatfield.rst
• changes/<PR#>.fringe.rst
• changes/<PR#>.pathloss.rst
• changes/<PR#>.barshadow.rst
• changes/<PR#>.photom.rst
• changes/<PR#>.pixel_replace.rst
• changes/<PR#>.resample_spec.rst
• changes/<PR#>.residual_fringe.rst
• changes/<PR#>.cube_build.rst
• changes/<PR#>.extract_1d.rst
• changes/<PR#>.resample.rst

stage 3

• changes/<PR#>.assign_mtwcs.rst
• changes/<PR#>.mrs_imatch.rst
• changes/<PR#>.tweakreg.rst
• changes/<PR#>.skymatch.rst
• changes/<PR#>.exp_to_source.rst
• changes/<PR#>.outlier_detection.rst
• changes/<PR#>.tso_photometry.rst
• changes/<PR#>.stack_refs.rst
• changes/<PR#>.align_refs.rst
• changes/<PR#>.klip.rst
• changes/<PR#>.spectral_leak.rst
• changes/<PR#>.source_catalog.rst
• changes/<PR#>.combine_1d.rst
• changes/<PR#>.ami.rst

other

• changes/<PR#>.wfs_combine.rst
• changes/<PR#>.white_light.rst
• changes/<PR#>.cube_skymatch.rst
• changes/<PR#>.engdb_tools.rst
• changes/<PR#>.guider_cds.rst

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.68%. Comparing base (0659b9c) to head (e6452da).
Report is 10 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8958   +/-   ##
=======================================
  Coverage   63.68%   63.68%           
=======================================
  Files         375      375           
  Lines       38691    38696    +5     
=======================================
+ Hits        24640    24645    +5     
  Misses      14051    14051           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tapastro]

I'm not sure what fraction of JWST users access HDRTAB, but I think this should be an improvement for those who do!