- Add outlier_detection median calculators from jwst. (#292)
- Deprecate wcs_from_footprints. Use wcs_from_sregions instead. (#307)
- Add wcs_from_sregions function to compute a combined WCS from a list of s_regions. (#307)
- Fix IntEnum saturation flag issue with numpy 2+ for romancal. (#305)
- Fix abs_deriv handling of off-edge and nan values. (#311)
- Added fillval option to
gwcs_blotutility. (#291) - Update downstream tests for jwst and romancal to fix pytest configurations. (#297)
- Changed the default ramp fitting CI test algorithm to
OLS_C. This also revealed a bug in control flow that allowed for the CHARGELOSS recalculation in error, which resulted in a crash while attempting to dereference aNULLpointer. Further, when creating the optional results product, the object creation was changed to PyArray_ZEROS to ensure invalid data was set to zero. The use of PyArray_EMPTY does not initialize memory, so junk data could be in used array elements. (#298) - Add infrastructure for testing memory usage (#299)
- Preparing ramp fitting for the upgrade to python 3.13. In python 3.13, the
C-API
function
PyLong_AsLongraises an exception if the object passed to it isNoneType. There are two integer attributes for theRampDataclass that can beNoneType, so a check forNoneTypefor these attributes was added. (#303)
- [ramp_fitting] Add the likelihood algorithm to ramp fitting. (#278)
- [saturation] Add option for using the readout pattern information to improve saturation flagging in grouped data. (#283)
- Add clip_accum parameter to alignment methods. (#286)
- Improve handling of catalog web service connectivity issues. (#286)
- Implement byteorder swap method that is forward-compatible with numpy 2.0 in jwst ramp_fitting. (#282)
- [jump] Fix a logical bug in the jump step for usage of > vs >= per JP-3689. (#285)
- [ramp_fitting] Moving the read noise recalculation due to CHARGELOSS flagging from the JWST ramp fit step code into the STCAL ramp fit C-extension. (#275)
- Fixed memory leak in C-extension. (#281)
- use
towncrierto handle changelog entries (#284)
- Add TweakReg submodule. [#267]
- Move the CHARGELOSS read noise variance recalculation from the JWST step code to the C extension to simplify the code and improve performance.[#275]
- Add
outlier_detectionsubmodule withutilsincluded from jwst. [#270]
- Fix bugs in the C algorithm Poisson variance calculation when provided with an average dark current. [#269]
- When OLS_C was selected as the ramp fitting algorithm with multiprocessing, the C extension was not called. The old python code was called. This bug has been fixed, so the C extension is properly run when selecting multiprocessing. [#268]
- build with Numpy 2.0 release candidate [#260]
- Flag asymmetrical snowballs that are missed by the current code (JP-3638). This was changed to not require that the center of the snowball jump ellipse is a saturated pixel. [#261]
- Catch some additional warnings about all-NaN slices. [#258]
- Fix a bug in Poisson variance calculation visible when providing an average dark current value in which the specified dark current was not converted to the appropriate units for pixels with negative slopes. This resulted in incorrect SCI, ERR, and VAR_POISSON values. Also required revising the approach for catching all-zero variance cases when average dark current was not specified. [#255]
- Refactor ramp fitting using a C extension to improve performance. [#156]
- Switch multiprocessing method to
fork_server. [#249]
- Switch multiprocessing method to
fork_server. [#249]
- Updated the shower flagging code to mask reference pixels, require a minimum number of groups to trigger the detection, and use all integrations to determine the median value. [#248]
- Changed the data type of three variables that are used in measuring the jump free segments of integrations. The variables were uint8 and they would yield wrong results for integrations with more than 256 groups. [#251]
- Use
sqrtfinstead ofsqrtin ols_cas22 ramp fitting with jump detection to avoid small numerical errors on different systems due to a cast to/from double. [#252]
- Enable the use of multiple integrations to find outliers. Also, when the number of groups is above a threshold, use single pass outlier flagging rather than the iterative flagging. [#242]
- Use
sqrtfinstead ofsqrtin ols_cas22 ramp fitting with jump detection to avoid small numerical errors on different systems due to a cast to/from double. [#252]
- Add
average_dark_currentto calculations of poisson variance. [#243]
- Add in the flagging of groups in the integration after a snowball occurs. The saturated core of the snowball gets flagged as jump for a number of groups passed in as a parameter [#238]
- Fixed the computation of the number of rows per slice for multiprocessing, which was causing different results when running the step with multiprocess [#239]
- Fix the code to at least always flag the group with the shower and the requested groups after the primary shower. [#237]
- Reorganize jump docs between the jwst and stcal repos. [#240]
- Reorganize ramp_fitting docs between the jwst and stcal repos. [#240]
- non-code updates to testing and development infrastructure
- re-release to publish source distribution
- Added
alignmentsub-package. [#179] - Enable automatic linting and code style checks [#187]
- Refactor Casertano, et.al, 2022 uneven ramp fitting and incorporate the matching jump detection algorithm into it. [#215]
- Fix memory issue with Cas22 uneven ramp fitting [#226]
- Fix some bugs in the jump detection algorithm within the Cas22 ramp fitting [#227]
- Moving some CI tests from JWST to STCAL. [#228, spacetelescope/jwst#6080]
- Significantly improve the performance of the Cas22 uneven ramp fitting algorithm. [#229]
- small hotfix for Numpy 2.0 deprecations [#211]
- Added read_pattern argument to flag_saturated_pixels. When used, this argument adjusts the saturation group-by-group to handle different numbers of frames entering different groups for Roman. When not set, the original behavior is preserved. [#188]
- Fixed failures with Numpy 2.0. [#210, #211]
- enable the detection of snowballs that occur in frames that are within a group. [#207]
- Added more allowable selections for the number of cores to use for multiprocessing [#183]
- Fixed the computation of the number of rows per slice for multiprocessing, which caused different results when running the step with multiprocess [#239]
- Added more allowable selections for the number of cores to use for multiprocessing [#183]
- Updating variance computation for invalid integrations, as well as updating the median rate computation by excluding groups marked as DO_NOT_USE. [#208]
- Implement the Casertano, et.al, 2022 uneven ramp fitting [#175]
- Added setting of number_extended_events for non-multiprocessing mode. This is the value that is put into the header keyword EXTNCRS. [#178]
- Added setting of number_extended_events for non-multiprocessing mode. This is the value that is put into the header keyword EXTNCRS. [#178]
1.4.1 (2023-06-29)
- Added statement to prevent the number of cores used in multiprocessing from being larger than the number of rows. This was causing some CI tests to fail. [#176]
- Updated the jump detection to switch to using the numpy sigmaclip routine to find the actual rms across integrations when there are at least 101 integrations in the exposure. This still allows cosmic rays and snowballs/showers to be flagged without being affected by slope variations due to either brigher-fatter/charge-spilling or errors in the nonlinearity correction. Also added the counting of the number of cosmic rays and snowballs/showers that is then placed in the FITS header in the JWST routines. [#174]
- Changing where time division occurs during ramp fitting in order to properly handle special cases where the time is not group time, such as when ZEROFRAME data is used, so the time is frame time. [#173]
- Added another line of code to be included in the section where warnings are turned off. The large number of warnings can cause a hang in the Jupyter notebook when running with multiprocessing. [#174]
- Fixed handling of MIRI segmented data files so that the correct dark integrations get subtracted from the correct science integrations. [#165]
- Correct the "averaging" of the final image slope by properly excluding variances as a part of the denominator from integrations with invalid slopes. [#167]
- Removing the usage of
numpy.wherewhere possible for performance reasons. [#169]
- Correctly compute the number of groups in a segment to properly compute the optimal weights for the OLS ramp fitting algorithm. Originally, this computation had the potential to include groups not in the segment being computed. [#163]
- Drop support for Python 3.8 [#162]
- The
metatag was missing when checking fordrop_frame1. It has been added to the check. [#161]
- Remove use of deprecated
pytest-openfilespytestplugin. This has been replaced by catchingResourceWarning. [#159]
- Updated the code for both NIR Snowballs and MIRI Showers. The snowball flagging will now extend the saturated core of snowballs. Also, circles are no longer used for snowballs preventing the huge circles of flagged pixels from a glancing CR. Shower code is completely new and is now able to find extended emission far below the single pixel SNR. It also allows detected showers to flag groups after the detection. [#144]
- During multiprocessing, if the number of processors requested are greater than the number of rows in the image, then ramp fitting errors out. To prevent this error, during multiprocessing, the number of processors actually used will be no greater than the number of rows in the image. [#154]
- Remove the
dqflags,dynamicdq, andbasic_utilsmodules and replace them with thin imports fromstdatamodelswhere the code as been moved. [#146] - update minimum version of
numpyto1.20and add minimum dependency testing to CI [#153] - restore
opencv-pythonto a hard dependency [#155]
- Changed computations for ramps that have only one good group in the 0th group. Ramps that have a non-zero groupgap should not use group_time, but (NFrames+1)*TFrame/2, instead. [#142]
- Fixed zeros that should be NaNs in rate and rateints product and suppressed a cast warning due to attempts to cast NaN to an integer. [#141]
- Modified dark class to support quantities in Roman.[#140]
- Changed a cast due to numpy deprecation that now throws a warning. The negation of a DQ flag then cast to a np.uint32 caused an over flow. The flag is now cast to a np.uint32 before negation. [#139]
- improve exception handling when attempting to use ellipses without
opencv-pythoninstalled [#136]
- use
toxenvironments [#130]
- Added support for Quantities in models required for the RomanCAL pipeline. [#124]
- Set values in the rate and rateints product to NaN when no usable data is available to compute slopes. [#131]
- Moved build configuration from
setup.cfgtopyproject.tomlto support PEP621 [#95] - made dependency on
opencv-pythonconditional [#126]
- Set saturation flag only for full saturation. The rateints product will have the saturation flag set for an integration only if saturation starts in group 0. The rate product will have the saturation flag set only if each integration for a pixel is marked as fully saturated. [#125]
- Changes to limit the expansion of MIRI shower ellipses to be the same number of pixels for both the major and minor axis. JP-2944 [#123]
- Bug fix for computation of the total number of frames when science data use on-board frame averaging and/or group gaps. [#121]
- Changes to flag both NIR snowballs and MIRI showers for JP-#2645. [#118]
- Early in the step, the object arrays are converted from DN to electrons by multiplying by the gain. The values need to be reverted back to DN at the end of the step. [#116]
- Made style changes due to the new 5.0.3 version of flake8, which noted many missing white spaces after keywords. [#114]
- Updating multi-integration processing to correctly combine multiple integration computations for the final image information. [#108]
- Fixed crash due to two group ramps with saturated groups that used an intermediate array with an incorrect shape. [#109]
- Updating how NaNs and DO_NOT_USE flags are handled in the rateints product. [#112]
- Updating how GLS handles bad gain values. NaNs and negative gain values have the DO_NOT_USE and NO_GAIN_VALUE flag set. Any NaNs found in the image data are set to 0.0 and the corresponding DQ flag is set to DO_NOT_USE. [#115]
- Added flagging after detected ramp jumps based on two DN thresholds and two number of groups to flag [#110]
- Adding special case handler for GLS to handle one group ramps. [#97]
- Updating how one group suppression and ZEROFRAME processing works with multiprocessing, as well as fixing the multiprocessing failure. [#99]
- Changing how ramp fitting handles fully saturated ramps. [#102]
- Modified the saturation threshold applied to pixels flagged with NO_SAT_CHECK, so that they never get flagged as saturated. [#106]
- The tuple
integ_infono longer returnsint_timesas a part of it, so the tuple is one element shorter. [#99] - For fully saturated exposures, all returned values are
None, instead of tuples. [#102]
- Changing parameter name in twopoint_difference from 'normal_rej_thresh' to rejection_thresh' for consistency. [#105]
- Update CI workflows to cache test environments and depend upon style and security checks [#96]
- Increased required
Pythonversion from>=3.7to>=3.8(to align withastropy) [#98]
- Update
twopoint_difference.py[#90]
- Updating the one good group ramp suppression handler works. [#92]
- Fix for accessing zero-frame in model to account for Roman data not using zero-frame. [#89]
- Enable multiprocessing for jump detection, which is controlled by the 'max_cores' parameter. [#87]
- Added functionality to linearly process ZEROFRAME data the same way as the SCI data. [#81]
- Added functionality to use ZEROFRAME data in place of group 0 data for ramps that are fully saturated, but still have good ZEROFRAME data. [#81]
- Added functionality to process ZEROFRAME data for saturation the same way as the SCI data. [#81]
- Added in functionality to deal with charge spilling from saturated pixels onto neighboring pixels [#83]
- Pin astropy min version to 5.0.4. [#82]
- Fix for jumps in first good group after dropping groups [#84]
- Neighboring pixels with 'SATURATION' or 'DONOTUSE' flags are no longer flagged as jumps. [#79]
- Adding feature to use ZEROFRAME for ramps that are fully saturated, but the ZEROFRAME data for that ramp is good. [#81]
- Adding feature to suppress calculations for saturated ramps having only the 0th group be a good group. [#76]
- Adding GLS code back to ramp fitting. [#64]
- Fix issue in jump detection that occurred when there were only 2 usable differences with no other groups flagged. This PR also added tests and fixed some of the logging statements in twopoint difference. [#74]
- fixes to several existing errors in the jump detection step. added additional tests to ensure step is no longer flagging jumps for pixels with only two usable groups / one usable diff. [#72]
- Moved dark current code from JWST to STCAL. [#63]
- Let software set the pixel dq flag to NO_LIN_CORR if linear term of linearity coefficient is zero. [#65]
- Fix special handling for 2 group ramp. [#70]
- Fix issue with inappropriately including a flagged group at the beginning of a ramp segment. [#68]
- Changed Ramp Fitting Documentation [#61]
- For slopes with negative median rates, the Poisson variance is zero. [#59]
- Changed the way the final DQ array gets computed when handling the DO_NOT_USE flag for multi-integration data. [#60]
- Reverts "Fix issue with flagging for MIRI three and four group integrations. [#44]
- Fix issue with flagging for MIRI three and four group integrations. [#44]
- Adds common code for linearity correction [#55]
- Global DQ variable removed [#54]
- Adds common code for saturation [#39]
- added tests for two point difference [#37]
- Adds support for Roman ramp data. [#43] [#49]
Workaround for setuptools_scm issues with recent versions of pip. [#45]
- documentation changes + docs for jump detection [#14]
- Fix ramp fitting multiprocessing. [#30]
- Move common
jumpcode to stcal [#27]
- Implemented multiprocessing for OLS. [#30]
- Added DQ flag parameter to ramp_fit [#25]
- Reduced data model dependency [#26]
- Fixed bug for median ramp rate computation in report JP-1950. [#12]
- Added ramp fitting code [#6]
- Added code to manipulate bitmasks.