JP-3682: split outlier detection into separate steps for each mode

docs/jwst/outlier_detection/index.rst

@@ -1,15 +1,9 @@
 .. _outlier_detection_step:
 
-=================
-Outlier Detection
-=================
-
 .. toctree::
    :maxdepth: 2
 
    main.rst
-   arguments.rst
-   outlier_detection_step.rst
    outlier_detection_imaging.rst
    outlier_detection_coron.rst
    outlier_detection_tso.rst

docs/jwst/outlier_detection/main.rst

@@ -1,58 +1,16 @@
-Description
-===========
+=================
+Outlier Detection
+=================
 
-:Classes: `jwst.outlier_detection.OutlierDetectionStep`
-:Aliases: outlier_detection
+Outlier detection uses different algorithms based on the input data type, and so is handled
+in separate steps, as follows:
 
-Processing multiple datasets together allows for the identification of bad pixels
-or cosmic-rays that remain in each of the input images, many times at levels which
-were not detectable by the :ref:`jump <jump_step>` step. The ``outlier_detection`` step 
-implements the following algorithm to identify and flag any remaining cosmic-rays or
-other artifacts left over from previous calibrations:
+#. **Image modes**: 'FGS_IMAGE', 'MIR_IMAGE', 'NRC_IMAGE', 'NIS_IMAGE'
+#. **Spectroscopic modes**: 'MIR_LRS-FIXEDSLIT', 'NRS_FIXEDSLIT', 'NRS_MSASPEC'
+#. **Time-Series-Observation(TSO) Spectroscopic modes**: 'MIR_LRS-SLITLESS', 'NRC_TSGRISM', 'NIS_SOSS', 'NRS_BRIGHTOBJ'
+#. **IFU Spectroscopic modes**: 'MIR_MRS', 'NRS_IFU'
+#. **TSO Image modes**: 'NRC_TSIMAGE'
+#. **Coronagraphic Image modes**: 'MIR_LYOT', 'MIR_4QPM', 'NRC_CORON'
 
-#. build a stack of input data
-
-   - all inputs will need to have the same WCS since outlier detection assumes 
-     the same flux for each point on the sky, and variations from one image to
-     the next would indicate a problem with the detector during readout of that pixel
-   - if needed, each input will be resampled to a common output WCS
-
-#. create a median image from the stack of input data
 
-   - this median operation will ignore any input pixels which have a weight 
-     which is too low (<70% max weight)
-
-#. create "blotted" data from the median image to exactly match each original
-   input dataset
-
-#. perform a statistical comparison (pixel-by-pixel) between the median blotted
-   data with the original input data to look for pixels with values that are  
-   different from the mean value by more than some specified sigma 
-   based on the noise model
-
-   - the noise model used relies on the error array computed by previous 
-     calibration steps based on the readnoise and calibration errors
-
-#. flag the DQ array for the input data for any pixel (or affected neighboring
-   pixels) identified as a statistical outlier
-
-The outlier detection step serves as a single interface to apply this general 
-process to any JWST data, with specific variations of this algorithm for each 
-type of data.  Sub-classes of the outlier detection algorithm have been developed
-specifically for:
-
-#. Imaging data
-#. IFU spectroscopic data
-#. TSO data
-#. coronagraphic data
-#. spectroscopic data
-
-This allows the outlier_detection step to be tuned to the variations in each type 
-of JWST data.
-
-Reference Files
-===============
-
-The ``outlier_detection`` step uses the PARS-OUTLIERDETECTIONSTEP parameter reference file.
-
-.. include:: ../references_general/pars-outlierdetectionstep_reffile.inc
+FIXME: add links to the docs for each of these

docs/jwst/outlier_detection/outlier_detection_coron.rst

@@ -3,14 +3,15 @@
 Outlier Detection for Coronagraphic Data
 ========================================
 
-This module serves as the interface for applying ``outlier_detection`` to coronagraphic
+:Classes: `jwst.outlier_detection.OutlierDetectionCoronStep`
+:Aliases: outlier_detection_coron
+
+This module serves as the interface for applying outlier detection to coronagraphic
 image observations.
 
 Specifically, this routine performs the following operations:
 
 #. Extract parameter settings from input model and merge them with any user-provided values.
-   See :ref:`outlier detection arguments <outlier_detection_step_args>` for the full list
-   of parameters.
 
 #. Convert input data, as needed, to make sure it is in a format that can be processed.
    A :py:class:`~jwst.datamodels.CubeModel` serves as the basic format for all processing
@@ -30,4 +31,33 @@ Specifically, this routine performs the following operations:
 
 #. Update input data model DQ arrays with mask of detected outliers.
 
-.. automodapi:: jwst.outlier_detection.coron
+
+Step Arguments
+==============
+
+``--maskpt``
+  The percent of maximum weight to use as lower-limit for valid data;
+  valid values go from 0.0 to 1.0.
+
+``--snr``
+  The signal-to-noise values to use for bad pixel identification.
+  Valid values are any positive float.
+
+``--save_intermediate_results``
+  Specifies whether or not to save any intermediate products created
+  during step processing.
+
+``--good_bits``
+  The DQ bit values from the input image DQ arrays
+  that should be considered 'good' when building the weight mask. See
+  DQ flag :ref:`dq_parameter_specification` for details.
+
+
+Reference Files
+===============
+
+The ``outlier_detection_coron`` step uses the PARS-OUTLIERDETECTIONCORONSTEP parameter reference file.
+
+.. include:: ../references_general/pars-outlierdetectioncoronstep_reffile.inc
+
+.. automodapi:: jwst.outlier_detection.outlier_detection_coron_step

docs/jwst/outlier_detection/outlier_detection_ifu.rst

@@ -3,6 +3,9 @@
 Outlier Detection for IFU Data
 ==============================
 
+:Classes: `jwst.outlier_detection.OutlierDetectionIFUStep`
+:Aliases: outlier_detection_ifu
+
 This module serves as the interface for applying ``outlier_detection`` to IFU
 observations, like those taken with NIRSpec and MIRI.  The code implements the
 basic outlier detection algorithm searching for pixels that are consistent outliers
@@ -30,5 +33,31 @@ through this stack is determined and normalized. The normalization uses a local
 is greater than the input threshold percentage.  Pixels that are found to be outliers are flaged in in the DQ array.
 
 
+Step Arguments
+==============
+
+``--kernel_size``
+  The size of the kernel to use to normalize the pixel differences. The kernel size
+  must only contain odd values. Valid values are a pair of ints in a single string
+  (for example "7 7").
+
+``--threshold_percent``
+  The threshold (in percent) of the normalized minimum pixel difference used to identify bad pixels.
+  Pixels with   a normalized minimum pixel difference above this percentage are flagged as a outlier.
+
+``--ifu_second_check``
+  FIXME: WHAT IS THIS?
+
+``--save_intermediate_results``
+  Specifies whether or not to save any intermediate products created
+  during step processing.
+
+
+Reference Files
+===============
+
+The ``outlier_detection_ifu`` step uses the PARS-OUTLIERDETECTIONIFUSTEP parameter reference file.
+
+.. include:: ../references_general/pars-outlierdetectionifustep_reffile.inc
 
-.. automodapi:: jwst.outlier_detection.ifu
+.. automodapi:: jwst.outlier_detection.outlier_detection_ifu_step