Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3527 Update to docs for IFU outlier detection #8360

Merged
merged 6 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ documentation

- Correct the names of parameter options ``usigma`` and ``lsigma`` for ``sky_match``. [#8356]

- Updated ``outlier_detection`` for IFU data to explain the method more clearly. [#8360]


emicorr
-------

Expand Down
15 changes: 10 additions & 5 deletions docs/jwst/outlier_detection/outlier_detection_ifu.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,16 @@ in the :ref:`calwebb_detector1 <calwebb_detector1>` pipeline.
At this time it is believed that NIRSpec IFU data also have bad pixels that vary with time, though the time variation is
still under study.

An algorithm was developed to flag pixels that are outliers when compared to their neighbors for a set of
input files contained in an association. The neighbor pixel differences are the neighbors in spatial direction. For MIRI data ,the neighbor differences are found to the left and right of every
science pixel. While for NIRSpec data neighbor differences are
found between the pixels above and below every science pixel. The pixel differences for each input model
in the association is determined and is stored in a stack of pixel differences.
The basis of the outlier detection flagging for IFU data is to look for pixels on the detector
that are regularly discrepant from their neighbors, with a sharper division than could be explained
by the detector PSF. The algorithm flags pixels that are outliers when compared to their neighbors for a set of
input files contained in an association. The neighbor pixel differences are the neighbors in spatial direction.
hbushouse marked this conversation as resolved.
Show resolved Hide resolved
For MIRI data neighbor differences are found to the left and right of every
science pixel, while for NIRSpec data the neighbor differences are
determined from pixels above and below every science pixel. The difference between the MIRI MRS and NIRPSpec algorithm for finding the
spatial pixel differences is due to the opposite dispersion directions between the two instruments.
The pixel differences for each input model
in the association are determined and stored in a stack of pixel differences.
For each pixel the minimum difference
through this stack is determined and normalized. The normalization uses a local median of the difference array
(set by the kernel size). A pixel is flagged as an outlier if this normalized minimum difference
Expand Down
10 changes: 6 additions & 4 deletions jwst/outlier_detection/outlier_detection_ifu.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,10 @@ class OutlierDetectionIFU(OutlierDetection):
a. read in science data
b. Store computed neighbor differences for all the pixels.
The neighbor pixel differences are defined by the dispersion axis.
For MIRI (disp axis = 1) the neighbors to find differences are to the left and right of pixel
For NIRSpec (disp axis = 0) the neighbors to find the differences are above and below the pixel
For MIRI, with the dispersion axis along the y axis, the neighbors that are used to
to find the differences are to the left and right of pixel being examined.
hbushouse marked this conversation as resolved.
Show resolved Hide resolved
For NIRSpec, with the dispersion along the x axis, the neighbors that are used to
find the differences are above and below the pixel being examined.
3. For each input file store the minimum of the pixel neighbor differences
4. Comparing all the differences from all the input data find the minimum neighbor difference
5. Normalize minimum difference to local median of difference array
Expand Down Expand Up @@ -206,8 +208,8 @@ def flag_outliers(self, idet, uq_det, ndet_files,
# set all science data that have DO_NOT_USE to NAN
sci[bad] = np.nan

# Compute left and right differences (MIRI dispersion axis = 1)
# For NIRSpec dispersion axis = 0, these differences are top, bottom
# Compute left and right differences (MIRI dispersion axis = 1 along y axis)
# For NIRSpec dispersion axis = 0 (along the x axis), these differences are top, bottom
hbushouse marked this conversation as resolved.
Show resolved Hide resolved
# prepend = 0 has the effect of keeping the same shape as sci and
# for MIRI data (disp axis = 1) the first column = sci data
# OR
Expand Down