Changeset 37906

Timestamp:

Feb 11, 2015, 3:15:35 PM (11 years ago)

Author:

watersc1

Message:

Merged handwritten notes from last week.

File:

• trunk/doc/release.2015/ps1.detrend/detrend.tex (modified) (7 diffs)

trunk/doc/release.2015/ps1.detrend/detrend.tex

@@ -142 +142 @@
 \section{Camera description}
 
+\czwdraft{reference to original paper}
+
 \czwdraft{60 otas}
 
@@ -149 +151 @@
 
 \czwdraft{Add summary of detrending steps}
+
+\czwdraft{Summary of detrending steps with references to the sections}
 
 \section{Burntool / Persistence effect}
@@ -168 +172 @@
 remnant charge leaks out, resulting in a trail that extends from the
 initial location of the bright source on the previous image towards
-the amplifier (vertically down along the pixel column).  This charge can remain on the detector for up
-to thirty minutes, so the locations of these ``burns'' needs to be
-retained between exposures.
-
-Both of these types of persistance trails are corrected via the BURNTOOL program.  This
-program does an initial scan of the images, and identifies stars
-brighter than a given threshold.  Then, the trail from that star is
-fit with a one-dimensional power law, based on empirical evidence that
-this is the functional form of this perseistence effect.  Once this
-fit is done, the model is subtracted from the image, and the location
-of the star is stored in a table along with the exposure PONTIME
-\czwdraft{obs time?}.
-
-For subsequent exposures, the table associated with the previous image
-is read in, and after correcting trails from its own stars, it
-attempts to find remnant trails from previous images.  These are fit
-and subtracted using a one-dimensional exponential model, again based
-to empirical studies.  If no significant model is determined, then
-this location is not included in the output table, allowing old burns
-to ``expire.''
-
-One problem with this method to correct the persistance trails is that
-it is based on fits to the image data, which may not be fully
-determined by the persistance effect.  The presence of other stars or
-artifacts along the path of the burn can result in an incorrect model
-to be determined, resulting in either an over- or under-subtraction of
-the persistance burn. \czwdraft{However, it's better than doing nothing.}
+the amplifier (vertically down along the pixel column).  This charge
+can remain on the detector for up to thirty minutes, requiring the
+locations of these ``burns'' needs to be retained between exposures.
+
+Both of these types of persistance trails are corrected via the
+BURNTOOL program.  This program does an initial scan of the images,
+and identifies stars brighter than a given threshold of 30000 DN.  The
+trail from that star is fit with a one-dimensional power law
+\czwdraft{in each pixel column}, based on empirical evidence that this
+is the functional form of this persistence effect.  Once this fit is
+done, the model is subtracted from the image, and the location of the
+star is stored in a table along with the exposure PONTIME, which
+denotes the number of seconds since the detector was last powered on.
+
+For a subsequent exposure, the table associated with the previous
+image is read in, and after correcting trails from the stars on that
+new image, it attempts to find remnant trails stored in the table.
+These are fit and subtracted using a one-dimensional exponential
+model, again based on empirical studies.  If a significant model with
+is determined \czwdraft{$\alpha$ < 4}, then this location is retained
+in the image output table.  If not, the old burn is allowed to
+``expire.''
+
+An issue with this method of correcting the persistance trails is that
+it is based on fits to the raw image data, which may have other
+signals not determined by the persistence effect.  The presence of
+other stars or artifacts along the path of the burn can result in an
+incorrect model to be determined, resulting in either an over- or
+under-subtraction of the persistance burn. \czwdraft{However, it's
+  better than doing nothing.}  
+
 Another issue is that the cores of very bright stars are deformed by
-this process, as it preferentially subtracts flux from one side of the
-star.  As most stars that result in burns already have the cores
-saturated, this does not significantly affect PSF determination or
-photometry.
-
-\section{Mask}
+this process, as the burntool fitting preferentially subtracts flux
+from one side of the star.  As most stars that result in burns already
+have the cores saturated, this does not significantly affect PSF
+determination or photometry. \czwdraft{reference to photometry paper?}
+
+\begin{figure}
+  \caption{Panel 1: Example image of burn trail.  Panel 2: example image of subsequent image persistence trail.  Panel 3: Repair of panel 1.  Panel 4: Repair of panel 2}
+\end{figure}
+
+\begin{figure}
+  \caption{example trail data and fit.}
+\end{figure}
+
+\section{Masking}
+
+\subsection{Static Masks}
 
 Due to the large size of the detector, it is to be expected that there
-will be a number of pixel defects that do not measure light as well as
-their neighbors.  To remove these pixels, we have constructed a static
-mask that contains information about these defects.  This mask is
-constructed in three phases.
+will be a number of pixel defects that \czwdraft{do not measure light}
+as well as their neighbors.  To remove these pixels, we have
+constructed a static mask that identifies the known defects.  This
+mask is constructed in three phases.
 
 First, a CTEMASK is constructed to mask out regions in which the
@@ -214 +232 @@
 CTE issues, with this pattern showing up (to varying degrees) in
 triangular sets of cells on the OTA. \czwdraft{probably a figure would
-  help explain this?}  To generate the mask, a sample set of flat
-images are used to generate a map of the image variance with some
-binning.  As the flat image largely illuminates the image uniformly,
-the expected variances should be Poissonian distributed with the flux
-level.  However, in regions with CTE issues, adjacent pixels are able
-to ``share'' their charge, resulting in a lower-than-expected
-variance.  This allows these regions to be identified and removed from
-processing in science images.
+  help explain this?}  To generate the mask, a sample set of evenly
+illuminated flat field images are measured to produce a map of the
+image variance in 20x20 pixel bins.  As the flat image largely
+illuminates the image uniformly, the expected variances should be
+Poissonian distributed with the flux level.  However, in regions with
+CTE issues, adjacent pixels are not independent, allowing the charge
+to spread.  This reduces the pixel-to-pixel differences, resulting in
+a lower-than-expected variance.  All regions with variance
+\czwdraft{X} smaller than expected are added to the static CTEMASK.
 
 The next step of mask construction is to examine the detector for
-bright columns and other pixel issues.  This is first done by \czwdraft{I
-  think Heather wrote a program to do this, but I'm not totally sure
-  how it works} scanning a set of images for pixels that have values
-that do not change throughout the sequence of exposures.  Such pixels
-cannot be caused by astronomical effects, and must be due to the
-detector itself.  This does an excellent job of removing the majority
-of the problem pixels, and greatly speeds up the manual inspection for
-defects.  This manual inspection allows human interaction to identify
-other odd detector issues that should not be allowed through to
-science processing.  This is also where the vignetted regions around
-the edge of the detector are masked out.  \czwdraft{This might be a lie}
-As the size of the vignetted region changes with filter, we have been
-somewhat aggressive about this, defining the smallest possible
-``good'' region by using the g-filter to set this.
-
-Finally, not all bad regions on the image are due to pixel level
-defects.  Crosstalk between electronics channels results in the
-appearance of faint ``stars'' that appear with the same cell (x,y)
-coordinate as a real star, but are shifted to another cell or to
-another OTA.  We believe we have identified all such crosstalk issues,
-and therefore place a mask over the crosstalk ghost when we detect a
-sufficiently bright star in a ``source'' location.
-
-Due to an issue with the anti-reflective coating, we also see large
-out of focus objects in the g-filter data.  These objects are the
-result of a bright source reflecting back off the surface of the
-detector, reflecting again off the \czwdraft{No clue} mirror, and then
-back down onto the focal plane.  These are also somewhat reasonable to
-identify, as a bright star in location (X,Y) on the focal plane
-creates a reflection ghost at (-X,-Y).  The exact location is fit as a
-\czwdraft{Nth} order polynomial, and seems to sufficiently cover these
-regions.
-
-\subsection{Optical ghosts}
+bright columns and other static pixel issues.  This is first done by
+\czwdraft{I think Heather wrote a program to do this, but I'm not
+  totally sure how it works} scanning a set of images for pixels that
+have values that do not change throughout a sequence of \czwdraft{N}
+exposures.  Such common pixel values cannot be caused by astronomical
+effects, and must be due to the detector itself.  This does an
+excellent job of removing the majority of the problem pixels.  A
+manual inspection allows human interaction to identify other
+inconsistent pixels including the vignetted regions around the edge of
+the detector.  \czwdraft{This might be a lie} As the size of the
+vignetted region changes with filter, we have taken the g filter as
+the baseline to define the static mask, resulting in the smallest
+possible unvignetted region.
+
+The final static mask is the union of the CTE mask, the manual mask, \czwdraft{make this a paragraph}.
+
+\begin{figure}
+  \caption{Image map of static mask.  color coded based on mask reason?  It won't be visible at true pixel scale.}
+\end{figure}
+
+\subsection{Dynamic masks}
+
+In addition to the static mask that removes the detector level
+defects, we also generate a set of dynamic masks that change with the
+astronomical features in the image.  These masks are advisory in
+nature, and no not completely exclude the pixel from further
+consideration.  The first of these dynamic masks indicates the
+presence of a corrected burntool trail.  These pixels are included for
+phtometry, but are rejected more readily in the stacking and
+difference image construction.
+
+The remaining dynamic masks are not generated until the IPP camera
+stage \czwdraft{IPP paper reference?}, at which point all object
+photometry is complete, and an astrometric solution is known for the
+exposure.  This added information provides the positions of bright
+sources, which are the origin for the image artifacts that the dynamic
+mask identifies.
+
+\subsubsection{Crosstalk ghosts}
+Due to electrical crosstalk between the flex cables connecting the
+individual detectors, ghost objects can be created on some OTAs due to
+the presence of a bright object in a different position.  Table
+\ref{tab:crosstalk_rules} summarizes the list of known crosstalk
+rules.  In each of these cases, a source object brighter than -14.47
+magnitude (instrumental) creates a ghost object many orders of
+magnitude fainter at the target location.  The cell (x,y) coordinate
+is identical between source and ghost, as a result of the transfer
+occurring as the devices are read.  A circular mask is asdded to the
+ghost location with radius $R = 3.44 \left(-14.47 - m_{source,
+  instrumental}\right)$.  Any objects in the photometric catalog found
+at the location of the ghost mask have a flag set, marking the object
+as a ghost.
+
+\begin{deluxetable}{lllc}
+  \tablecolumns{4}
+  \tablewidth{0pc}
+  \tablecaption{GPC1 Crosstalk Rules}
+  \tablehead{\colhead{Type}&\colhead{Source OTA/Cell}&\colhead{Ghost OTA/Cell}&\colhead{$\Delta m$}}
+  \startdata
+  Inter-OTA & OTA2Y XY3v & OTA3Y XY3v & 6.16 \\
+            & OTA3Y XY3v & OTA2Y XY3v &      \\
+            & OTA4Y XY3v & OTA5Y XY3v &      \\
+            & OTA5Y XY3v & OTA4Y XY3v &      \\
+  Intra-OTA & OTA2Y XY5v & OTA2Y XY6v & 7.07 \\
+            & OTA2Y XY6v & OTA2Y XY5v &      \\
+            & OTA5Y XY5v & OTA5Y XY6v &      \\
+            & OTA5Y XY6v & OTA5Y XY5v &      \\
+  One-way   & OTA2Y XY7v & OTA3Y XY2v & 7.34 \\
+            & OTA5Y XY7v & OTA4Y XY2v &      \\
+  \enddata
+  \label{tab:crosstalk_rules}
+\end{deluxetable}
+  
+
+\subsubsection{Optical ghosts}
+
+Due to an issue with the anti-reflective coating, bright sources can
+also result in large out of focus objects, particularly in the
+g-filter data.  These objects are the result of light reflecting back
+off the surface of the detector, reflecting again off the \czwdraft{No
+  clue} mirror, and then back down onto the focal plane.  Due to the
+extra travel distance, the resulting source is out of focus and
+elongated along the radial direction of the telescope. These optical
+ghosts can be modeled as a bright star in location (X,Y) on the focal
+plane creates a reflection ghost on the opposite side of the optical
+axis at (-X,-Y).  The exact location is fit as a third order
+polynomial in the focal plane x and y directions.  An elliptical
+annulus mask is constructed at the expected ghost location, with the
+major and minor axes defined by linear functions of the ghost distance
+from the optical axis, and orientation \czwdraft{pointing along
+  radius}.  All stars brighter than a filter-dependent threshold
+(listed in table \ref{tab:ghost_magnitudes}) have masks constructed.
+
+\begin{deluxetable}{lc}
+  \tablecolumns{2}
+  \tablewidth{0pc}
+  \tablecaption{Optical Ghost Magnitude Limits}
+  \tablehead{\colhead{Filter}&\colhead{$m_{inst}$}}
+  \startdata
+  g & -16.5 \\
+  r & -20.0 \\
+  i & -25.0 \\
+  z & -25.0 \\
+  y & -25.0 \\
+  w & -20.0 \\
+  \enddata
+  \label{tab:ghost_magnitudes}
+\end{deluxetable}
+
+\czwdraft{include full polynomial forms?  How best to do that?}
+
 
 %%
@@ -320 +416 @@
 %% END
 
-\subsection{Glints}
+\subsubsection{Glints}
+
+\czwdraft{I thought we stopped this because of a hardware change?  Is
+  that not true?}  Prior to \czwdraft{DATE}, a reflective surface at
+the edge of the camera aperture was open to light passing through the
+telescope.  Sources brighter than $m = -20$ that fell on this
+reflective surface resulted in light being scattered across the
+detector surface in a long narrow glint.  This surface was physically
+masked on \czwdraft{DATE} \czwdraft{right?}, but data prior to that
+have a dynamic mask constructed when a reference source falls on the
+focal plane within \czwdraft{approximately} one degree of the detector
+edge.  This mask is 150 pixels wide, and $L = 2500 \left(-20 -
+m_{inst}\right)$.
 
 %%
@@ -346 +454 @@
 %% END
 
+\subsubsection{Diffraction spikes}
+
+Bright objects also form diffraction spikes that are dynamically
+masked.  These are filter independent, and are modelled as rectangles
+with length $L = 10^{0.096 * (7.35 - m)} - 200$ and width $W = 8 + (L
+- 200) * 0.01$.  These spikes are dependent on the camera rotation,
+and are oriented at $\theta = n * \frac{pi}{2} - \mathrm{ROTANGLE} +
+0.798$.
+
+\subsubsection{Saturated stars}
+
+The cores of saturated stars are masked as well, with radius $r = 10.15 * (-15 - m_{inst})$.  \czwdraft{good job here.}
 
 \czwdraft{Write up something about the masking fraction.}
@@ -353 +473 @@
 One aspect of the OTAs in GPC1 is that an individual cell can be read
 off repeatedly while the other cells integrate, resulting in a video
-signal from that cell.  This is used for guiding purposes, and a
-single exposure is likely to have a number of these video cells.
-However, reading these cells while integrating on the others changes
-the characteristic dark model (see below) experienced by the other
-cells on the OTA.  The observational effect of this is that the glow
-related to the amplifiers in the corners of the cells is depressed
-during the video readout, relative to the nominal glow.  Because of
-this, the standard dark model oversubtracts this glow.  Due to camera
-configuration issues \czwdraft{I need to check this}, we are unable to
-obtain video dark images, preventing us from correctly modelling this
-change in the dark model.  Instead, we apply simple masks that block
-out these corner anti-glows from the data.  This is reasonable, as
-other than the corners, most pixels have the same dark model in either
-mode.
+signal from that cell.  This data is used for telescope guiding
+purposes, and a single exposure is likely to have a number of these
+video cells in different OTAs.  However, reading these cells while
+integrating on the others changes the characteristic dark model (see
+below) experienced by the other cells on the OTA.  The observed effect
+of this is that the glow associated with the amplifiers in the corners
+of the cells is depressed during the video readout, relative to the
+nominal glow.  Because of this, the standard dark model oversubtracts
+this glow.  Before the nature of this issue was fully understood,
+these poorly constrained corners were masked with 25-pixel radius
+quarter circles, centered on the (0,0) pixel nearest the cell
+amplifier.  The other corners of the cell were masked with a 15-pixel
+radius quarter circle, as the amplifier location is off the edge of
+the cell.
+
+
+\subsection{Masking fraction}
+
+\czwdraft{\% due to chip/cell gaps}
+
+\czwdraft{\% due to faulty pixels}
+
+\czwdraft{\% due to CTE}
+
+\czwdraft{\% due to vinetting}
+
+\czwdraft{\% average dynamic masking}
 
 \section{Overscan}