Index: /trunk/doc/release.2015/ps1.detrend/detrend.tex =================================================================== --- /trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 37905) +++ /trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 37906) @@ -142,4 +142,6 @@ \section{Camera description} +\czwdraft{reference to original paper} + \czwdraft{60 otas} @@ -149,4 +151,6 @@ \czwdraft{Add summary of detrending steps} + +\czwdraft{Summary of detrending steps with references to the sections} \section{Burntool / Persistence effect} @@ -168,44 +172,58 @@ remnant charge leaks out, resulting in a trail that extends from the initial location of the bright source on the previous image towards -the amplifier (vertically down along the pixel column). This charge can remain on the detector for up -to thirty minutes, so the locations of these ``burns'' needs to be -retained between exposures. - -Both of these types of persistance trails are corrected via the BURNTOOL program. This -program does an initial scan of the images, and identifies stars -brighter than a given threshold. Then, the trail from that star is -fit with a one-dimensional power law, based on empirical evidence that -this is the functional form of this perseistence effect. Once this -fit is done, the model is subtracted from the image, and the location -of the star is stored in a table along with the exposure PONTIME -\czwdraft{obs time?}. - -For subsequent exposures, the table associated with the previous image -is read in, and after correcting trails from its own stars, it -attempts to find remnant trails from previous images. These are fit -and subtracted using a one-dimensional exponential model, again based -to empirical studies. If no significant model is determined, then -this location is not included in the output table, allowing old burns -to ``expire.'' - -One problem with this method to correct the persistance trails is that -it is based on fits to the image data, which may not be fully -determined by the persistance effect. The presence of other stars or -artifacts along the path of the burn can result in an incorrect model -to be determined, resulting in either an over- or under-subtraction of -the persistance burn. \czwdraft{However, it's better than doing nothing.} +the amplifier (vertically down along the pixel column). This charge +can remain on the detector for up to thirty minutes, requiring the +locations of these ``burns'' needs to be retained between exposures. + +Both of these types of persistance trails are corrected via the +BURNTOOL program. This program does an initial scan of the images, +and identifies stars brighter than a given threshold of 30000 DN. The +trail from that star is fit with a one-dimensional power law +\czwdraft{in each pixel column}, based on empirical evidence that this +is the functional form of this persistence effect. Once this fit is +done, the model is subtracted from the image, and the location of the +star is stored in a table along with the exposure PONTIME, which +denotes the number of seconds since the detector was last powered on. + +For a subsequent exposure, the table associated with the previous +image is read in, and after correcting trails from the stars on that +new image, it attempts to find remnant trails stored in the table. +These are fit and subtracted using a one-dimensional exponential +model, again based on empirical studies. If a significant model with +is determined \czwdraft{$\alpha$ < 4}, then this location is retained +in the image output table. If not, the old burn is allowed to +``expire.'' + +An issue with this method of correcting the persistance trails is that +it is based on fits to the raw image data, which may have other +signals not determined by the persistence effect. The presence of +other stars or artifacts along the path of the burn can result in an +incorrect model to be determined, resulting in either an over- or +under-subtraction of the persistance burn. \czwdraft{However, it's + better than doing nothing.} + Another issue is that the cores of very bright stars are deformed by -this process, as it preferentially subtracts flux from one side of the -star. As most stars that result in burns already have the cores -saturated, this does not significantly affect PSF determination or -photometry. - -\section{Mask} +this process, as the burntool fitting preferentially subtracts flux +from one side of the star. As most stars that result in burns already +have the cores saturated, this does not significantly affect PSF +determination or photometry. \czwdraft{reference to photometry paper?} + +\begin{figure} + \caption{Panel 1: Example image of burn trail. Panel 2: example image of subsequent image persistence trail. Panel 3: Repair of panel 1. Panel 4: Repair of panel 2} +\end{figure} + +\begin{figure} + \caption{example trail data and fit.} +\end{figure} + +\section{Masking} + +\subsection{Static Masks} Due to the large size of the detector, it is to be expected that there -will be a number of pixel defects that do not measure light as well as -their neighbors. To remove these pixels, we have constructed a static -mask that contains information about these defects. This mask is -constructed in three phases. +will be a number of pixel defects that \czwdraft{do not measure light} +as well as their neighbors. To remove these pixels, we have +constructed a static mask that identifies the known defects. This +mask is constructed in three phases. First, a CTEMASK is constructed to mask out regions in which the @@ -214,48 +232,126 @@ CTE issues, with this pattern showing up (to varying degrees) in triangular sets of cells on the OTA. \czwdraft{probably a figure would - help explain this?} To generate the mask, a sample set of flat -images are used to generate a map of the image variance with some -binning. As the flat image largely illuminates the image uniformly, -the expected variances should be Poissonian distributed with the flux -level. However, in regions with CTE issues, adjacent pixels are able -to ``share'' their charge, resulting in a lower-than-expected -variance. This allows these regions to be identified and removed from -processing in science images. + help explain this?} To generate the mask, a sample set of evenly +illuminated flat field images are measured to produce a map of the +image variance in 20x20 pixel bins. As the flat image largely +illuminates the image uniformly, the expected variances should be +Poissonian distributed with the flux level. However, in regions with +CTE issues, adjacent pixels are not independent, allowing the charge +to spread. This reduces the pixel-to-pixel differences, resulting in +a lower-than-expected variance. All regions with variance +\czwdraft{X} smaller than expected are added to the static CTEMASK. The next step of mask construction is to examine the detector for -bright columns and other pixel issues. This is first done by \czwdraft{I - think Heather wrote a program to do this, but I'm not totally sure - how it works} scanning a set of images for pixels that have values -that do not change throughout the sequence of exposures. Such pixels -cannot be caused by astronomical effects, and must be due to the -detector itself. This does an excellent job of removing the majority -of the problem pixels, and greatly speeds up the manual inspection for -defects. This manual inspection allows human interaction to identify -other odd detector issues that should not be allowed through to -science processing. This is also where the vignetted regions around -the edge of the detector are masked out. \czwdraft{This might be a lie} -As the size of the vignetted region changes with filter, we have been -somewhat aggressive about this, defining the smallest possible -``good'' region by using the g-filter to set this. - -Finally, not all bad regions on the image are due to pixel level -defects. Crosstalk between electronics channels results in the -appearance of faint ``stars'' that appear with the same cell (x,y) -coordinate as a real star, but are shifted to another cell or to -another OTA. We believe we have identified all such crosstalk issues, -and therefore place a mask over the crosstalk ghost when we detect a -sufficiently bright star in a ``source'' location. - -Due to an issue with the anti-reflective coating, we also see large -out of focus objects in the g-filter data. These objects are the -result of a bright source reflecting back off the surface of the -detector, reflecting again off the \czwdraft{No clue} mirror, and then -back down onto the focal plane. These are also somewhat reasonable to -identify, as a bright star in location (X,Y) on the focal plane -creates a reflection ghost at (-X,-Y). The exact location is fit as a -\czwdraft{Nth} order polynomial, and seems to sufficiently cover these -regions. - -\subsection{Optical ghosts} +bright columns and other static pixel issues. This is first done by +\czwdraft{I think Heather wrote a program to do this, but I'm not + totally sure how it works} scanning a set of images for pixels that +have values that do not change throughout a sequence of \czwdraft{N} +exposures. Such common pixel values cannot be caused by astronomical +effects, and must be due to the detector itself. This does an +excellent job of removing the majority of the problem pixels. A +manual inspection allows human interaction to identify other +inconsistent pixels including the vignetted regions around the edge of +the detector. \czwdraft{This might be a lie} As the size of the +vignetted region changes with filter, we have taken the g filter as +the baseline to define the static mask, resulting in the smallest +possible unvignetted region. + +The final static mask is the union of the CTE mask, the manual mask, \czwdraft{make this a paragraph}. + +\begin{figure} + \caption{Image map of static mask. color coded based on mask reason? It won't be visible at true pixel scale.} +\end{figure} + +\subsection{Dynamic masks} + +In addition to the static mask that removes the detector level +defects, we also generate a set of dynamic masks that change with the +astronomical features in the image. These masks are advisory in +nature, and no not completely exclude the pixel from further +consideration. The first of these dynamic masks indicates the +presence of a corrected burntool trail. These pixels are included for +phtometry, but are rejected more readily in the stacking and +difference image construction. + +The remaining dynamic masks are not generated until the IPP camera +stage \czwdraft{IPP paper reference?}, at which point all object +photometry is complete, and an astrometric solution is known for the +exposure. This added information provides the positions of bright +sources, which are the origin for the image artifacts that the dynamic +mask identifies. + +\subsubsection{Crosstalk ghosts} +Due to electrical crosstalk between the flex cables connecting the +individual detectors, ghost objects can be created on some OTAs due to +the presence of a bright object in a different position. Table +\ref{tab:crosstalk_rules} summarizes the list of known crosstalk +rules. In each of these cases, a source object brighter than -14.47 +magnitude (instrumental) creates a ghost object many orders of +magnitude fainter at the target location. The cell (x,y) coordinate +is identical between source and ghost, as a result of the transfer +occurring as the devices are read. A circular mask is asdded to the +ghost location with radius $R = 3.44 \left(-14.47 - m_{source, + instrumental}\right)$. Any objects in the photometric catalog found +at the location of the ghost mask have a flag set, marking the object +as a ghost. + +\begin{deluxetable}{lllc} + \tablecolumns{4} + \tablewidth{0pc} + \tablecaption{GPC1 Crosstalk Rules} + \tablehead{\colhead{Type}&\colhead{Source OTA/Cell}&\colhead{Ghost OTA/Cell}&\colhead{$\Delta m$}} + \startdata + Inter-OTA & OTA2Y XY3v & OTA3Y XY3v & 6.16 \\ + & OTA3Y XY3v & OTA2Y XY3v & \\ + & OTA4Y XY3v & OTA5Y XY3v & \\ + & OTA5Y XY3v & OTA4Y XY3v & \\ + Intra-OTA & OTA2Y XY5v & OTA2Y XY6v & 7.07 \\ + & OTA2Y XY6v & OTA2Y XY5v & \\ + & OTA5Y XY5v & OTA5Y XY6v & \\ + & OTA5Y XY6v & OTA5Y XY5v & \\ + One-way & OTA2Y XY7v & OTA3Y XY2v & 7.34 \\ + & OTA5Y XY7v & OTA4Y XY2v & \\ + \enddata + \label{tab:crosstalk_rules} +\end{deluxetable} + + +\subsubsection{Optical ghosts} + +Due to an issue with the anti-reflective coating, bright sources can +also result in large out of focus objects, particularly in the +g-filter data. These objects are the result of light reflecting back +off the surface of the detector, reflecting again off the \czwdraft{No + clue} mirror, and then back down onto the focal plane. Due to the +extra travel distance, the resulting source is out of focus and +elongated along the radial direction of the telescope. These optical +ghosts can be modeled as a bright star in location (X,Y) on the focal +plane creates a reflection ghost on the opposite side of the optical +axis at (-X,-Y). The exact location is fit as a third order +polynomial in the focal plane x and y directions. An elliptical +annulus mask is constructed at the expected ghost location, with the +major and minor axes defined by linear functions of the ghost distance +from the optical axis, and orientation \czwdraft{pointing along + radius}. All stars brighter than a filter-dependent threshold +(listed in table \ref{tab:ghost_magnitudes}) have masks constructed. + +\begin{deluxetable}{lc} + \tablecolumns{2} + \tablewidth{0pc} + \tablecaption{Optical Ghost Magnitude Limits} + \tablehead{\colhead{Filter}&\colhead{$m_{inst}$}} + \startdata + g & -16.5 \\ + r & -20.0 \\ + i & -25.0 \\ + z & -25.0 \\ + y & -25.0 \\ + w & -20.0 \\ + \enddata + \label{tab:ghost_magnitudes} +\end{deluxetable} + +\czwdraft{include full polynomial forms? How best to do that?} + %% @@ -320,5 +416,17 @@ %% END -\subsection{Glints} +\subsubsection{Glints} + +\czwdraft{I thought we stopped this because of a hardware change? Is + that not true?} Prior to \czwdraft{DATE}, a reflective surface at +the edge of the camera aperture was open to light passing through the +telescope. Sources brighter than $m = -20$ that fell on this +reflective surface resulted in light being scattered across the +detector surface in a long narrow glint. This surface was physically +masked on \czwdraft{DATE} \czwdraft{right?}, but data prior to that +have a dynamic mask constructed when a reference source falls on the +focal plane within \czwdraft{approximately} one degree of the detector +edge. This mask is 150 pixels wide, and $L = 2500 \left(-20 - +m_{inst}\right)$. %% @@ -346,4 +454,16 @@ %% END +\subsubsection{Diffraction spikes} + +Bright objects also form diffraction spikes that are dynamically +masked. These are filter independent, and are modelled as rectangles +with length $L = 10^{0.096 * (7.35 - m)} - 200$ and width $W = 8 + (L +- 200) * 0.01$. These spikes are dependent on the camera rotation, +and are oriented at $\theta = n * \frac{pi}{2} - \mathrm{ROTANGLE} + +0.798$. + +\subsubsection{Saturated stars} + +The cores of saturated stars are masked as well, with radius $r = 10.15 * (-15 - m_{inst})$. \czwdraft{good job here.} \czwdraft{Write up something about the masking fraction.} @@ -353,18 +473,31 @@ One aspect of the OTAs in GPC1 is that an individual cell can be read off repeatedly while the other cells integrate, resulting in a video -signal from that cell. This is used for guiding purposes, and a -single exposure is likely to have a number of these video cells. -However, reading these cells while integrating on the others changes -the characteristic dark model (see below) experienced by the other -cells on the OTA. The observational effect of this is that the glow -related to the amplifiers in the corners of the cells is depressed -during the video readout, relative to the nominal glow. Because of -this, the standard dark model oversubtracts this glow. Due to camera -configuration issues \czwdraft{I need to check this}, we are unable to -obtain video dark images, preventing us from correctly modelling this -change in the dark model. Instead, we apply simple masks that block -out these corner anti-glows from the data. This is reasonable, as -other than the corners, most pixels have the same dark model in either -mode. +signal from that cell. This data is used for telescope guiding +purposes, and a single exposure is likely to have a number of these +video cells in different OTAs. However, reading these cells while +integrating on the others changes the characteristic dark model (see +below) experienced by the other cells on the OTA. The observed effect +of this is that the glow associated with the amplifiers in the corners +of the cells is depressed during the video readout, relative to the +nominal glow. Because of this, the standard dark model oversubtracts +this glow. Before the nature of this issue was fully understood, +these poorly constrained corners were masked with 25-pixel radius +quarter circles, centered on the (0,0) pixel nearest the cell +amplifier. The other corners of the cell were masked with a 15-pixel +radius quarter circle, as the amplifier location is off the edge of +the cell. + + +\subsection{Masking fraction} + +\czwdraft{\% due to chip/cell gaps} + +\czwdraft{\% due to faulty pixels} + +\czwdraft{\% due to CTE} + +\czwdraft{\% due to vinetting} + +\czwdraft{\% average dynamic masking} \section{Overscan}