Index: trunk/doc/release.2015/ps1.detrend/detrend.tex =================================================================== --- trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 40637) +++ trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 40638) @@ -236,6 +236,4 @@ will be made available in a future data release. -% \note{DS notes fonts are not consistent for keywords, etc} - \section{Background} @@ -252,5 +250,5 @@ The Pan-STARRS image processing pipeline (IPP) is described elsewhere -\citep{magnier2017.datasystem}, but a short summary follows. The raw +(Paper II), but a short summary follows. The raw image data is stored on the processing cluster, with a database containing the metadata of exposure parameters. These raw images can @@ -258,5 +256,5 @@ stage performs the image detrending (described below in section \ref{sec:detrending}), as well as the single epoch photometry -\citep{magnier2017.analysis}, in parallel on the individual OTA device +(Paper IV), in parallel on the individual OTA device data. Following the \IPPstage{chip} stage is the \IPPstage{camera} stage, in which the astrometry and photometry for the entire exposure @@ -282,5 +280,22 @@ uses the objects detected in that to perform forced photometry on the individual \IPPstage{warp} stage images. The details of these stages -are provided in \citet{magnier2017.analysis}. +are provided in Paper IV. + +\begin{figure}[htpb] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{{images/gpc1.layout}.pdf} + \caption{Diagram illustrating layout of OTA devices in GPC1. The + blue dots mark the locations of the amplifiers for xy00 cells in + each chip. When cells are mosaicked to a single pixel grid, the + pixel in this corner is at chip coordinate (1,1). The figure + illustrates the orientation of the OTA devices relative to the + parity of the sky. An exposure taken with North at the top of the + field-of-view will have East to the left when the OTA devices are + mosaicked as shown. Note that the devices OTA0Y - OTA3Y are + rotated by 180\degrees\ relative to the other half of the camera. + The labeling of the non-existent corner OTAs is provided to orient + the focal plane.} + \label{fig:gpc1.layout} +\end{figure} A limited version of the same reduction procedure described above is also @@ -308,4 +323,16 @@ section \ref{sec:discussion}. +\begin{figure*}[htpb] + \centering + \begin{minipage}{0.45\hsize} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_M_OS_NL_XY23_sm.png} + \end{minipage}% + \begin{minipage}{0.45\hsize} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_to_DARK_XY23_sm.png} + \end{minipage} + \caption{{\bf Dark Correction:} An example of the dark model application to exposure o5677g0123o, OTA23 (2011-04-26, 43s \gps{} filter). The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, and the detector non-linearity corrected. The right panel, shows the same exposure with the dark applied in addition to the processing shown on the left, removing the amplifier glows in the cell corners.} + \label{fig:dark image} +\end{figure*} + As mentioned above, the GPC1 camera is composed of 60 orthogonal transfer array (OTA) devices arranged in an $8\times{}8$ grid, @@ -313,8 +340,9 @@ $8\times{}8$ grid of readout cells consisting of $590 \times 598$ pixels. We label the OTAs by their coordinate in the camera grid in -the form `OTAXY', where X and Y each range from 0 - 7, e.g., OTA12 would -be the chip in the $(1,2)$ position of the grid. Similarly, we +the form `OTAXY', where X and Y each range from 0 - 7, e.g., OTA12 +would be the chip in the $(1,2)$ position of the grid. Similarly, we identify the cells as `xyXY' where X and Y again each range from 0 - -7. +7. Figure~\ref{fig:gpc1.layout} illustrates the physical layout of +the devices in the camera. Image products presented in figures have been mosaicked to arrange @@ -412,16 +440,4 @@ \label{sec:dark} -\begin{figure} - \centering - \begin{minipage}{0.45\hsize} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_M_OS_NL_XY23_sm.png} - \end{minipage}% - \begin{minipage}{0.45\hsize} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_to_DARK_XY23_sm.png} - \end{minipage} - \caption{{\bf Dark Correction:} An example of the dark model application to exposure o5677g0123o, OTA23 (2011-04-26, 43s \gps{} filter). The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, and the detector non-linearity corrected. The right panel, shows the same exposure with the dark applied in addition to the processing shown on the left, removing the amplifier glows in the cell corners.} - \label{fig:dark image} -\end{figure} - The dark current in the GPC1 detectors has significant variations across each cell. The model we make to remove this signal considers @@ -447,5 +463,5 @@ \subsubsection{Time evolution} -\begin{figure} +\begin{figure}[htpb] \centering \includegraphics[width=0.9\hsize,angle=0,clip]{images/B_profile_v1.pdf} @@ -524,4 +540,16 @@ significantly impact detrending. +\begin{figure*}[htpb] + \centering + \begin{minipage}{0.45\hsize} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_Rdark_XY22_sm.png} + \end{minipage}% + \begin{minipage}{0.45\hsize} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_VDdark_XY22_sm.png} + \end{minipage} + \caption{{\bf Video Dark:} An example of the video dark model application to exposure o5677g0123o, OTA22 (2011-04-26, 43s \gps{} filter), which has a video cell located in cell xy16. The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, the detector non-linearity corrected, and a regular dark applied. The right panel, shows the same exposure with a video dark applied instead of the standard dark. The main impact of this change is the improved correction of the corner glows, which are over subtracted with the standard dark.} + \label{fig:video_darks} +\end{figure*} + \subsubsection{Video Dark} \label{sec:video_darks} @@ -560,16 +588,4 @@ darks, with the early video dark constructed in such a manner. -\begin{figure} - \centering - \begin{minipage}{0.45\hsize} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_Rdark_XY22_sm.png} - \end{minipage}% - \begin{minipage}{0.45\hsize} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_VDdark_XY22_sm.png} - \end{minipage} - \caption{{\bf Video Dark:} An example of the video dark model application to exposure o5677g0123o, OTA22 (2011-04-26, 43s \gps{} filter), which has a video cell located in cell xy16. The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, the detector non-linearity corrected, and a regular dark applied. The right panel, shows the same exposure with a video dark applied instead of the standard dark. The main impact of this change is the improved correction of the corner glows, which are over subtracted with the standard dark.} - \label{fig:video_darks} -\end{figure} - \subsection{Noisemap} \label{sec:noisemap} @@ -610,96 +626,5 @@ from random Gaussian noise, we estimated the true read noise level. -As the noisemap uses bias frames that have had a dark model -subtracted, we constructed noisemaps for each dark model used for -science processing. There is some evidence that the noise has changed -over time as measured on full cells, so matching the noisemap to the -dark model allows for these changes to be tracked. There is no -evidence that the noisemap has the A/B modes found in the dark, so we -do not generate separate models for that time period. - -The noisemap detrend is not directly applied to the science image. -Instead, it is used to construct the weight image that contains the -pixel-by-pixel variance for the \IPPstage{chip} stage image. The -initial weight image is constructed by dividing the science image by -the cell gain (approximately 1.0 e$^{-} /$ DN). This weight image -contains the expected Poissonian variance in electrons measured. The -square of the noisemap is then added to this initial weight, adding -the additional empirical variance term in place of a single read noise -value. - -\subsection{Flat} - -Determining a flat field correction for GPC1 is a challenging -endeavor, as the wide field of view makes it difficult to construct a -uniformly illuminated image. Using a dome screen is not possible, as -the variations in illumination and screen rigidity create large -scatter between different images that are not caused by the detector -response function. Because of this, we use sky flat images taken at -twilight, which are more consistently illuminated than screen flats. -We calculate the mean of these images to determine the initial flat -model. - -From this starting skyflat model, we construct a photometric -correction to remove the effect of the illumination differences over -the detector surface. This is done by dithering a series of science -exposures with a given pointing, as described in -\citet{2004PASP..116..449M}. By fully calibrating these exposures -with the initial flat model, and then comparing the measured fluxes -for the same star as a function of position on the detector, we can -determine position dependent scaling factors. From the set of scaling -factors for the full catalog of stars observed in the dithered -sequence, we can construct a model of the error in the initial flat -model as a function of detector position. Applying a correction that -reduces the amplitude of these errors produces a flat field model that -better represents the true detector response. - -In addition to this flat field applied to the individual images, the -``ubercal'' analysis -- in which photometric data are used define -image zero points -\citep[][]{2012ApJ...756..158S,magnier2017.calibration} and in turn -used used to calibrate the database of all detections -- constructs -``in catalog'' flat field corrections. Although a single set of image -flat fields was used for the PV3 processing of the entire $3\pi$ -survey, five separate ``seasons'' of database flat fields were needed -to ensure proper calibration. This indicates that the flat field -response is not completely fixed in time. More details on this -process are contained in \citet{magnier2017.calibration}. - -\subsection{Fringe correction} -\label{sec:fringe} -% det_id 296 is the fringe we use. - -Due to variations in the thickness of the detectors, we observe -interference patterns at the infrared end of the filter set, as the -wavelength of the light becomes comparable to the thickness of the -detectors. Visually inspecting the images shows that the fringing is -most prevalent in the \yps{} filter images, with negligible fringing in the -other bands. As a result of this, we only apply a fringe correction -to the \yps{} filter data. - -The fringe used for PV3 processing was constructed from a set of 20 -120s science exposures. These exposures are overscan subtracted, and -corrected for non-linearity, and have the dark and flat models -applied. These images are smoothed with a Gaussian kernel with -$\sigma = 2$ pixels to minimize pixel to pixel noise. The fringe -image data is then constructed by calculating the clipped mean of the -input images with two iteration of clipping at the $3\sigma$ level. - -A coarse background model for each cell is constructed by calculating -the median on a 3x3 grid (approximately 200x200 pixels each). A set -of 1000 points are randomly selected from the fringe image for each -cell, and a median calculated for this position in a 10x10 pixel box, -with the background level subtracted. These sample locations provide -scale points to allow the amplitude of the measured fringe to be -compared to that found on science images. - -To apply the fringe, the same sample locations are measured on the -science image to determine the relative strength of the fringing in -that particular image. A least squares fit between the fringe -measurements and the corresponding measurements on the science image -provides the scale factor multiplied to the fringe before it is -subtracted from the science image. An example of the fringe correction can be seen in Figure~\ref{fig: fringe example}. - -\begin{figure} +\begin{figure*}[htpb] \centering \begin{minipage}{0.45\hsize} @@ -717,82 +642,83 @@ patterns. } \label{fig: fringe example} -\end{figure} - -\subsection{Masking} -\label{sec:masking} - -\subsubsection{Static Masks} -\label{sec:static_masks} - -Due to the large size of the detector, it is expected that there are a -number of pixels that respond poorly. To remove these pixels, we have -constructed a mask that identifies the known defects. This mask is -referred to as the ``static'' mask, as it is applied to all images -processed. The ``dynamic'' mask (Section \ref{sec:dynamic_masks}) is -calculated based on objects in the field, and so changes between -images. Construction of the static mask consists of three phases. - -First, regions in which the charge transfer efficiency (CTE) is low -compared to the rest of the detector are identified. Twenty-five of -the sixty OTAs in GPC1 show some evidence of poor CTE, with this -pattern appearing (to varying degrees) in roughly triangular patches. -During the manufacture of the devices, an improperly tuned -semiconductor process step resulted in a radial pattern of poor -performance on some silicon wafers. When the OTAs were cut from these -wafers, the outer corners exhibited the issue. To generate the mask -for these regions, a sample set of 26 evenly-illuminated flat-field -images were measured to produce a map of the image variance in 20x20 -pixel bins. As the flat screen is expected to illuminate the image -uniformly on this scale, the expected variances in each bin should be -Poissonian distributed with the flux level. However, in regions with -poor CTE, adjacent pixels are not independent, as the charge in those -pixels is more free to spread along the image columns. This reduces -the pixel-to-pixel differences, resulting in a lower than expected -variance. All regions with variance less than half the average image -level are added to the static mask. - - -The next step of mask construction is to examine the flat and dark -models, and exclude pixels that appear to be poorly corrected by these -models. The DARKMASK process looks for pixels that are more than -$8\sigma$ discrepant in $10\%$ of the 100 input dark frame images -after those images have had the dark model applied to them. These -pixels are assumed to be unstable with respect to the dark model, and -have the DARK bit set in the static mask, indicating that they are -unreliable in scientific observing. Similarly, the FLATMASK process -looks for pixels that are $3\sigma$ discrepant in the same fraction of -16 input flat field images after both the dark and flat models have -been applied. Those pixels that do not follow the flat field model of -the rest of image are assigned the FLAT mask bit in the static mask, -removing the pixels that cannot be corrected to a linear response. - -% http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/StaticMasks20101215 -The final step of mask construction is to examine the detector for -bright columns and other static pixel issues. This is first done by -processing a set of 100 \ips{} filter science images in the same fashion as -for the DARKMASK. A median image is constructed from these inputs -along with the per-pixel variance. These images are used to identify -pixels that have unexpectedly low variation between all inputs, as -well as those that significantly deviate from the global median value. -Once this initial set of bad pixels is identified, a $3\times{}3$ -pixel triangular kernel is convolved with the initial set, and any -convolved pixel with value greater than 1 is assigned to the static -mask. This does an excellent job of removing the majority of the -problem pixels. A subsequent manual inspection allows human -interaction to identify other inconsistent pixels including the -vignetted regions around the edge of the detector. - -Figure \ref{fig:static mask} shows an example of the static mask for -the full GPC1 field of view. Table \ref{tab:mask_values} lists the -bit mask values used for the different sources of masking. - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/gpc1_mask_indexed.png} - \caption{Image map of the GPC1 static mask. The CTE regions are clearly visible as roughly triangular patches covering the corners of some OTAs. Some entire cells are masked, including an entire column of cells on OTA14. Calcite cells remove large areas from OTA17 AND OTA76.} - \label{fig:static mask} -\end{figure} - -\begin{deluxetable*}{ccl} +\end{figure*} + +As the noisemap uses bias frames that have had a dark model +subtracted, we constructed noisemaps for each dark model used for +science processing. There is some evidence that the noise has changed +over time as measured on full cells, so matching the noisemap to the +dark model allows for these changes to be tracked. There is no +evidence that the noisemap has the A/B modes found in the dark, so we +do not generate separate models for that time period. + +The noisemap detrend is not directly applied to the science image. +Instead, it is used to construct the weight image that contains the +pixel-by-pixel variance for the \IPPstage{chip} stage image. The +initial weight image is constructed by dividing the science image by +the cell gain (approximately 1.0 e$^{-} /$ DN). This weight image +contains the expected Poissonian variance in electrons measured. The +square of the noisemap is then added to this initial weight, adding +the additional empirical variance term in place of a single read noise +value. + +\subsection{Flat} + +Determining a flat field correction for GPC1 is a challenging +endeavor, as the wide field of view makes it difficult to construct a +uniformly illuminated image. Using a dome screen is not possible, as +the variations in illumination and screen rigidity create large +scatter between different images that are not caused by the detector +response function. Because of this, we use sky flat images taken at +twilight, which are more consistently illuminated than screen flats. +We calculate the mean of these images to determine the initial flat +model. + +From this starting skyflat model, we construct a photometric +correction to remove the effect of the illumination differences over +the detector surface. This is done by dithering a series of science +exposures with a given pointing, as described in +\citet{2004PASP..116..449M}. By fully calibrating these exposures +with the initial flat model, and then comparing the measured fluxes +for the same star as a function of position on the detector, we can +determine position dependent scaling factors. From the set of scaling +factors for the full catalog of stars observed in the dithered +sequence, we can construct a model of the error in the initial flat +model as a function of detector position. Applying a correction that +reduces the amplitude of these errors produces a flat field model that +better represents the true detector response. + +In addition to this flat field applied to the individual images, the +``ubercal'' analysis -- in which photometric data are used define +image zero points +\citep[][]{2012ApJ...756..158S,magnier2017.calibration} and in turn +used used to calibrate the database of all detections -- constructs +``in catalog'' flat field corrections. Although a single set of image +flat fields was used for the PV3 processing of the entire $3\pi$ +survey, five separate ``seasons'' of database flat fields were needed +to ensure proper calibration. This indicates that the flat field +response is not completely fixed in time. More details on this +process are contained in Paper V. + +\subsection{Fringe correction} +\label{sec:fringe} +% det_id 296 is the fringe we use. + +Due to variations in the thickness of the detectors, we observe +interference patterns at the infrared end of the filter set, as the +wavelength of the light becomes comparable to the thickness of the +detectors. Visually inspecting the images shows that the fringing is +most prevalent in the \yps{} filter images, with negligible fringing in the +other bands. As a result of this, we only apply a fringe correction +to the \yps{} filter data. + +The fringe used for PV3 processing was constructed from a set of 20 +120s science exposures. These exposures are overscan subtracted, and +corrected for non-linearity, and have the dark and flat models +applied. These images are smoothed with a Gaussian kernel with +$\sigma = 2$ pixels to minimize pixel to pixel noise. The fringe +image data is then constructed by calculating the clipped mean of the +input images with two iteration of clipping at the $3\sigma$ level. + +\begin{deluxetable*}{ccl}[htp] \tablecolumns{3} \tablewidth{0pc} @@ -822,4 +748,114 @@ \end{deluxetable*} +A coarse background model for each cell is constructed by calculating +the median on a 3x3 grid (approximately 200x200 pixels each). A set +of 1000 points are randomly selected from the fringe image for each +cell, and a median calculated for this position in a 10x10 pixel box, +with the background level subtracted. These sample locations provide +scale points to allow the amplitude of the measured fringe to be +compared to that found on science images. + +To apply the fringe, the same sample locations are measured on the +science image to determine the relative strength of the fringing in +that particular image. A least squares fit between the fringe +measurements and the corresponding measurements on the science image +provides the scale factor multiplied to the fringe before it is +subtracted from the science image. An example of the fringe +correction can be seen in Figure~\ref{fig: fringe example}. + +\subsection{Masking} +\label{sec:masking} + +\subsubsection{Static Masks} +\label{sec:static_masks} + +Due to the large size of the detector, it is expected that there are a +number of pixels that respond poorly. To remove these pixels, we have +constructed a mask that identifies the known defects. This mask is +referred to as the ``static'' mask, as it is applied to all images +processed. The ``dynamic'' mask (Section \ref{sec:dynamic_masks}) is +calculated based on objects in the field, and so changes between +images. Construction of the static mask consists of three phases. + +First, regions in which the charge transfer efficiency (CTE) is low +compared to the rest of the detector are identified. Twenty-five of +the sixty OTAs in GPC1 show some evidence of poor CTE, with this +pattern appearing (to varying degrees) in roughly triangular patches. +During the manufacture of the devices, an improperly tuned +semiconductor process step resulted in a radial pattern of poor +performance on some silicon wafers. When the OTAs were cut from these +wafers, the outer corners exhibited the issue. To generate the mask +for these regions, a sample set of 26 evenly-illuminated flat-field +images were measured to produce a map of the image variance in 20x20 +pixel bins. As the flat screen is expected to illuminate the image +uniformly on this scale, the expected variances in each bin should be +Poissonian distributed with the flux level. However, in regions with +poor CTE, adjacent pixels are not independent, as the charge in those +pixels is more free to spread along the image columns. This reduces +the pixel-to-pixel differences, resulting in a lower than expected +variance. All regions with variance less than half the average image +level are added to the static mask. + +The next step of mask construction is to examine the flat and dark +models, and exclude pixels that appear to be poorly corrected by these +models. The DARKMASK process looks for pixels that are more than +$8\sigma$ discrepant in $10\%$ of the 100 input dark frame images +after those images have had the dark model applied to them. These +pixels are assumed to be unstable with respect to the dark model, and +have the DARK bit set in the static mask, indicating that they are +unreliable in scientific observing. Similarly, the FLATMASK process +looks for pixels that are $3\sigma$ discrepant in the same fraction of +16 input flat field images after both the dark and flat models have +been applied. Those pixels that do not follow the flat field model of +the rest of image are assigned the FLAT mask bit in the static mask, +removing the pixels that cannot be corrected to a linear response. + +\begin{figure}[b] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/gpc1_mask_indexed.png} + \caption{Image map of the GPC1 static mask. The CTE regions are clearly visible as roughly triangular patches covering the corners of some OTAs. Some entire cells are masked, including an entire column of cells on OTA14. Calcite cells remove large areas from OTA17 AND OTA76.} + \label{fig:static mask} +\end{figure} + +\begin{deluxetable}{lllc}[htpb] + \tablecolumns{4} + \tablewidth{0pc} + \tablecaption{GPC1 Crosstalk Rules} + \tablehead{\colhead{Type}&\colhead{Source OTA/Cell}&\colhead{Ghost OTA/Cell}&\colhead{$\Delta m$}} + \startdata + Inter-OTA & OTA2Y XY3v & OTA3Y XY3v & 6.16 \\ + & OTA3Y XY3v & OTA2Y XY3v & \\ + & OTA4Y XY3v & OTA5Y XY3v & \\ + & OTA5Y XY3v & OTA4Y XY3v & \\ + Intra-OTA & OTA2Y XY5v & OTA2Y XY6v & 7.07 \\ + & OTA2Y XY6v & OTA2Y XY5v & \\ + & OTA5Y XY5v & OTA5Y XY6v & \\ + & OTA5Y XY6v & OTA5Y XY5v & \\ + One-way & OTA2Y XY7v & OTA3Y XY2v & 7.34 \\ + & OTA5Y XY7v & OTA4Y XY2v & \\ + \enddata + \label{tab:crosstalk_rules} +\end{deluxetable} + +% http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/StaticMasks20101215 +The final step of mask construction is to examine the detector for +bright columns and other static pixel issues. This is first done by +processing a set of 100 \ips{} filter science images in the same fashion as +for the DARKMASK. A median image is constructed from these inputs +along with the per-pixel variance. These images are used to identify +pixels that have unexpectedly low variation between all inputs, as +well as those that significantly deviate from the global median value. +Once this initial set of bad pixels is identified, a $3\times{}3$ +pixel triangular kernel is convolved with the initial set, and any +convolved pixel with value greater than 1 is assigned to the static +mask. This does an excellent job of removing the majority of the +problem pixels. A subsequent manual inspection allows human +interaction to identify other inconsistent pixels including the +vignetted regions around the edge of the detector. + +Figure \ref{fig:static mask} shows an example of the static mask for +the full GPC1 field of view. Table~\ref{tab:mask_values} lists the +bit mask values used for the different sources of masking. + \subsubsection{Dynamic masks} \label{sec:dynamic_masks} @@ -884,55 +920,5 @@ pixels. -\paragraph{Optical ghosts} -\label{sec:optical_ghosts} - -The anti-reflective coating on the optical surfaces of GPC1 is less -effective at shorter wavelengths, which can allow bright sources to -reflect back onto the focal plane and generate large out-of-focus -objects. Due to the wavelength dependence, these objects are most -prominent in the \gps{} filter data. These objects are the result of -light reflecting back off the surface of the detector, reflecting -again off the lower surfaces of the optics (particularly the L1 -corrector lens), and then back down onto the focal plane. Due to the -extra travel distance, the resulting source is out of focus and -elongated along the radial direction of the camera focal -plane. Figure~\ref{fig:optical ghosts} shows an example exposure with -several prominent optical ghosts. - -These optical ghosts can be modeled in the focal plane coordinates -($L,M$) which has its origin at the center of the focal plane. In -this system, a bright object at location ($L,M$) on the focal plane -creates a reflection ghost on the opposite side of the optical axis -near ($-L,-M$). The exact location is fit as a third order polynomial -in the focal plane $L$ and $M$ directions (as listed in Table -\ref{tab:ghost_centers}). An elliptical annulus mask is constructed -at the expected ghost location, with the major and minor axes of the inner and outer elliptical annuli defined -by linear functions of the ghost distance from the optical axis, and -oriented with the ellipse major axis is along the radial direction -(Table \ref{tab:ghost_radii}). All stars brighter than a -filter-dependent threshold (listed in Table -\ref{tab:ghost_magnitudes}) have such masks constructed. - -\begin{deluxetable}{lllc} - \tablecolumns{4} - \tablewidth{0pc} - \tablecaption{GPC1 Crosstalk Rules} - \tablehead{\colhead{Type}&\colhead{Source OTA/Cell}&\colhead{Ghost OTA/Cell}&\colhead{$\Delta m$}} - \startdata - Inter-OTA & OTA2Y XY3v & OTA3Y XY3v & 6.16 \\ - & OTA3Y XY3v & OTA2Y XY3v & \\ - & OTA4Y XY3v & OTA5Y XY3v & \\ - & OTA5Y XY3v & OTA4Y XY3v & \\ - Intra-OTA & OTA2Y XY5v & OTA2Y XY6v & 7.07 \\ - & OTA2Y XY6v & OTA2Y XY5v & \\ - & OTA5Y XY5v & OTA5Y XY6v & \\ - & OTA5Y XY6v & OTA5Y XY5v & \\ - One-way & OTA2Y XY7v & OTA3Y XY2v & 7.34 \\ - & OTA5Y XY7v & OTA4Y XY2v & \\ - \enddata - \label{tab:crosstalk_rules} -\end{deluxetable} - -\begin{deluxetable}{lcc} +\begin{deluxetable}{lcc}[htpb] \tablecolumns{3} \tablewidth{0pc} @@ -954,9 +940,26 @@ \end{deluxetable} -\begin{deluxetable*}{lcccc} +\paragraph{Optical ghosts} +\label{sec:optical_ghosts} + +The anti-reflective coating on the optical surfaces of GPC1 is less +effective at shorter wavelengths, which can allow bright sources to +reflect back onto the focal plane and generate large out-of-focus +objects. Due to the wavelength dependence, these objects are most +prominent in the \gps{} filter data. These objects are the result of +light reflecting back off the surface of the detector, reflecting +again off the lower surfaces of the optics (particularly the L1 +corrector lens), and then back down onto the focal plane. Due to the +extra travel distance, the resulting source is out of focus and +elongated along the radial direction of the camera focal +plane. Figure~\ref{fig:optical ghosts} shows an example exposure with +several prominent optical ghosts. + +\begin{deluxetable*}{lcccc}[htpb] \tablecolumns{5} \tablewidth{0pc} \tablecaption{Optical Ghost Annulus Axis Length} \tablehead{\colhead{Radial Order}&\colhead{Inner Major Axis}&\colhead{Inner Minor Axis}&\colhead{Outer Major Axis}&\colhead{Outer Minor Axis}} + % \tablehead{\colhead{Order}&\colhead{Maj$_{\rm in}$}&\colhead{Min$_{\rm in}$}& \colhead{Maj$_{\rm out}$}&\colhead{Min$_{\rm out}$}} \startdata $r^0$ & 3.926693e+01 & 5.287548e+01 & 7.928722e+01 & 1.314265e+02 \\ @@ -966,17 +969,178 @@ \end{deluxetable*} -%% \begin{deluxetable}{lcccc} -%% \tablecolumns{5} -%% \tablewidth{0pc} -%% \tablecaption{Optical Ghost Annulus Axis Length} -%% \tablehead{\colhead{Order}&\colhead{Maj$_{\rm in}$}&\colhead{Min$_{\rm in}$}& \colhead{Maj$_{\rm out}$}&\colhead{Min$_{\rm out}$}} -%% \startdata +These optical ghosts can be modeled in the focal plane coordinates +($L,M$) which has its origin at the center of the focal plane. In +this system, a bright object at location ($L,M$) on the focal plane +creates a reflection ghost on the opposite side of the optical axis +near ($-L,-M$). The exact location is fit as a third order polynomial +in the focal plane $L$ and $M$ directions (as listed in Table +\ref{tab:ghost_centers}). An elliptical annulus mask is constructed +at the expected ghost location, with the major and minor axes of the inner and outer elliptical annuli defined +by linear functions of the ghost distance from the optical axis, and +oriented with the ellipse major axis is along the radial direction +(Table \ref{tab:ghost_radii}). All stars brighter than a +filter-dependent threshold (listed in Table +\ref{tab:ghost_magnitudes}) have such masks constructed. + +%% \begin{table*}[htpb] +%% \begin{center} +%% % \tablecolumns{5} +%% % \tablewidth{0pc} +%% % \tablecaption{Optical Ghost Annulus Axis Length} +%% \caption{Optical Ghost Annulus Axis Length\label{tab:ghost_radii}} +%% \begin{tabular}{lcccc} +%% % \tablehead{\colhead{Radial Order}&\colhead{Inner Major Axis}&\colhead{Inner Minor Axis}&\colhead{Outer Major Axis}&\colhead{Outer Minor Axis}} +%% % \startdata +%% \hline +%% \hline +%% {\bf Radial Order}&{\bf Inner Major Axis}&{\bf Inner Minor Axis}&{\bf Outer Major Axis}&{\bf Outer Minor Axis} \\ +%% \hline %% $r^0$ & 3.926693e+01 & 5.287548e+01 & 7.928722e+01 & 1.314265e+02 \\ %% $r^1$ & 5.325759e-03 &-2.191669e-03 & 1.722181e-02 & -2.627153e-03 \\ -%% \enddata -%% \label{tab:ghost_radii} -%% \end{deluxetable} - -\begin{deluxetable}{lrr} +%% \hline +%% \end{tabular} +%% \end{center} +%% \end{table*} + +\paragraph{Optical glints} +\label{sec:glints} + +Prior to 2010-08-24, a reflective surface at the edge of the camera +aperture was incompletely screened to light passing through the +telescope. Sources brighter than $m_{inst} = -21$ ($\rps \lesssim +7.5$) that fell on this reflective surface resulted in light being +scattered across the detector surface in a long narrow glint. +Figure~\ref{fig:optical glints} shows an example exposure with +a prominent optical glint. + +This reflective surface in the camera was physically masked on +2010-08-24, removing the possibility of glints in subsequent data, but +images that were taken prior to this date have an advisory dynamic +mask constructed when a reference source falls on the focal plane +within one degree of the detector edge. This mask is 150 pixels wide, +with length $L = 2500 \left(-20 - m_{inst}\right)$ pixels. These +glint masks are constructed by selecting sufficiently bright sources +in the reference catalog that fall within rectangular regions around +each edge of the GPC1 camera. These regions are separated from the +edge of the camera by 17 arcminutes, and extend outwards an additional +degree. + +\paragraph{Diffraction Spikes and Saturated Stars} +\label{sec:diffraction_spikes} + +Bright sources also form diffraction spikes that are dynamically +masked. These are filter independent, and are modeled as rectangles +with length $L = 10^{0.096 \times (7.35 - m_{inst})} - 200$ and +width $W = 8 + (L - 200) \times 0.01$, with negative values indicating no +mask is constructed, as the source is likely too faint to produce the +feature. These spikes are dependent on the camera rotation, and are +oriented based on the header keyword at $\theta = n \times \frac{\pi}{2} - +\mathrm{ROTANGLE} + 0.798$, for $n = {0,1,2,3}$. + +The cores of stars that are saturated are masked as well, with a +circular mask radius $r = 10.15 \times (-15 - m_{inst})$. An +example of a saturated star, with the masked regions for the +diffraction spikes and core saturation highlighted, is shown in Figure +\ref{fig:saturated star}. + +Saturation for the GPC1 detectors varies from chip to chip and cell to +cell. Saturation levels have been measured in the lab for each cell +and are recorded in the headers. The IPP analysis code reads the +header value to determine the appropriate saturation point. Of the +3840 cells in GPC1, the median saturation level is 60,400; 95\% have +saturation levels $> 54,500$ DN; 99\% have saturation levels $> +41,000$ DN. A small number of cells have recorded saturation values +much lower than these values, but these also tend to be the cells for +which other cosmetic effects (\eg, CTE \& dark current) are strong, +likely affecting the measurement of the saturation value. + +\begin{figure*}[htpb] + \centering +% \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_ghosts.jpg} +% \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_ghosts_sm.png} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/GPC1_Ghosts_with_Zoom.png} + \caption{{\bf Ghosts:} Example of optical ghosts in GPC1. The + central $6 \times 6$ detectors from exposure o5677g0123o + (2011-04-26, 43s \gps{} filter) are shown. The dashed red lines + link three example sets of stellar sources and the destinations of + the corresponding ghosts. The insets zoom in on these ghosts and + highlight the increasingly distorted images away from the optical + axis. The bright star on OTA33 results in a nearly circular ghost + on the opposite OTA. In contrast, the trio of stars on OTA11 + result in very elongated ghosts on OTA66, in the upper left + corner.} + \label{fig:optical ghosts} +\end{figure*} + +\begin{figure*}[htpb] + \centering +% \includegraphics[width=0.9\hsize,angle=0,clip]{images/glint_example_o5379g0103o.jpg} + \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_glints_sm.png} + \caption{{\bf Glints:} Example of a glint on exposure o5379g0103o (2010-07-02, 45s \ips{} filter). The source star out of the field of view creates a long reflection that extends through OTA73 and OTA63.} + \label{fig:optical glints} +\end{figure*} + +\begin{figure}[htpb] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/o6802g0338o_SATSTAR_XY51_sm.png} + \caption{Example of saturated star, with diffraction spikes extending from the core on exposure o6802g0338o, OTA51 (2014-05-25, 45s \gps{} filter).} + \label{fig:saturated star} +\end{figure} + +\subsubsection{Masking Fraction} +\label{sec:masking_fraction} + +The GPC1 camera was designed such that where possible, OTAs with CTE +issues were placed towards the edge of the detector. Because of this, +the main analysis of the mask fraction is based not on the total +footprint of the detector, but upon a circular reference field of view +with a radius of 1.5 degrees. This radius corresponds approximately +to half the width and height of the detector. This field of view +underestimates the unvignetted region of GPC1. A second ``maximum'' +field of view is also used to estimate the mask fraction within a +larger 1.628 degree radius. This larger radius includes far larger +missing fractions due to the circular regions outside region populated +with OTAs, but does include the contribution from well-illuminated +pixels that are ignored by the reference radius. + +The results of simulating the footprint of the detector as a grid of +uniformly sized pixels of $0\farcs{}258$ size are provided in Table +\ref{tab:mask fraction}. Both fields of view contain circular +segments outside of the footprint of the detector, which increase the +area estimate that is unpopulated. This category also accounts for +the inter-OTA and inter-cell gaps. The regions with poor CTE also +contribute to a significant fraction of the masked pixels. The +remaining mask category accounts for known bad columns, cells that do +not calibrate well, and vignetting. There are also a small fraction +that have static advisory masks marked on all images. These masks +mark regions where bright columns on one cell periodically create +cross talk ghosts on other cells. + +%% summary of different masking fractions: +%% 64 60 Ch 3.00 3.25 +%% Good pix : 71.28 76.030 76.0 78.9 71.1 +%% Off Chip : 15.700 10.083 10.1 13.1 19.6 +%% Flaws : 3.296 3.515 10.7 +%% Flat : 4.541 4.844 +%% Various : 2.157 2.303 +%% CTE : 2.104 2.244 2.2 2.3 2.6 +%% Other : 0.638 0.681 1.0 5.4 6.4 +%% advisory : 0.3 0.3 +%% +%% 64, 60 : from CZW comment in Chambers et al: masking fractions +%% counting the full set of 64 (theoretical) or 60 chips + +%% Ch : totals from Table 3 in Chambers et al, matches '60' + +%% 3.00, 3.25 : from Table 6 this paper: masking fractions for 3 and +%% 3.25 deg FOV circles assuming a theoretical fixed focal plane pixel +%% grid. This analysis uses the accounting in the gpc1 database table +%% and compares with a nominal number of pixels in the circles. + +%% Unpopulated = BLANK, DETECTOR, FLAT, DARK, CTE +%% I'm not sure where his CTE value comes from (not the database query) +%% Other = CR, SPIKE, GHOST, STARCORE [Ghost & Spike probably dominate] + +\begin{deluxetable}{lrr}[b] \tablecolumns{3} \tablewidth{0pc} @@ -995,113 +1159,4 @@ \end{deluxetable} -\paragraph{Optical glints} -\label{sec:glints} - -Prior to 2010-08-24, a reflective surface at the edge of the camera -aperture was incompletely screened to light passing through the -telescope. Sources brighter than $m_{inst} = -21$ ($\rps \lesssim -7.5$) that fell on this reflective surface resulted in light being -scattered across the detector surface in a long narrow glint. -Figure~\ref{fig:optical glints} shows an example exposure with -a prominent optical glint. - -This reflective surface in the camera was physically masked on -2010-08-24, removing the possibility of glints in subsequent data, but -images that were taken prior to this date have an advisory dynamic -mask constructed when a reference source falls on the focal plane -within one degree of the detector edge. This mask is 150 pixels wide, -with length $L = 2500 \left(-20 - m_{inst}\right)$ pixels. These -glint masks are constructed by selecting sufficiently bright sources -in the reference catalog that fall within rectangular regions around -each edge of the GPC1 camera. These regions are separated from the -edge of the camera by 17 arcminutes, and extend outwards an additional -degree. - -\paragraph{Diffraction Spikes and Saturated Stars} -\label{sec:diffraction_spikes} - -Bright sources also form diffraction spikes that are dynamically -masked. These are filter independent, and are modeled as rectangles -with length $L = 10^{0.096 \times (7.35 - m_{inst})} - 200$ and -width $W = 8 + (L - 200) \times 0.01$, with negative values indicating no -mask is constructed, as the source is likely too faint to produce the -feature. These spikes are dependent on the camera rotation, and are -oriented based on the header keyword at $\theta = n \times \frac{\pi}{2} - -\mathrm{ROTANGLE} + 0.798$, for $n = {0,1,2,3}$. - -The cores of stars that are saturated are masked as well, with a -circular mask radius $r = 10.15 \times (-15 - m_{inst})$. An -example of a saturated star, with the masked regions for the -diffraction spikes and core saturation highlighted, is shown in Figure -\ref{fig:saturated star}. - -Saturation for the GPC1 detectors varies from chip to chip and cell to -cell. Saturation levels have been measured in the lab for each cell -and are recorded in the headers. The IPP analysis code reads the -header value to determine the appropriate saturation point. Of the -3840 cells in GPC1, the median saturation level is 60,400; 95\% have -saturation levels $> 54,500$ DN; 99\% have saturation levels $> -41,000$ DN. A small number of cells have recorded saturation values -much lower than these values, but these also tend to be the cells for -which other cosmetic effects (\eg, CTE \& dark current) are strong, -likely affecting the measurement of the saturation value. - -\begin{figure} - \centering -% \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_ghosts.jpg} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_ghosts_sm.png} - \caption{{\bf Ghosts:} Example of the full GPC1 field of view - illustrating the sources and destinations of optical ghosts on - exposure o5677g0123o (2011-04-26, 43s \gps{} filter). The bright - stars on OTA33 and OTA44 result in nearly circular ghosts on the - opposite OTA. In contrast, the trio of stars on OTA11 result in - very elongated ghosts on OTA66.} - \label{fig:optical ghosts} -\end{figure} - -\begin{figure} - \centering -% \includegraphics[width=0.9\hsize,angle=0,clip]{images/glint_example_o5379g0103o.jpg} - \includegraphics[width=0.9\hsize,angle=0,clip]{images/full_fpa_glints_sm.png} - \caption{{\bf Glints:} Example of a glint on exposure o5379g0103o (2010-07-02, 45s \ips{} filter). The source star out of the field of view creates a long reflection that extends through OTA73 and OTA63.} - \label{fig:optical glints} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/o6802g0338o_SATSTAR_XY51_sm.png} - \caption{Example of saturated star, with diffraction spikes extending from the core on exposure o6802g0338o, OTA51 (2014-05-25, 45s \gps{} filter).} - \label{fig:saturated star} -\end{figure} - -\subsubsection{Masking Fraction} -\label{sec:masking_fraction} - -The GPC1 camera was designed such that where possible, OTAs with CTE -issues were placed towards the edge of the detector. Because of this, -the main analysis of the mask fraction is based not on the total -footprint of the detector, but upon a circular reference field of view -with a radius of 1.5 degrees. This radius corresponds approximately -to half the width and height of the detector. This field of view -underestimates the unvignetted region of GPC1. A second ``maximum'' -field of view is also used to estimate the mask fraction within a -larger 1.628 degree radius. This larger radius includes far larger -missing fractions due to the circular regions outside region populated -with OTAs, but does include the contribution from well-illuminated -pixels that are ignored by the reference radius. - -The results of simulating the footprint of the detector as a grid of -uniformly sized pixels of $0\farcs{}258$ size are provided in Table -\ref{tab:mask fraction}. Both fields of view contain circular -segments outside of the footprint of the detector, which increase the -area estimate that is unpopulated. This category also accounts for -the inter-OTA and inter-cell gaps. The regions with poor CTE also -contribute to a significant fraction of the masked pixels. The -remaining mask category accounts for known bad columns, cells that do -not calibrate well, and vignetting. There are also a small fraction -that have static advisory masks marked on all images. These masks -mark regions where bright columns on one cell periodically create -cross talk ghosts on other cells. - During the \IPPstage{camera} processing, a separate estimate of the mask fraction for a given exposure is calculated by counting the @@ -1117,19 +1172,4 @@ The significant advisory value is a result of applying such masks to all burntool corrected pixels. - -\begin{deluxetable}{lcc} - \tablecolumns{3} - \tablewidth{0pc} - \tablecaption{Mask Fraction by Mask Source} - \tablehead{\colhead{Mask Source}&\colhead{3 Degree FOV}&\colhead{3.25 Degree FOV}} - \startdata - Good pixel & 78.9\% & 71.1\% \\ - Unpopulated & 13.1\% & 19.6\% \\ - CTE issue & 2.3\% & 2.6\% \\ - Other issue & 5.4\% & 6.4\% \\ - Static advisory & 0.3\% & 0.3\% \\ - \enddata - \label{tab:mask fraction} -\end{deluxetable} \subsection{Background subtraction} @@ -1237,4 +1277,21 @@ model mean and standard deviation. +\begin{deluxetable}{lcc}[htpb] + \tablecolumns{3} + \tablewidth{0pc} + \tablecaption{Mask Fraction by Mask Source} + \tablehead{ + &\multicolumn{2}{c}{Field of View} \\ + \colhead{Mask Source}&\colhead{3\degree}&\colhead{3.25\degree}} + \startdata + Good pixel & 78.9\% & 71.1\% \\ + Unpopulated & 13.1\% & 19.6\% \\ + CTE issue & 2.3\% & 2.6\% \\ + Other issue & 5.4\% & 6.4\% \\ + Static advisory & 0.3\% & 0.3\% \\ + \enddata + \label{tab:mask fraction} +\end{deluxetable} + Although this background modeling process works well for most of the sky, astronomical sources that are large compared to the @@ -1271,52 +1328,5 @@ minutes. -Both of these types of persistence trails are measured and optionally -repaired via the \IPPprog{burntool} program. This program does an -initial scan of the image, and identifies objects with pixel values -higher than a conservative threshold of 30000 DN. The trail from the -peak of that object is fit with a one-dimensional power law in each -pixel column above the threshold, based on empirical evidence that -this is the functional form of this persistence effect. This fit also -matches the expectation that a constant fraction of charge is -incompletely transferred at each shift beyond the persistence -threshold. Once the fit is done, the model can be subtracted from -the image. The location of the source is stored in a table along -with the exposure PONTIME, which denotes the number of seconds since -the detector was last powered on and provides an internally -consistent time scale. - -For subsequent exposures, the table associated with the previous image -is read in, and after correcting trails from the stars on the new -image, the positions of the bright stars from the table are used to -check for remnant trails from previous exposures on the image. These -are fit and subtracted using a one-dimensional exponential model, -again based on empirical studies. The output table retains this -remnant position for 2000 seconds after the initial PONTIME recorded. -This allows fits to be attempted well beyond the nominal lifetime of -these trails. Figure \ref{fig:burntool images} shows an example of a -cell with a persistence trail from a bright star, the post-correction -result, as well as the pre and post correction versions of the same -cell on the subsequent exposure. The profiles along the detector -columns for these two exposures are presented in Figure -\ref{fig:burntool plot}. - -Using this method of correcting the persistence trails has the -challenge that it is based on fits to the raw image data, which may -have other signal sources not determined by the persistence effect. -The presence of other stars or artifacts in the detector column can -result in a poor model to be fit, resulting in either an over- or -under-subtraction of the trail. For this reason, the image mask is -marked with a value indicating that this correction has been applied. -These pixels are not fully excluded, but they are marked as suspect, -which allows them to be excluded from consideration in subsequent -stages, such as image stacking. - -The cores of very bright stars can also be deformed by this process, -as the burntool fitting subtracts flux from only one side of the star. -As most stars that result in persistence trails already have saturated -cores, they are already ignored for the purpose of PSF determination -and are flagged as saturated by the photometry reduction. - -\begin{figure} +\begin{figure}[htpb] \centering \begin{minipage}{0.45\hsize} @@ -1336,6 +1346,35 @@ \end{figure} - -\begin{figure} +Both of these types of persistence trails are measured and optionally +repaired via the \IPPprog{burntool} program. This program does an +initial scan of the image, and identifies objects with pixel values +higher than a conservative threshold of 30000 DN. The trail from the +peak of that object is fit with a one-dimensional power law in each +pixel column above the threshold, based on empirical evidence that +this is the functional form of this persistence effect. This fit also +matches the expectation that a constant fraction of charge is +incompletely transferred at each shift beyond the persistence +threshold. Once the fit is done, the model can be subtracted from +the image. The location of the source is stored in a table along +with the exposure PONTIME, which denotes the number of seconds since +the detector was last powered on and provides an internally +consistent time scale. + +For subsequent exposures, the table associated with the previous image +is read in, and after correcting trails from the stars on the new +image, the positions of the bright stars from the table are used to +check for remnant trails from previous exposures on the image. These +are fit and subtracted using a one-dimensional exponential model, +again based on empirical studies. The output table retains this +remnant position for 2000 seconds after the initial PONTIME recorded. +This allows fits to be attempted well beyond the nominal lifetime of +these trails. Figure \ref{fig:burntool images} shows an example of a +cell with a persistence trail from a bright star, the post-correction +result, as well as the pre and post correction versions of the same +cell on the subsequent exposure. The profiles along the detector +columns for these two exposures are presented in Figure +\ref{fig:burntool plot}. + +\begin{figure}[htpb] \centering \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123n4o_XY11_bt_trail.pdf} @@ -1353,4 +1392,21 @@ \label{fig:burntool plot} \end{figure} + +Using this method of correcting the persistence trails has the +challenge that it is based on fits to the raw image data, which may +have other signal sources not determined by the persistence effect. +The presence of other stars or artifacts in the detector column can +result in a poor model to be fit, resulting in either an over- or +under-subtraction of the trail. For this reason, the image mask is +marked with a value indicating that this correction has been applied. +These pixels are not fully excluded, but they are marked as suspect, +which allows them to be excluded from consideration in subsequent +stages, such as image stacking. + +The cores of very bright stars can also be deformed by this process, +as the burntool fitting subtracts flux from only one side of the star. +As most stars that result in persistence trails already have saturated +cores, they are already ignored for the purpose of PSF determination +and are flagged as saturated by the photometry reduction. \subsection{Non-linearity Correction} @@ -1402,6 +1458,27 @@ rejected. +\begin{deluxetable}{lcccc}[htpb] + \tablecolumns{3} + \tablewidth{0pc} + \tablecaption{Cells which have PATTERN.ROW correction applied} + \tablehead{\colhead{OTA} & \colhead{Cell columns} & \colhead{Additional cells}} + \startdata + OTA11 & & xy02, xy03, xy04, xy07 \\ + OTA14 & & xy23 \\ + OTA15 & 0 & \\ + OTA27 & 0, 1, 2, 3, 7 & \\ + OTA31 & 7 & \\ + OTA32 & 3, 7 & \\ + OTA45 & 3, 7 & \\ + OTA47 & 0, 3, 5, 7 & \\ + OTA57 & 0, 1, 2, 6, 7 & \\ + OTA60 & & xy55 \\ + OTA74 & 2, 7 & \\ + \enddata + \label{tab:pattern_row_cells} +\end{deluxetable} + % this figure does not really clarify anything -% \begin{figure} +% \begin{figure}[htpb] % \centering % \includegraphics[width=0.9\hsize,angle=0,clip]{images/linearity_XY27_xy16.png} @@ -1448,4 +1525,11 @@ linear ramp that exists in the sky. +\begin{figure}[htpb] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/pattern_row_edit.png} + \caption{Diagram illustrating in red which cells on GPC1 require the PATTERN.ROW correction to be applied. The footprint of each OTA is outlined, and cell xy00 is marked with either a filled box or an outline. The labeling of the non-existent corner OTAs is provided to orient the focal plane.} + \label{fig: pattern row cells} +\end{figure} + These row-by-row variations have the largest impact on data taken in the \gps{} filter, as the read noise is the dominant noise source in @@ -1477,33 +1561,5 @@ shows an example of a cell pre- and post-correction. -\begin{deluxetable}{lcccc} - \tablecolumns{3} - \tablewidth{0pc} - \tablecaption{Cells which have PATTERN.ROW correction applied} - \tablehead{\colhead{OTA} & \colhead{Cell columns} & \colhead{Additional cells}} - \startdata - OTA11 & & xy02, xy03, xy04, xy07 \\ - OTA14 & & xy23 \\ - OTA15 & 0 & \\ - OTA27 & 0, 1, 2, 3, 7 & \\ - OTA31 & 7 & \\ - OTA32 & 3, 7 & \\ - OTA45 & 3, 7 & \\ - OTA47 & 0, 3, 5, 7 & \\ - OTA57 & 0, 1, 2, 6, 7 & \\ - OTA60 & & xy55 \\ - OTA74 & 2, 7 & \\ - \enddata - \label{tab:pattern_row_cells} -\end{deluxetable} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/pattern_row_edit.png} - \caption{Diagram illustrating in red which cells on GPC1 require the PATTERN.ROW correction to be applied. The footprint of each OTA is outlined, and cell xy00 is marked with either a filled box or an outline. The labeling of the non-existent corner OTAs is provided to orient the focal plane.} - \label{fig: pattern row cells} -\end{figure} - -\begin{figure} +\begin{figure*}[htpb] \centering \begin{minipage}{0.45\hsize} @@ -1515,5 +1571,5 @@ \caption{{\bf Correlated Noise:} Example of the PATTERN.ROW correction on exposure o5379g0103o OTA57 cell xy01 (\ips{} filter 45s). The left panel shows the cell with all appropriate detrending except the PATTERN.ROW, and the right shows the same cell with PATTERN.ROW applied. The correction reduces the correlated noise on the right side, which is most distant from the read out amplifier. There is a slight over subtraction along the rows near the bright star.} \label{fig: pattern row example} -\end{figure} +\end{figure*} \subsubsection{Pattern Continuity} @@ -1593,5 +1649,5 @@ the PV3 processing. -\begin{deluxetable*}{lcccc} +\begin{deluxetable*}{lcccc}[htpb] \tablecolumns{5} \tablewidth{0pc} @@ -1616,5 +1672,5 @@ -\begin{deluxetable*}{lcccc} +\begin{deluxetable*}{lcccc}[htpb] \tablecolumns{5} \tablewidth{0pc} @@ -1633,5 +1689,5 @@ \end{deluxetable*} -\begin{deluxetable*}{lclll} +\begin{deluxetable*}{lclll}[htpb] \tablecolumns{5} \tablewidth{0pc} @@ -1682,105 +1738,5 @@ \label{sec:warping} -In order to perform image combination operations (stacking and -differences), the individual OTA images are geometrically transformed -to a set of images with a consistent and uniform relationship between -sky coordinates and image pixels. This warping operation transforms -the image pixels from the regular grid laid out on the chips in the -camera to a system of pixels with consistent geometry for a location -on the sky. - -The new image coordinate system is defined by one of a number of -``tessellations'' which specify how the sky is divided into individual -images. A single tessellation starts with a collection of projection -centers distributed across the sky. A grid of image pixels about each -projection center corresponds to sky positions via a projection with a -specified pixel scale and rotation. In general, the pixel grid within -the projection is defined as a simplified grid with the y-axis aligned -to the Declination lines and no distortion terms. The projection -centers are typically separated by several degrees on the sky; for -pixel scales appropriate to GPC1, the resulting collection of pixels -would be unwieldy in terms of memory in the processing computer. The -pixel grid is thus subdivided into smaller sub-images called -'skycells'. - -A tessellation can be defined for a limited region, with only a small -number of projection centers (e.g., for processing the M31 region), or -even a single projection center (e.g., for the Medium Deep fields). -For the $3\pi$ survey, the tessellation contains projection centers -covering the entire sky. The version used to for the PV3 analysis is -called the \ippmisc{RINGS.V3}. This tessellation consists of 2643 -projection centers spaced every four degrees in DEC, with RA spacing -of approximately four degrees, adjusted to ensure an integer number of -equal-sized regions. \ippmisc{RINGS.V3} uses a pixel scale of -$0\farcs{}25$ per pixel. The projections subdivided into a -$10\times{}10$ grid of skycells, with an overlap region of -60\arcsec\ between adjacent skycells to ensure that objects of modest -size are not split on all images. The coordinate system used for -these images matches the parity of the sky, with north in the positive -$y$ direction and east to the negative $x$ direction. - -After the detrending and photometry, the detection catalog for the -full camera is fit to the reference catalog, producing astrometric -solutions that map the detector focal plane to the sky, and map the -individual OTA pixels to the detector focal plane -\citep[see][]{magnier2017.calibration}. This solution is then used to -determine which skycells the exposure OTAs overlap. - -For each output skycell, all overlapping OTAs and the calibrated -catalog are read into the \IPPprog{pswarp} program. The output warp -image is broken into $128\times{}128$ pixel grid boxes. For purposes -of speed, each grid box has a locally linear map calculated that -converts the output warp image coordinates to the input chip image -coordinates. By doing the transformation in this direction, each -output pixel has a unique sampling position on the input image -(although it may be off the image frame and therefore not populated), -guaranteing that all output pixels are addressed, and thus preventing -gaps in the output image due to the spacing of the input pixels. - -With the locally linear grid defined, Lanczos interpolation -\citep{lanczos1956applied} with filter size parameter $a = 3$ on the -input image is used to determine the values to assign to the output -pixel location. This interpolation kernel was chosen as a compromise -between simple interpolations and higher-order Lanczos kernels, with -the goal of limiting the smear in the output image while avoiding -the high-frequency ringing generated by higher order kernels. This -process is repeated for all grid boxes, for all input images, and for -each output image product: the science image, the variance, and the -mask. The image values are scaled by the absolute value of the -Jacobian determinant of the transformation for each grid box. This -corrects the pixel values for the possible change in pixel area due to -the transformation. Similarly, the variance image is scaled by the -square of this value, again to correctly account for the pixel area -change. - -The interpolation constructs the output pixels from more than one -input pixel, which introduces covariance between pixels. For each -locally-linear grid box, the covariance matrix is calculated from the -kernel in the center of the 128 pixel range. Once the image has been -fully populated, this set of individual covariance matrices are -averaged to create the final covariance for the full image. - -An output catalog is also constructed from the full exposure input -catalog, including only those objects that fall on the new warped image. -These detections are transformed to match the new image location, and -to scale the position uncertainties based on the new orientation. - -The output image also contains header keywords SRC\_nnnn, SEC\_nnnn, -MPX\_nnnn, and MPY\_nnnn that define the mappings from the warped -pixel space to the input images. The 'nnnn' for each keyword has the -values 0000, 0001, etc., up to the number of input images. The SRC -keyword lists the input OTA name, and the SEC keyword lists the image -section that the mapping covers. The MPX and MPY contain the -back-transformation linearized across the full chip. These parameters -are stored in a string listing the reference position in the chip -coordinate frame, the slope of the relation in the warp $x$ axis, and -the slope of the relation in the warp $y$ axis. From these keywords, -any position in the warp can be mapped back to the location in any of -the input OTA images, with some reduction in accuracy. - -Examples of a warped signal, variance, and mask image are illustrated -in Figures~\ref{fig:warp image} through \ref{fig:warp mask}. - -\begin{figure} +\begin{figure}[htpb] \centering \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_sci_sm.png} @@ -1795,5 +1751,5 @@ \end{figure} -\begin{figure} +\begin{figure}[htpb] \centering \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_var_sm.png} @@ -1810,5 +1766,5 @@ \end{figure} -\begin{figure} +\begin{figure}[htpb] \centering \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_mask.png} @@ -1828,6 +1784,136 @@ \end{figure} +In order to perform image combination operations (stacking and +differences), the individual OTA images are geometrically transformed +to a set of images with a consistent and uniform relationship between +sky coordinates and image pixels. This warping operation transforms +the image pixels from the regular grid laid out on the chips in the +camera to a system of pixels with consistent geometry for a location +on the sky. + +The new image coordinate system is defined by one of a number of +``tessellations'' which specify how the sky is divided into individual +images. A single tessellation starts with a collection of projection +centers distributed across the sky. A grid of image pixels about each +projection center corresponds to sky positions via a projection with a +specified pixel scale and rotation. In general, the pixel grid within +the projection is defined as a simplified grid with the y-axis aligned +to the Declination lines and no distortion terms. The projection +centers are typically separated by several degrees on the sky; for +pixel scales appropriate to GPC1, the resulting collection of pixels +would be unwieldy in terms of memory in the processing computer. The +pixel grid is thus subdivided into smaller sub-images called +'skycells'. + +A tessellation can be defined for a limited region, with only a small +number of projection centers (e.g., for processing the M31 region), or +even a single projection center (e.g., for the Medium Deep fields). +For the $3\pi$ survey, the tessellation contains projection centers +covering the entire sky. The version used to for the PV3 analysis is +called the \ippmisc{RINGS.V3}. This tessellation consists of 2643 +projection centers spaced every four degrees in DEC, with RA spacing +of approximately four degrees, adjusted to ensure an integer number of +equal-sized regions. \ippmisc{RINGS.V3} uses a pixel scale of +$0\farcs{}25$ per pixel. The projections subdivided into a +$10\times{}10$ grid of skycells, with an overlap region of +60\arcsec\ between adjacent skycells to ensure that objects of modest +size are not split on all images. The coordinate system used for +these images matches the parity of the sky, with north in the positive +$y$ direction and east to the negative $x$ direction. + +After the detrending and photometry, the detection catalog for the +full camera is fit to the reference catalog, producing astrometric +solutions that map the detector focal plane to the sky, and map the +individual OTA pixels to the detector focal plane +(see Paper V). This solution is then used to +determine which skycells the exposure OTAs overlap. + +For each output skycell, all overlapping OTAs and the calibrated +catalog are read into the \IPPprog{pswarp} program. The output warp +image is broken into $128\times{}128$ pixel grid boxes. For purposes +of speed, each grid box has a locally linear map calculated that +converts the output warp image coordinates to the input chip image +coordinates. By doing the transformation in this direction, each +output pixel has a unique sampling position on the input image +(although it may be off the image frame and therefore not populated), +guaranteing that all output pixels are addressed, and thus preventing +gaps in the output image due to the spacing of the input pixels. + +With the locally linear grid defined, Lanczos interpolation +\citep{lanczos1956applied} with filter size parameter $a = 3$ on the +input image is used to determine the values to assign to the output +pixel location. This interpolation kernel was chosen as a compromise +between simple interpolations and higher-order Lanczos kernels, with +the goal of limiting the smear in the output image while avoiding +the high-frequency ringing generated by higher order kernels. This +process is repeated for all grid boxes, for all input images, and for +each output image product: the science image, the variance, and the +mask. The image values are scaled by the absolute value of the +Jacobian determinant of the transformation for each grid box. This +corrects the pixel values for the possible change in pixel area due to +the transformation. Similarly, the variance image is scaled by the +square of this value, again to correctly account for the pixel area +change. + +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_sci_sm.png} + \caption{Example of the stack image for skycell skycell.1146.095 + centered at ($\alpha,\delta$) = (11.934, -4.197) in the \rps{} + filter, stack\_id 3956997. This stack includes 39 input images + including o5104g0266o, the warp image in Figure \ref{fig:warp + image}, and has a combined exposure time of 1880s. Combining + such a large number of input images removes the inter-cell and + inter-chip gaps, providing a fully populated image. In addition, + the combined signal allows many more faint objects to be found + than were visible on the single frame warp image.} + + \label{fig:stack image} +\end{figure} + +The interpolation constructs the output pixels from more than one +input pixel, which introduces covariance between pixels. For each +locally-linear grid box, the covariance matrix is calculated from the +kernel in the center of the 128 pixel range. Once the image has been +fully populated, this set of individual covariance matrices are +averaged to create the final covariance for the full image. + +An output catalog is also constructed from the full exposure input +catalog, including only those objects that fall on the new warped image. +These detections are transformed to match the new image location, and +to scale the position uncertainties based on the new orientation. + +The output image also contains header keywords SRC\_nnnn, SEC\_nnnn, +MPX\_nnnn, and MPY\_nnnn that define the mappings from the warped +pixel space to the input images. The 'nnnn' for each keyword has the +values 0000, 0001, etc., up to the number of input images. The SRC +keyword lists the input OTA name, and the SEC keyword lists the image +section that the mapping covers. The MPX and MPY contain the +back-transformation linearized across the full chip. These parameters +are stored in a string listing the reference position in the chip +coordinate frame, the slope of the relation in the warp $x$ axis, and +the slope of the relation in the warp $y$ axis. From these keywords, +any position in the warp can be mapped back to the location in any of +the input OTA images, with some reduction in accuracy. + +Examples of a warped signal, variance, and mask image are illustrated +in Figures~\ref{fig:warp image} through \ref{fig:warp mask}. + \section{Stacking} \label{sec:stacking} + +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_var_sm.png} + \caption{Example of the stack variance image for skycell + skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) + in the \rps{} filter, stack\_id 3956997. The variance + map for this stack is reasonably smooth, with the mottled pattern + from the inter-chip and inter-cell gaps printing through. Some + regions with higher variance are found where the number of inputs + is lower.} + + \label{fig:stack wt image} +\end{figure} Once individual exposures have been warped onto a common projection @@ -1857,4 +1943,17 @@ and image components are loaded into the \IPPprog{ppStack} program to prepare the inputs and stack the frames. + +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_mask.png} + \caption{Example of the stack mask image for skycell + skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) + in the \rps{} filter, stack\_id 3956997. The entire frame is + largely unmasked after combining inputs, with the only remaining + masks falling on the cores of bright stars, and in small regions + around the brightest objects where the overlapping of diffraction + spike masks have removed all inputs.} + \label{fig:stack mask image} +\end{figure} Once all files are ingested, the first step is to measure the size and @@ -1893,4 +1992,17 @@ included in the zeropoint and transparency values. +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_num_sm.png} + \caption{Example of the stack number image for skycell + skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) + in the \rps{} filter, stack\_id 3956997. This map shows + the number of inputs contributing to each pixel of the output + stack. Again, the pattern of the inter-chip and inter-cell gaps + is visible, along with other mask features. } + + \label{fig:stack num image} +\end{figure} + The zeropoint calibration performed here uses the calibration of the individual input exposures against the reference catalog. Upon the @@ -1900,5 +2012,5 @@ the entire region of the sky imaged. This further calibration is not available at the time of stacking, and so there may be small residuals -in the transparency values as a result of this \citep{magnier2017.calibration}. +in the transparency values as a result of this (Paper V). With the flux normalization factors and target PSF chosen, the @@ -1927,4 +2039,16 @@ the square of it, scaling all inputs to the common zeropoint. +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_exp_sm.png} + \caption{Example of the stack exposure time image for skycell + skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) + in the \rps{} filter, stack\_id 3956997. Since the input + exposures had exposures times of 40 and 60 seconds, the pattern + observed here similar to, but subtly different from the number + map.} + \label{fig:stack exp image} +\end{figure} + Once the convolution kernels are defined for each image, they are used to convolve the image to match the target PSF. Any input image that @@ -1971,4 +2095,17 @@ The output mask value is taken to be zero (no masked bits), unless there were no valid inputs, in which case the BLANK mask bit is set. + +\begin{figure}[t] + \centering + \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_expwt_sm.png} + \caption{Example of the stack weighted exposure image for skycell + skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) + in the \rps{} filter, stack\_id 3956997. This map shows + the weighted average exposure time, as described in the text. It + is similar to the simple exposure time map, but shows how some + input exposures have their contributions weighted down due to the + observed larger image variances.} + \label{fig:stack exp wtimage} +\end{figure} Due to uncorrected artifacts that can occur on GPC1, and the fact that @@ -2113,85 +2250,4 @@ such that: $L = \mathrm{BOFFSET} + \mathrm{BSOFTEN} \cdot \left(\exp(C / \alpha) - \exp(-C / \alpha)\right)$. - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_sci_sm.png} - \caption{Example of the stack image for skycell skycell.1146.095 - centered at ($\alpha,\delta$) = (11.934, -4.197) in the \rps{} - filter, stack\_id 3956997. This stack includes 39 input images - including o5104g0266o, the warp image in Figure \ref{fig:warp - image}, and has a combined exposure time of 1880s. Combining - such a large number of input images removes the inter-cell and - inter-chip gaps, providing a fully populated image. In addition, - the combined signal allows many more faint objects to be found - than were visible on the single frame warp image.} - - \label{fig:stack image} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_mask.png} - \caption{Example of the stack mask image for skycell - skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) - in the \rps{} filter, stack\_id 3956997. The entire frame is - largely unmasked after combining inputs, with the only remaining - masks falling on the cores of bright stars, and in small regions - around the brightest objects where the overlapping of diffraction - spike masks have removed all inputs.} - \label{fig:stack mask image} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_var_sm.png} - \caption{Example of the stack variance image for skycell - skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) - in the \rps{} filter, stack\_id 3956997. The variance - map for this stack is reasonably smooth, with the mottled pattern - from the inter-chip and inter-cell gaps printing through. Some - regions with higher variance are found where the number of inputs - is lower.} - - \label{fig:stack wt image} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_num_sm.png} - \caption{Example of the stack number image for skycell - skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) - in the \rps{} filter, stack\_id 3956997. This map shows - the number of inputs contributing to each pixel of the output - stack. Again, the pattern of the inter-chip and inter-cell gaps - is visible, along with other mask features. } - - \label{fig:stack num image} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_exp_sm.png} - \caption{Example of the stack exposure time image for skycell - skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) - in the \rps{} filter, stack\_id 3956997. Since the input - exposures had exposures times of 40 and 60 seconds, the pattern - observed here similar to, but subtly different from the number - map.} - \label{fig:stack exp image} -\end{figure} - -\begin{figure} - \centering - \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_expwt_sm.png} - \caption{Example of the stack weighted exposure image for skycell - skycell.1146.095 centered at ($\alpha,\delta$) = (11.934, -4.197) - in the \rps{} filter, stack\_id 3956997. This map shows - the weighted average exposure time, as described in the text. It - is similar to the simple exposure time map, but shows how some - input exposures have their contributions weighted down due to the - observed larger image variances.} - \label{fig:stack exp wtimage} -\end{figure} \section{Difference Images} @@ -2250,5 +2306,5 @@ pointings are as close to identical as possible. The observing strategy to enable this is discussed in more detail in -\citet{chambers2017}. +Paper I.