Index: trunk/doc/release.2015/ps1.detrend/detrend.tex =================================================================== --- trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 40051) +++ trunk/doc/release.2015/ps1.detrend/detrend.tex (revision 40052) @@ -11,7 +11,9 @@ \RequirePackage{color} +\RequirePackage{code} \input{astro.sty} -%\usepackage{subcaption} %\usepackage{natbib} + +\usepackage[T1]{fontenc}% (2) specify encoding % online version may use color, but print version needs b/w @@ -198,33 +200,33 @@ the metadata of exposure parameters. For the PV3 processing, large contiguous regions were defined, and the images for all exposures -within that region launched for the \ippstage{chip} stage processing. +within that region launched for the \IPPstage{chip} stage processing. This stage performs the image detrending (described below in section \ref{sec:detrending}), as well as the single epoch photometry \citep{magnier2017b}, in parallel on the individual OTA device data. -Following the \ippstage{chip} stage is the \ippstage{camera} stage, in +Following the \IPPstage{chip} stage is the \IPPstage{camera} stage, in which the astrometry and photometry for the entire exposure is calibrated by matching the detections against the reference catalog. This stage also performs masking updates based on the now-known positions and brightnesses of stars that create dynamic features (see -Section \ref{sec:dynamic_masks} below). The \ippstage{warp} stage is +Section \ref{sec:dynamic_masks} below). The \IPPstage{warp} stage is the next to operate on the data, transforming the detector oriented -\ippstage{chip} stage images onto common sky oriented images that have +\IPPstage{chip} stage images onto common sky oriented images that have fixed sky projections (Section \ref{sec:warping}). When all -\ippstage{warp} stage processing is done for the region of the sky, -\ippstage{stack} processing is performed (Section \ref{sec:stacking}) +\IPPstage{warp} stage processing is done for the region of the sky, +\IPPstage{stack} processing is performed (Section \ref{sec:stacking}) to construct deeper, fully populated images from the set of -\ippstage{warp} images that cover that region of the sky. Beyond the -\ippstage{stack} stage, a series of additional stages are done that +\IPPstage{warp} images that cover that region of the sky. Beyond the +\IPPstage{stack} stage, a series of additional stages are done that are more fully described in other papers. Transient features are -identified in the \ippstage{diff} stage, which takes input -\ippstage{warp} and/or \ippstage{stack} data and performs image +identified in the \IPPstage{diff} stage, which takes input +\IPPstage{warp} and/or \IPPstage{stack} data and performs image differencing (Section \ref{sec:diffs}). Further photometry is -performed in the \ippstage{staticsky} and \ippstage{skycal} stages, +performed in the \IPPstage{staticsky} and \IPPstage{skycal} stages, which add extended source fitting to the point source photometry of -objects detected in the \ippstage{stack} images, and calibrate the -results against the reference catalog. The \ippstage{fullforce} stage -takes the catalog output of the \ippstage{skycal} stage, and uses the +objects detected in the \IPPstage{stack} images, and calibrate the +results against the reference catalog. The \IPPstage{fullforce} stage +takes the catalog output of the \IPPstage{skycal} stage, and uses the objects detected in that to perform forced photometry on the -individual \ippstage{warp} stage images. The details of these stages +individual \IPPstage{warp} stage images. The details of these stages are provided in \citet{magnier2017b}. @@ -234,7 +236,7 @@ the summit to the main IPP processing cluster at the Maui Research and Technology Center in Kihei, and registered into the processing -database. This triggers a new \ippstage{chip} stage reduction for +database. This triggers a new \IPPstage{chip} stage reduction for science exposures, advancing processing upon completion through to the -\ippstage{diff} stage. This allows the ongoing solar system moving +\IPPstage{diff} stage. This allows the ongoing solar system moving object search to identify candidates for follow up observations within 24 hours of the initial set of observations \citep{2015IAUGA..2251124W}. @@ -244,6 +246,6 @@ details of the construction of those detrends in Section \ref{sec:detrend construction}. An analysis of the algorithms used to -complete the \ippstage{warp} (section \ref{sec:warping}), -\ippstage{stack} (section \ref{sec:stacking}), and \ippstage{diff} +complete the \IPPstage{warp} (section \ref{sec:warping}), +\IPPstage{stack} (section \ref{sec:stacking}), and \IPPstage{diff} (section \ref{sec:diffs}) stage transformations of the image data to from the detector frame to a common sky frame, and the co-adding of @@ -297,5 +299,5 @@ case with GPC1, and this requires an additional set of detrending steps to remove these non-linear responses. The first of these is the -\ippprog{burntool} correction, which removes the persistence trails +\IPPprog{burntool} correction, which removes the persistence trails caused by the incomplete transfer of charge along the readout columns. This bright-end nonlinearity is generally only evident for the @@ -321,5 +323,5 @@ For the PV3 processing, all detrending is done by the -\ippprog{ppImage} program. This program applies the detrends to the +\IPPprog{ppImage} program. This program applies the detrends to the individual cells, and then an OTA level mosaic is constructed for the science image, the mask image, and the variance map image. The single @@ -354,5 +356,5 @@ Both of these types of persistence trails are measured and optionally -repaired via the \ippprog{burntool} program. This program does an +repaired via the \IPPprog{burntool} program. This program does an initial scan of the images, and identifies objects with pixel values brighter than a conservative threshold of 30000 DN. The trail from @@ -716,5 +718,5 @@ models for each individual dark model. The additional pixel-to-pixel variance from this noisemap is added to the Poissonian variance to -form the science variance image generated by the \ippstage{chip} +form the science variance image generated by the \IPPstage{chip} processing. @@ -1064,5 +1066,5 @@ The remaining dynamic masks are not generated until the IPP -\ippstage{camera} stage, at which point all object photometry is +\IPPstage{camera} stage, at which point all object photometry is complete, and an astrometric solution is known for the exposure. This added information provides the positions of bright sources based on @@ -1261,32 +1263,44 @@ \label{sec:masking_fraction} -For the full field of view that falls on the sixty OTAs, 14.7\% of all -pixels are masked. The large fraction of this masking is due to -regions that fall within the vignetted region. Defining the diameter -of the unvignetted region to have be 3 degrees, and excluding pixels -that fall beyond this point reduces the static masking fraction to -9.7\%. - -Unfortunately, due to the design of the OTAs and readout cells, a -non-negligible fraction of the field of view falls onto an area that -does not have a detector pixel. For a given OTA mosaicked to a -$4846\times{}4868$ pixel image, the 64 $590\times{}598$ pixel readout -cells cover 95.7\% of the OTA area, providing an additional 4.3\% -masking in the unvignetted field of view due to the absence of a -detector pixel. - -For the inter-chip gap area loss, we use two field of view -calculations to estimate the masking fraction. The reference field of -view of GPC1 is 3 degrees, which at the nominal plate scale of 0.258 -arcseconds per pixel, translates to a 20930 FPA pixel radius. Summing -mask fractions from these three contributions within the unvignetted -field of view results in an average of $\sim 20\%$ masking fraction -across the field of view. Dynamic masking adds an additional $2-3\%$ -on average, with advisory burntool masking contributing the largest -single component. Table \ref{tab:mask fraction} contains estimates of -the mask fraction in the GPC1 detector footprint by the sources of the -masking for the 3 degree field of view, as well as for a larger 3.25 -degree field of view that allows addition unvignetted regions in the -corners to contribute. +Although there are a large number of masked pixels within the sixty +OTAs of GPC1, the camera was designed to move chips with problematic +areas (most notably CTE issues) to the edges of the detector. Because +of this, the main analysis of the mask fraction is based not on the +total footprint of the detector, but upon a circular reference field +of view with a radius of 1.5 degrees. This field of view corresponds +approximately to half the width and height of the detector. This +field of view underestimates the unvignetted region of GPC1. A second +``maximum'' field of view is also used to estimate the mask fraction +within a larger 1.628 degree radius. This larger radius includes far +larger missing fractions due to the circular regions outside region +populated with OTAs, but does include the contribution from +well-illuminated pixels that are ignored by the reference radius. + +The results of simulating simulating the footprint of the detector as +a grid of uniformly sized pixels of $0\farcs{}258$ size are provided +in Table \ref{tab:mask fraction}. Both fields of view contain +circular segments outside of the footprint of the detector, which +increase the area estimate that is unpopulated. This category also +accounts for the inter-OTA and inter-cell gaps. The regions with poor +CTE also account for a significant fraction of the masked pixels. The +remaining mask category accounts for known bad columns, cells that do +not calibrate well, and vignetting. There are also a small fraction +that have static advisory masks marked on all images. These masks +mark regions where bright columns on one cell periodically create +cross talk ghosts on other cells. + +During the \IPPstage{camera} processing, a separate estimate of the +mask fraction for a given exposure is calculated by counting the +fraction of pixels with static, dynamic, and advisory mask bits set +within the two field of view radii. The static mask fraction is then +augmented by an estimate of the unpopulated inter-chip gaps (as the +input masks already account for the inter-cell gaps). This estimate +does not include the circular segments outside of the detector +footprint. This is minor for the reference field of view (1\% +difference), but does underestimate the static mask fraction for the +maximum radius by 7.3\%. This analysis does provide a the observed +dynamic and advisory mask fractions, which are 0.03\% and 3\% +respectively. The significant advisory value is a result of applying +such masks to all burntool corrected pixels. \begin{deluxetable}{lcc} @@ -1296,7 +1310,9 @@ \tablehead{\colhead{Mask Source}&\colhead{3 Degree FOV}&\colhead{3.25 Degree FOV}} \startdata - No pixel & 4.44\% & 9.47\% \\ - Detector defect & 6.37\% & 7.91\% \\ - CTE issue & 2.62\% & 3.13\% \\ + Good pixel & 78.9\% & 71.1\% \\ + Unpopulated & 13.1\% & 19.6\% \\ + CTE issue & 2.3\% & 2.6\% \\ + Other issue & 5.4\% & 6.4\% \\ + Static advisory & 0.3\% & 0.3\% \\ \enddata \label{tab:mask fraction} @@ -1413,5 +1429,5 @@ can have their own features modeled as being part of the background. For the specialized processing of M31, which covers an entire pointing -of GPC1, the measured background was added back to the \ippstage{chip} +of GPC1, the measured background was added back to the \IPPstage{chip} stage images, but this special processing was not used for the large scale $3\Pi$ PV3 reduction. @@ -1422,6 +1438,6 @@ The various detrends for GPC1 are constructed in similar ways. A series of appropriate exposures is selected from the database, and -processed with the \ippprog{ppImage} program. This program is used -for the \ippstage{chip} stage processing as well, and is designed to +processed with the \IPPprog{ppImage} program. This program is used +for the \IPPstage{chip} stage processing as well, and is designed to do multiple image processing operations. The extent of this processing is dependent on the order in which the detrend to be @@ -1431,5 +1447,5 @@ for the detrends we construct. -Once the input data has been prepared, the \ippprog{ppMerge} program +Once the input data has been prepared, the \IPPprog{ppMerge} program is used to construct some sort of ``average'' of the inputs. This step need not be a mathematical average, but is used to combine the @@ -1557,5 +1573,5 @@ For each output skycell, all overlapping OTAs and the calibrated -catalog are read into the \ippprog{pswarp} program. Each input image +catalog are read into the \IPPprog{pswarp} program. Each input image is examined in order, and the same transformation performed. This transformation breaks the output warp image into $128\times{}128$ @@ -1666,5 +1682,5 @@ The stacked image is comprised of all warp frames for a given skycell in a single filter. The source catalogs and image components are -loaded into the \ippprog{ppStack} program to prepare the inputs and +loaded into the \IPPprog{ppStack} program to prepare the inputs and stack the frames.