Index: /trunk/doc/modules/CameraImages.tex =================================================================== --- /trunk/doc/modules/CameraImages.tex (revision 5366) +++ /trunk/doc/modules/CameraImages.tex (revision 5367) @@ -863,9 +863,9 @@ entry must contain the following user-defined parameters: \begin{itemize} -\item GRID_OFFSET : maximum allowed displacement in search -\item GRID_SCALE : grid bin size in focal-plane coordinate units -\item MIN_ANGLE : minimum tested relative rotation -\item MAX_ANGLE : maximum tested relative rotation -\item DEL_ANGLE : relative rotation step size +\item \code{GRID_OFFSET} : maximum allowed displacement in search +\item \code{GRID_SCALE} : grid bin size in focal-plane coordinate units +\item \code{MIN_ANGLE} : minimum tested relative rotation +\item \code{MAX_ANGLE} : maximum tested relative rotation +\item \code{DEL_ANGLE} : relative rotation step size \end{itemize} The output from this process is the statistics of the best-fit @@ -930,5 +930,5 @@ following function performs this operation: \begin{prototype} -bool pmAstromMatchedListFit (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config); +bool pmAstromFitFPA (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config); \end{prototype} This function accepts the raw and reference source lists and the list @@ -949,4 +949,24 @@ linear terms and the higher-order terms of the polynomial fits. +An alternative to fitting the rotation of the FPA relative to the +Tangent Plane is to treat the fitted transformation as a measurement +of the chip within the FPA. The following function performs this +operation in the same way as \code{pmAstromFitFPA}: +\begin{prototype} +bool pmAstromFitChip (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config); +\end{prototype} +This function accepts the raw and reference source lists for a single +chip and the list of matched entries. It uses the matched list to +determine a polynomial transformation between the two coordinate +systems. The fitting uses clipping to exclude outliers, likely +representing poor matches. The \code{config} element must contain the +information \code{ASTROM_NSIGMA} (specifying the number of sigma used +in the clipping) and \code{ASTROM_NCLIP} (specifying the number of +clipping iterations must be performed). The \code{config} element +must also specify the order of the polynomial fit (keyword: +\code{ASTROM_ORDER}). The result of this fit is a set of +modifications of the components of the \code{pmChip.toFPA} +transformation. + A mosaic represents a particular set of challenges when determining an astrometric solution. There is substantial degeneracy between the Index: /trunk/doc/modules/ChangeLogSDRS.tex =================================================================== --- /trunk/doc/modules/ChangeLogSDRS.tex (revision 5366) +++ /trunk/doc/modules/ChangeLogSDRS.tex (revision 5367) @@ -1,3 +1,3 @@ -%%% $Id: ChangeLogSDRS.tex,v 1.33 2005-09-20 22:01:33 jhoblitt Exp $ +%%% $Id: ChangeLogSDRS.tex,v 1.34 2005-10-18 20:15:05 eugene Exp $ \subsection{Changes from version 00 (16 August 2004) to version 01 (12 October 2004)} @@ -142,3 +142,14 @@ \begin{itemize} \item fix enum syntax + \item added Astrometry Fitting Support (matching / fitting routines) + \item added \code{pmAstromRadiusMatch} + \item added \code{pmAstromGridMatch} + \item added \code{pmAstromApplyGridMatch} + \item added \code{pmAstromMeasureGradients} + \item added \code{pmAstromFitFPA} + \item added \code{pmAstromFitChip} + \item added \code{pmAstromFitDistortion} + \item renamed section 7 to Detrend Creation, moved \code{pmReadoutCombine} to a subsection + \item added \code{pmFringeStats} (Fringe Amplitude subsection) + \item added \code{pmFlatNormalization} (Flat-field renorm section) \end{itemize} Index: /trunk/doc/modules/ModulesSDRS.tex =================================================================== --- /trunk/doc/modules/ModulesSDRS.tex (revision 5366) +++ /trunk/doc/modules/ModulesSDRS.tex (revision 5367) @@ -1,3 +1,3 @@ -%%% $Id: ModulesSDRS.tex,v 1.61 2005-10-18 14:32:25 eugene Exp $ +%%% $Id: ModulesSDRS.tex,v 1.62 2005-10-18 20:15:05 eugene Exp $ \documentclass[panstarrs,spec]{panstarrs} @@ -1205,5 +1205,155 @@ \end{itemize} -\subsection{Fringes} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\section{Detrend Creation} + +In the detrend creation process, a collection of raw images are +combined to produce a clean, high-quality master image for correcting +the effect of interest. The input images may potentially be processed +and scaled in some way. The resulting output images may be to be +re-scaled to have a consistent signal for all chips in the mosaic. +The simplest example is the construction of a bias image, in the case +where there is signficant 2-D bias structure. In this case, the input +raw bias images are probably combined without any additional +processing. In another example, flat-field image must be +bias-corrected and scaled to a consistent normalization before being +combined, and the flat-field images from the different chips must be +normalized so that each chip will be flattened consistently across the +mosaic. A complex example is the fringe pattern, in which the input +images must be bias-corrected and flattened, and the resulting images +must be scaled by the amplitude of the fringe pattern on each image, +rather than by the average flux level. In this section, we define the +tools necessary to perform the detrend creation process. + +\subsection{Image Stacking} + +A basic operation in generating the master detrend images is using a +stack of many input images of a particular type and combining them, +with perhaps some additional scaling, in order to build up +signal-to-noise and to reject deviant pixel. For this, we require a +general purpose image combination module. We forsee this module as +only acting upon data from the same detector, and so each input image +will have the same noise characteristics. + +\begin{datatype} +typedef struct { + psStats *stats; // Statistics to use in combining pixels + unsigned int maskVal, // Mask pixels where mask & maskVal == 1 + float fracHigh; // Fraction of high pixels to throw + float fracLow; // Fraction of low pixels to throw + int nKeep; // Number of pixels to be sure to keep +} pmCombineParams; +\end{datatype} + +\begin{prototype} +psImage * +pmReadoutCombine(psImage *output, // Output image, or NULL + const psList *inputs, // List of input readouts + pmCombineParams *params, // Combination parameters + const psVector *zero, // Offsets to apply for each image + const psVector *scale, // Scales to apply for each image + bool applyZeroScale, // Are zero and scale for application, or only noise properties? + float gain, // Gain in e/ADU + float readnoise // Read noise in e + ); +\end{prototype} + +\code{pmReadoutCombine} combines input images pixel by pixel --- for +each pixel of the output image, a stack of contributing input pixels +is formed and combined. Several of its input parameters are lists or +vectors, and if these are not all of the same length (or \code{NULL}), +the module shall generate an error and return \code{NULL}. + +If the provided \code{output} is \code{NULL}, then the module shall +allocate a new image of sufficient size for the input images. If the +\code{output} image is non-\code{NULL} and is not of sufficient size +for the combined image, the module shall generate an error and return +\code{NULL}. + +If the \code{inputs} is \code{NULL}, the module shall generate an +error and return \code{NULL}. Otherwise, the \code{inputs} shall be a +list of \code{pmReadout}s. The images contained within the +\code{pmReadout}s need not all be of the same size, but the module +shall take into account the offsets (\code{col0,row0}) from the corner +of the detector when comparing pixels, so that it is the same +\textit{physical} pixels that are combined. + +The parameters used in the combination, including how the pixels are +to be combined, and how the rejection is performed is contained within +the \code{params}, which may not be \code{NULL} (otherwise the module +shall generate an error and return \code{NULL}). We choose to use +this structure instead of supplying the values separately in order to +keep down the number of parameters to \code{pmReadoutCombine}; the +\code{pmCombineParams} may be recycled for subsequent calls to +\code{pmReadoutCombine} since the values are not dependent upon the +choice of inputs, but merely specify how the combination is to be +performed. + +The particular statistic specified by \code{stats} shall be used to +combine each stack of pixels from the input images. Only one of the +statistics choices may be specified, otherwise the module shall +generate an error and return \code{NULL}. + +If the \code{maskVal} is non-zero, then pixels in the \code{mask} of +each \code{pmReadout} in the \code{inputs} which satisfy the +\code{maskVal} shall not have the corresponding pixels placed in the +stack for combination. + +After masking, but before performing the combination, the highest +\code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels +in the stack are immediately rejected, unless this would leave less +than \code{nKeep} pixels in the stack, in which case no immediate +rejection is performed. + +If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale} +is \code{true}, then the appropriate values shall be added to the +\code{inputs} before rejection is performed. If \code{zero} is +non-\code{NULL} and \code{applyZeroScale} is false, then the values +shall only be used in calculating the Poisson variances. + +If the \code{scale} vector is non-\code{NULL} and +\code{applyZeroScale} is \code{true}, then the appropriate values +shall multiply the \code{inputs} before rejection is performed. If +\code{scale} is non-\code{NULL} and \code{applyZeroScale} is false, +then the values shall only be used in calculating the Poisson +variances. + +The purpose of \code{applyZeroScale} is to allow combination of fringe +frames, where the frames have been deliberately sky-subtracted and +rescaled (to get the fringes amplitudes running from -1 to 1), which +actions should not be undone when combining, but yet it is desirable +to provide the \code{zero} and \code{scale} values so that the correct +noise properties are used in the combination. + +If the \code{gain} and \code{readnoise} are positive and non-negative +(respectively), then these shall be used to provide weights for the +combination using Poisson statistics ($\sigma_i$ below). + +In summary, pixels corresponding to the same physical pixel are +combined, having values $x_i \pm \sigma_i$. In the case that +\code{applyZeroScale} is \code{true}, then: +\begin{eqnarray} +x_i & = & s_i f_i + z_i \\ +\sigma_i & = & [g x_i + r^2]^{1/2} / g +\end{eqnarray} +Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale +applied to image $i$, $z_i$ is the zero offset applied to image $i$, +$g$ is the gain, and $r$ is the read noise. If scales are not +provided, they are set to unity; if zero offsets are not provided, +they are set to zero. + +If \code{applyZeroScale} is \code{false}, then the values are: +\begin{eqnarray} +x_i & = & f_i \\ +\sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g +\end{eqnarray} +where the same symbols are used as above. + +The \code{inputs, zero} and \code{scale} may be of U16, S32 and F32 +types, and must all be of the same type. The \code{output} shall be +of the same type. + +\subsection{Fringe Amplitude} Some images contain a signal caused by thin-film interference in the @@ -1225,11 +1375,14 @@ A simple approach to the fringe pattern is to subtract a master fringe -frame scaled by the amplitude of the fringe pattern. We thus need a -method of measuring the fringe amplitude which is robust in the -presence of stars and which is fast. We implement a method used at -CFHT in which the fringe pattern is mapped by a series of points pairs -which correspond to peaks and valleys of the fringe pattern. We -define the following function to measure the global fringe amplitude -of an image given a collection of fringe point pairs. +frame scaled by the amplitude of the fringe pattern. The amplitude of +the fringe pattern is used both in the process of constructing the +master image and in scaling the master image when it is applied to +science image. We thus need a method of measuring the fringe +amplitude which is robust in the presence of stars and which is fast. +We implement a method used at CFHT in which the fringe pattern is +mapped by a series of points pairs which correspond to peaks and +valleys of the fringe pattern. We define the following function to +measure the global fringe amplitude of an image given a collection of +fringe point pairs. \begin{prototype} stats *pmFringeStats (psArray *fringePoints, psImage *image, psMetadata *config); @@ -1256,132 +1409,96 @@ \end{datatype} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\section{Calibration} - -The calibration module essentially consists of combining multiple -images of a particular type in order to build up signal-to-noise. For -this, we require a general purpose image combination module. We -forsee this module as only acting upon data from the same detector, -and so each input image will have the same noise characteristics. - -\begin{datatype} -typedef struct { - psStats *stats; // Statistics to use in combining pixels - unsigned int maskVal, // Mask pixels where mask & maskVal == 1 - float fracHigh; // Fraction of high pixels to throw - float fracLow; // Fraction of low pixels to throw - int nKeep; // Number of pixels to be sure to keep -} pmCombineParams; -\end{datatype} - -\begin{prototype} -psImage * -pmReadoutCombine(psImage *output, // Output image, or NULL - const psList *inputs, // List of input readouts - pmCombineParams *params, // Combination parameters - const psVector *zero, // Offsets to apply for each image - const psVector *scale, // Scales to apply for each image - bool applyZeroScale, // Are zero and scale for application, or only noise properties? - float gain, // Gain in e/ADU - float readnoise // Read noise in e - ); -\end{prototype} - -\code{pmReadoutCombine} combines input images pixel by pixel --- for -each pixel of the output image, a stack of contributing input pixels -is formed and combined. Several of its input parameters are lists or -vectors, and if these are not all of the same length (or \code{NULL}), -the module shall generate an error and return \code{NULL}. - -If the provided \code{output} is \code{NULL}, then the module shall -allocate a new image of sufficient size for the input images. If the -\code{output} image is non-\code{NULL} and is not of sufficient size -for the combined image, the module shall generate an error and return -\code{NULL}. - -If the \code{inputs} is \code{NULL}, the module shall generate an -error and return \code{NULL}. Otherwise, the \code{inputs} shall be a -list of \code{pmReadout}s. The images contained within the -\code{pmReadout}s need not all be of the same size, but the module -shall take into account the offsets (\code{col0,row0}) from the corner -of the detector when comparing pixels, so that it is the same -\textit{physical} pixels that are combined. - -The parameters used in the combination, including how the pixels are -to be combined, and how the rejection is performed is contained within -the \code{params}, which may not be \code{NULL} (otherwise the module -shall generate an error and return \code{NULL}). We choose to use -this structure instead of supplying the values separately in order to -keep down the number of parameters to \code{pmReadoutCombine}; the -\code{pmCombineParams} may be recycled for subsequent calls to -\code{pmReadoutCombine} since the values are not dependent upon the -choice of inputs, but merely specify how the combination is to be -performed. - -The particular statistic specified by \code{stats} shall be used to -combine each stack of pixels from the input images. Only one of the -statistics choices may be specified, otherwise the module shall -generate an error and return \code{NULL}. - -If the \code{maskVal} is non-zero, then pixels in the \code{mask} of -each \code{pmReadout} in the \code{inputs} which satisfy the -\code{maskVal} shall not have the corresponding pixels placed in the -stack for combination. - -After masking, but before performing the combination, the highest -\code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels -in the stack are immediately rejected, unless this would leave less -than \code{nKeep} pixels in the stack, in which case no immediate -rejection is performed. - -If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale} -is \code{true}, then the appropriate values shall be added to the -\code{inputs} before rejection is performed. If \code{zero} is -non-\code{NULL} and \code{applyZeroScale} is false, then the values -shall only be used in calculating the Poisson variances. - -If the \code{scale} vector is non-\code{NULL} and -\code{applyZeroScale} is \code{true}, then the appropriate values -shall multiply the \code{inputs} before rejection is performed. If -\code{scale} is non-\code{NULL} and \code{applyZeroScale} is false, -then the values shall only be used in calculating the Poisson -variances. - -The purpose of \code{applyZeroScale} is to allow combination of fringe -frames, where the frames have been deliberately sky-subtracted and -rescaled (to get the fringes amplitudes running from -1 to 1), which -actions should not be undone when combining, but yet it is desirable -to provide the \code{zero} and \code{scale} values so that the correct -noise properties are used in the combination. - -If the \code{gain} and \code{readnoise} are positive and non-negative -(respectively), then these shall be used to provide weights for the -combination using Poisson statistics ($\sigma_i$ below). - -In summary, pixels corresponding to the same physical pixel are -combined, having values $x_i \pm \sigma_i$. In the case that -\code{applyZeroScale} is \code{true}, then: -\begin{eqnarray} -x_i & = & s_i f_i + z_i \\ -\sigma_i & = & [g x_i + r^2]^{1/2} / g -\end{eqnarray} -Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale -applied to image $i$, $z_i$ is the zero offset applied to image $i$, -$g$ is the gain, and $r$ is the read noise. If scales are not -provided, they are set to unity; if zero offsets are not provided, -they are set to zero. - -If \code{applyZeroScale} is \code{false}, then the values are: -\begin{eqnarray} -x_i & = & f_i \\ -\sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g -\end{eqnarray} -where the same symbols are used as above. - -The \code{inputs, zero} and \code{scale} may be of U16, S32 and F32 -types, and must all be of the same type. The \code{output} shall be -of the same type. +\subsection{Flat-field Re-Normalization} + +Consider a collection of $N_i$ flat-field images obtained with a +mosaic camera consisting of $N_j$ chips. Each image is exposed to an +illumination source which should be a uniform surface +brightness\footnote{This is likely a false assumption: the +illumination source likely has spatial variations. However, for the +purposes of this discussion, it only matters that such spatial +variations scale consistently as a function of illumination intensity. +The spatial errors are corrected by the photometric flat-field +correction technique (eg., Magnier \& Cuillandre 2004).} Two factors +determine the actual measured flux level (in Digital Numbers) on each +of the chips in each image: the gain of each chip ($\mbox{gain}_j$) +and the flux level from the illumination source ($\mbox{source}_i$). +When the images are combined, the input images must be scaled so that +the flux levels can be consistently compared. After combining the +collection of images, it is necessary to determine an appropriate +re-normalization for the resulting flat-field images. In effect, the +individual chips must be adjusted so that the master flat-field image +has a flux level which varies from chip to chip in proportion to the +actual chip gain. In this case, if a uniform illumination source +illuminates the mosaic, the resulting flux levels will be corrected by +the flat-field to a single, consistent flux level. + +In order to determine the correct relative scaling between the +devices, it is thus necessary to know the individual chip gains, or at +least the gain ratios. A typical technique scaled all chips relative +to a reference chip, or by a statistic measured for the complete +collection. These techniques fail if the input collection of images +does not always consist of the same set of chips; for the GPC on +Pan-STARRS, we must expect that individual cells or even chips may be +disabled on a frequent basis, so our algorithms must not be limited by +the assumption that all chips are available in all images. We +therefore define the following algorithm to measure the relative chip +gains for a collection of input flat-field images, each with a +measured flux $\mbox{flux}_{i,j}$. We want to solve for the chip +gains and the source illumination fluxes which would make the best +prediction of the measured input image fluxes: +\[ +\mbox{flux}^{\rm pred}_{i,j} = \mbox{gain}_j \times \mbox{source}_i +\] +This relationship is easiest to determine if we take the logarithm of +both sides of the equation: +\[ +M^{\rm pred}_{i,j} = G_j + S_i +\] +where $M^{\rm pred}_{i,j} = \log \mbox{flux}^{\rm pred}_{i,j}$, $G_j = +\log \mbox{gain}_j$, and $\log \mbox{source}_i = S_i$. We can then +write the chi-square which we want to minimize as: +\[ +\chi^2 = \sum_{i,j} (M^{\rm obs}_{i,j} - G_j - S_i)^2 +\] +where we ignore the weights of the different measured flux levels. +Taking the derivatives with respect to the parameters of interest +($G_j, S_i$), and setting them to 0, we determine the following set of +equations which must be solved: +\[ +G_j \times N_i = \sum_i M^{\rm obs}_{i,j} - \sum_i S_i +\] +\[ +S_j \times N_j = \sum_j M^{\rm obs}_{i,j} - \sum_j G_j \\ +\] +This set of equations can be solved iteratively, starting from the +assumption that all chip gains are 1.0, ($G_j = 0$), or by supplying +a guess for the chip gains. The result of this analysis is the +measured chip gains and the measured source illumination levels for +each of the input flat-field images. The chip gains can then be used +to modify the flux levels on the master flat-field images. + +We define the following function to perform the analysis discussed +above: +\begin{prototype} +bool pmFlatNormalization (psVector *sourceFlux, psVector *chipGains, psArray *fluxLevels); +\end{prototype} +The input array \code{fluxLevels} consists of $N_i$ vectors, one per +mosaic image. Each vector consists of $N_j$ elements, each a +measurement of the input flat-field image flux levels. All of these +vectors must be constructed with the same number of elements, or the +function will return an error. If a chip is missing from a particular +image, that element should be set to \code{NaN}. The vector +\code{chipGains} supplies initial guesses for the chip gains. If the +vector contains the values 0.0 or \code{NaN} for any of the elements, +the gain is set to the mean of the valid values. If the vector length +does not match the number of chips, an warning is raised, all chip +gain guesses will be set to 1.0, and the vector length modified to +match the number of chips defined by the supplied \code{fluxLevels}. +The \code{sourceFlux} input vector must be allocated (not +\code{NULL}), but the routine will set the vector length to the number +of source images regardless of the initial state of the vector. All +vectors used by this function must be of type \code{PS_DATA_F64}. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Objects on Images}