Index: /trunk/doc/modules/ModulesSDRS.tex =================================================================== --- /trunk/doc/modules/ModulesSDRS.tex (revision 1103) +++ /trunk/doc/modules/ModulesSDRS.tex (revision 1104) @@ -1,3 +1,3 @@ -%%% $Id: ModulesSDRS.tex,v 1.2 2004-06-10 01:57:54 price Exp $ +%%% $Id: ModulesSDRS.tex,v 1.3 2004-06-26 00:34:15 price Exp $ \documentclass[panstarrs]{panstarrs} @@ -52,5 +52,5 @@ This document describes the Pan-STARRS Image Processing Pipeline (IPP) -Phase 2 processing modules. Phase 2 is the processing stage wherein +Phase 2 image processing modules. Phase 2 is the processing stage wherein the instrumental signatures are removed from the detector images, in preparation for the combination of multiple images in Phase 4. @@ -77,15 +77,18 @@ \item Mask bad pixels; \item Mask cosmic rays; -\item Mask optical defects; -\item Measure the PSF; -\item Find and measure objects; -\item Correct astrometry; and -\item Get postage stamps. +\item \tbd{Mask optical defects;} +\item \tbd{Measure the PSF;} +\item \tbd{Find and measure objects;} +\item \tbd{Correct astrometry; and} +\item \tbd{Get postage stamps.} \end{itemize} -Each of these shall be discussed in turn, below. +Each of these shall be discussed in turn, below. Those modules which +are \tbd{TBD} will be deferred until they may be properly defined. These modules are built on top of, and are dependent upon, the \PS{} -Library, PSLib. +Library, PSLib. Some functions are specified below which should be +added to PSLib, since we anticipate that these will also find use in +other modules. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -97,15 +100,53 @@ a kernel derived from the list of OT shifts made during the exposure. +\subsection{Kernel definition} + +In order to perform a convolution, we need to define the convolution +kernel. We need a more general object than a \code{psImage} so that +we can incorporate the offset from the $(0,0)$ pixel to the $(0,0)$ +value of the kernel\footnote{The kernel has both positive and negative +indices to convey the positive and negative shifts. One might +consider setting the \code{x0} and \code{y0} members of a +\code{psImage} to the appropriate offsets, but this is not the purpose +of these members, and doing so may affect the behavior of other +\code{psImage} operations.}. Hence we define a \code{psKernel}: + +\begin{verbatim} +/** A convolution kernel */ +typedef struct { + int xMin, yMin; ///< Most negative indices + int xMax, yMax; ///< Most positive indices + float **kernel; ///< The kernel data +} psKernel; +\end{verbatim} + +The \code{kernel} member shall point to an image that contains the +kernel values. The kernel has both positive and negative indices to +convey the positive and negative shifts. The maximum extent of these +shifts shall be defined by the \code{xMin}, \code{xMax}, \code{yMin} +and \code{yMax} members. Note that \code{xMin} and \code{yMin}, under +normal circumstances, should be negative numbers. That is, +\code{myKernel->kernel[-3][-2]} may be defined if \code{yMin} and +\code{xMin} are equal to or more negative than -3 and -2, +respectively. + +Of course, we require the appropriate constructor and destructor: +\begin{verbatim} +psKernel *psKernelAlloc(int xMin, int xMax, int yMin, int yMax); +void psKernelFree(psKernel *myKernel); +\end{verbatim} + + \subsection{Calculate the convolution kernel} Given a list of pixel shifts made in the course of OT guiding, -\code{psPhase2GetKernel} shall return an image that represents the -kernel. The API shall be the following: -\begin{verbatim} -/** Returns the kernel of OT shifts made during the course of the exposure. */ -psImage *psPhase2GetKernel(psVector *xShifts, ///< List of OT shifts in x; integers - psVector *yShifts ///< List of OT shifts in y; integers; xshifts->n == yShifts->n - ); -\end{verbatim} +\code{psPhase2GetKernel} shall return the kernel. The API shall be +the following: +\begin{verbatim} +psKernel *psPhase2GetKernel(const psVector *xShifts, const psVector *yShifts); +\end{verbatim} + +The shift vectors, \code{xShifts} and \code{yShifts}, will be supplied +by the user, most probably from the FITS file containing the image. The elements of the shift vectors should be of an integer type, @@ -114,31 +155,38 @@ The format of the shifts vectors will be a list of the net shift -position relative to the starting point for each timestep (there will -likely be about 10~Hz $\times$ 30~sec = 300 timesteps). - -If the vectors are not of the same size, then the function shall -generate an error. +position relative to the starting point for each timestep. In the +normal course of \PS{} observations using orthogonal transfer, there +will likely be about 10~Hz $\times$ 30~sec $=$ 300 timesteps. + +If the vectors are not of the same number of elements, then the +function shall generate a warning shall be generated, following which, +the longer vector trimmed to the length of the shorter, and the +function shall continue. \subsection{Convolve an image with the kernel} Given an input image and the convolution kernel, -\code{psPhase2ConvolveWithKernel} shall convolve the input image with -the kernel and return the convolved image. The API shall be the -following: -\begin{verbatim} -/** Returns an image that is the result of convolving the input image with the specified kernel. */ -psImage *psPhase2ConvolveWithKernel(psImage *out, ///< Output image, or NULL - const psImage *in, ///< Input image to be convolved - const psImage *kernel ///< Kernel by which to convolve - ); -\end{verbatim} - -Two methods shall be used to do the convolution: for small kernels -(smaller than 200 total pixels), the convolution shall be performed in -real space; for large kernels (larger than 200 total pixels), the -convolution shall be performed using Fast Fourier Transforms (FFTs). - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - +\code{psPhase2ConvolveWithKernel} shall convolve the input image, +\code{in}, with the kernel, \code{kernel} and return the convolved +image, \code{out}. The API shall be the following: +\begin{verbatim} +psImage *psImageConvolve(psImage *out, const psImage *in, const psKernel *kernel, bool direct); +\end{verbatim} + +Two methods shall be available for the convolution: if \code{direct} +is \code{true}, then the convolution shall be performed in real space +(appropriate for small kernels); otherwise, the convolution shall be +performed using Fast Fourier Transforms (FFTs; appropriate for larger +kernels). + +In the event that \code{out} is \code{NULL}, a new \code{psImage} +shall be allocated and returned. + +\subsection{PSLib Routines} + +\code{psKernel} and \code{psImageConvolve} shall be part of the \PS{} +Library, \code{PSLib}, so that they may be re-used for other modules. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Bias subtraction and trim} @@ -162,6 +210,6 @@ A \code{psImageRegion} specifies the offset from $(0,0)$ on the detector\footnote{Note that an image may be smaller than the entire -area of the detector if the detector is windowed.} and the size of the -region. +area of the detector if the detector is windowed, or if a subimage has +been generated.} and the size of the region. We also require a corresponding constructor and destructor: @@ -181,4 +229,68 @@ (\code{psList}) of \code{psImageRegion}s. +\subsection{Splines} + +A spline is a popular choice for fitting 1D data, such as overscans, +but we neglected to define them for PSLib. We now define +one-dimensional cubic splines, \code{psSpline1D}, which shall be +included into PSLib: + +\begin{verbatim} +/** A 1D cubic-spline */ +typedef struct { + int n; ///< Number of spline pieces + float *coeff[4]; ///< Coefficients. There are 4 coefficients for each spline piece. + float *coeffErr[4]; ///< Errors in the coefficients. + bool *mask[4]; ///< Mask for the coefficients. + float *domains; ///< The boundaries between each spline piece. Size is n+1. +} psSpline1D; +\end{verbatim} + +The \code{psSpline1D} structure is very similar to that of the +one-dimensional Chebyshev polynomial, \code{psPolynomial1D}, +except for the following: +\begin{itemize} +\item Each spline piece has 4 coefficients (with corresponding error +and mask), corresponding to each of the zeroth through third order +coefficients of a cubic; and +\item It includes an additional member, \code{domains}, which +specifies the boundaries between each spline piece (including the two +ends). +\end{itemize} + +Of course, we require the appropriate constructors and destructor: +\begin{verbatim} +/** Constructors */ +psSpline1D *psSpline1DAlloc(int n, float min, float max); +psSpline1D *psSpline1DAllocGeneric(const psVector *bounds); +void psSpline1DFree(psSpline1D *mySpline); +\end{verbatim} + +\code{psSpline1DAlloc} shall allocate and return a \code{psSpline1D}, +setting the \code{domains} on the basis of the input number of spline +pieces, \code{n}, and the minimum (\code{min}) and maximum +(\code{max}) data values. + +\code{psSpline1DAllocGeneric} shall allocate and return a +\code{psSpline1D}, using the \code{bounds} to define the number of +spline pieces and the \code{domains}. + +\code{psSpline1DFree} shall free the given spline, and its members. + +Also, as for the polynomials, we require an evaluator. Given a +\code{spline} and ordinate at which to evaluate, \code{x}, +\code{psSpline1DEval} shall evaluate and return the value of the +spline at the ordinate. If the ordinate is outside the bounds, then +the function shall generate a warning, and extrapolate the spline to +the ordinate and return the value. + +\begin{verbatim} +/** Evaluator */ +float psSpline1DEval(const psSpline1D *spline, ///< Spline to evaluate + float x ///< Normalised (-1,1) coordinates at which to evaluate + ); +\end{verbatim} + + \subsection{Bias subtraction} @@ -187,25 +299,36 @@ The API shall be the following: \begin{verbatim} -/** Subtracts an overscan and bias from the input image. */ -psReadout *psPhase2SubtractBias(psReadout *in, ///< Input image to be de-biased, and output - psPolynomial1D *polySpec, ///< Polynomial specification to use for fit, - ///< outputed - const psList *regions, ///< Linked list of psImageRegion types. - psOverscanAxis overscanAxis, ///< Overscan axis - const psStatsOptions *stat, ///< Statistic to use - const psImage *bias ///< Bias (or dark) image to subtract - ); -\end{verbatim} - -The input image shall have the bias subtracted in-place. - -The polynomial specification, \code{polySpec}, specifies the order of -a polynomial fit which may be made to the overscan if the -\code{overscanAxis} is \code{PS_OVERSCAN_ROWS_FIT} or -\code{PS_OVERSCAN_COLUMNS_FIT}. \code{polySpec} shall, upon return, -contain the coefficients of the overscan fit, if performed. - -The prescan and/or overscan regions to be used are specified in -\code{regions}. +psReadout *psPhase2SubtractBias(psReadout *in, void *fitSpec, psOverscanAxis overscanAxis, psBiasFit fit, + int nBin, const psList *regions, const psStats *stat, const psImage *bias); +\end{verbatim} + +The input image, \code{in}, shall have the bias subtracted in-place. + +The type of the overscan fit specification, \code{fitSpec}, shall be +dependent upon the value of \code{fit}, which specifies the type of +fit to be employed. \code{fit} is an enumerated type: + +\begin{verbatim} +/** Fit type for bias */ +typedef enum { + PS_OVERSCAN_FIT_NONE, ///< No fit to bias + PS_OVERSCAN_FIT_LINEAR, ///< Do linear interpolation on bias + PS_OVERSCAN_FIT_POLYNOMIAL, ///< Fit polynomial to bias + PS_OVERSCAN_FIT_CHEBYSHEV, ///< Fit chebyshev to bias + PS_OVERSCAN_FIT_SPLINE ///< Fit cubic splines to bias +} psBiasFit; +\end{verbatim} + +If \code{fitSpec} is \code{NULL}, or \code{fit} is +\code{PS_OVERSCAN_FIT_NONE}, then no fit shall be performed to the +overscan. If \code{fit} is \code{PS_OVERSCAN_FIT_LINEAR}, then the +function shall perform simple linear interpolation between points in +the overscan. Otherwise, \code{fitSpec} shall be interpreted to be a +structure of the appropriate type (\code{psPolynomial1D} for +\code{PS_OVERSCAN_FIT_POLYNOMIAL} and +\code{PS_OVERSCAN_FIT_CHEBYSHEV}, and \code{psSpline1D} for +\code{PS_OVERSCAN_FIT_SPLINE}), and the overscan shall be fit using +the specified functional form. Upon return, the \code{fitSpec} shall +contain the coefficients of the overscan fit. The \code{overscanAxis} specifies how the prescan/overscan subtraction @@ -214,42 +337,56 @@ /** Overscan axis */ typedef enum { - PS_OVERSCAN_NONE = 0, ///< No overscan subtraction - PS_OVERSCAN_ROWS = 1, ///< Subtract rows - PS_OVERSCAN_COLUMNS = 2, ///< Subtract columns - PS_OVERSCAN_ROWS_FIT = -1, ///< Subtract a fit to rows - PS_OVERSCAN_COLUMNS_FIT = -2, ///< Subtract a fit to columns - PS_OVERSCAN_ALL = 3 ///< Subtract the statistic of all pixels in overscan region + PS_OVERSCAN_NONE, ///< No overscan subtraction + PS_OVERSCAN_ROWS, ///< Subtract rows + PS_OVERSCAN_COLUMNS, ///< Subtract columns + PS_OVERSCAN_ALL ///< Subtract the statistic of all pixels in overscan region } psOverscanAxis; \end{verbatim} -No prescan/overscan subtraction shall be performed if the -\code{overscanAxis} is \code{PS_OVERSCAN_NONE} or if \code{regions} is -\code{NULL}. The subtraction shall be on the basis of rows or columns -if the \code{overscanAxis} is \code{PS_OVERSCAN_ROWS} or -\code{PS_OVERSCAN_COLUMNS}, respectively. If the \code{overscanAxis} -is \code{PS_OVERSCAN_ROWS_FIT} or \code{PS_OVERSCAN_COLUMNS_FIT}, a -fit to the prescan/overscan region shall be subtracted from the rows -or columns, respectively. Finally, if the \code{overscanAxis} is -\code{PS_OVERSCAN_ALL}, then the appropriate statistic (specified by -\code{stat}) of all of the pixels in the prescan/overscan region(s) -shall be subtracted from the image. +If the \code{overscanAxis} is \code{PS_OVERSCAN_NONE}, then the +function shall not perform any overscan subtraction. If the +\code{overscanAxis} is \code{PS_OVERSCAN_ALL}, then all the overscan +regions shall be used to generate a single statistic (specified by +\code{stat}) which shall be subtracted from the entire image. A +warning shall be generated if the \code{overscanAxis} is +\code{PS_OVERSCAN_NONE} or \code{PS_OVERSCAN_ALL} and the \code{fit} +is not \code{PS_OVERSCAN_FIT_NONE}. + +If the \code{overscanAxis} is \code{PS_OVERSCAN_ROWS} or +\code{PS_OVERSCAN_COLUMNS}, then the overscan shall be reduced to a +single vector in the dimension specified using the specified statistic +(\code{stat}). If \code{nBin} is positive and less than the dimension +of the vector, then the vector shall subsequently be binned down into +\code{nBin} bins, again using the specified statistic (\code{stat}). +If the overscan is not defined for each row/column, then the function +shall generate a warning and then interpolate using the provided +functional form if \code{fit} is not \code{PS_OVERSCAN_FIT_NONE}; +otherwise, the function shall generate an error. + +The prescan and/or overscan regions to be used are specified in +\code{regions}, which is a linked list of \code{psImageRegion}s. If +\code{regions} is non-\code{NULL} and \code{overscanAxis} is not +\code{PS_OVERSCAN_NONE}, then the function shall generate a warning. The statistic to use in combining multiple pixels in the -prescan/overscan regions is specified by \code{stat}. Legal values -shall be \code{PS_STAT_SAMPLE_MEAN}, \code{PS_STAT_SAMPLE_MEDIAN}, -\code{PS_STAT_ROBUST_MEAN}, \code{PS_STAT_ROBUST_MEDIAN}, -\code{PS_STAT_ROBUST_MODE}. \tbd{Note in particular that -%\code{PS_STAT_CLIPPED_MEAN} -a clipped mean is not supported, because that means we have to supply -clipping parameters which isn't very neat.} - -A full-frame bias (or dark) image shall be subtracted if \code{bias} -is non-NULL. Note that the input image, \code{in}, and the -\code{bias} image need not be the same size, but the function shall -use the offsets in the image (\code{in->x0} and \code{in->y0}) to -determine the appropriate offsets to obtain the correct pixel on the -\code{bias}. In the event that the \code{bias} image is too small -(i.e., pixels on the input image refer to pixels outside the range of -the \code{bias} image), the function shall generate an error. +prescan/overscan regions is specified by \code{stat}. \code{stat} is +of type \code{psStats} instead of simply \code{psStatsOptions} so that +clipping levels may be specified, if desired. In the event that +multiple options are specified by \code{stats}, a warning shall be +generated, and the option with the highest priority shall be used, +according to the following priority order: \code{PS_STAT_SAMPLE_MEAN}, +\code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN}, +\code{PS_STAT_ROBUST_MEAN},\ \code{PS_STAT_ROBUST_MEDIAN}, +\code{PS_STAT_ROBUST_MODE}. + +A bias (or dark) image shall be subtracted pixel-by-pixel from the +input image if \code{bias} is non-NULL. Note that the input image, +\code{in}, and the \code{bias} image need not be the same size, but +the function shall use the offsets in the image (\code{in->x0} and +\code{in->y0}) to determine the appropriate offsets to obtain the +correct pixel on the \code{bias}. In the event that the \code{bias} +image is too small (i.e., pixels on the input image refer to pixels +outside the range of the \code{bias} image), the function shall +generate an error.