Index: /trunk/doc/ipptools/pstask.tex =================================================================== --- /trunk/doc/ipptools/pstask.tex (revision 4901) +++ /trunk/doc/ipptools/pstask.tex (revision 4902) @@ -508,3 +508,275 @@ above, the 'process' stage is a null operation. +\section{Metadata Database Tables used for IPP Job Flow} +\label{sec:MetadataTableContents} + +The tables presented here define in greater detail the contents of the +Metadata tables show in the figures above. In some cases, the +quantities (eg, the analysis result statistics) are illustrative, not +definitive. In certain tables, data is provided which is redundant +(non-normal) for ease of use. In some cases, we may decide not to +keep this redundant information. In cases where the choice to +replicate the redundant information is uncertain, the field names are +written in {\it italics}. + +\begin{table}[bh] +\begin{center} +\caption{Pending Image Files\label{tab:PendingImageFiles}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +URL & string & file location & http://data/file001.fits \\ +exp ID & string & exposure ID & 654321o \\ +{\it camera} & string & camera name & MegaPrime / GPC \\ +class & string & file data class & Cell / Chip / FPA \\ +class ID & string & identify for class & chip00 / cell0102 \\ +state & string & state of transfer? & ready / copied \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{New Image Files\label{tab:NewImageFiles}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +URL & string & file location & neb://file001.fits \\ +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +class & string & file data class & Cell / Chip / FPA \\ +class ID & string & identify for class & chip00 / cell0102 \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{Raw Image Files\label{tab:RawImageFiles}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +URL & string & file location & neb://file001.fits \\ +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +class & string & file data class & Cell / Chip / FPA \\ +class ID & string & identify for class & chip00 / cell0102 \\ +type & string & exposure type & bias / flat / science \\ +Nreadout & int & number of readouts & 1 / 5 / 100 \\ +Treadout & float & readout exposure time & 1.0 (sec) \\ +ccdtemp & float & detector temperature & 90 (K) \\ +\hline +background & float & data area median & 5.0 (sec) \\ +FHWM & float & average image quality & 2.5 (arcsec) \\ +\hline +\multicolumn{3}{l}{note: stats below the line are measured, perhaps they go elsewhere?} \\ +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{Pending Metadata Tables\label{tab:PendingTables}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +URL & string & file location & neb://file001.fits \\ +table ID & string & unique table identifier & table-654321 \\ +table type & string & table content type & temps / skyprobe data \\ +state & string & copied? & ready / copied \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{Copied Metadata Tables\label{tab:CopiedTables}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +URL & string & file location & neb://file001.fits \\ +table ID & string & unique table identifier & table-654321 \\ +table type & string & table content type & temps / skyprobe data \\ +state & string & copied? & ready / copied \\ +time & int & table arrival time & 13243413 (s) \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{New Exposures\label{tab:NewExp}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +telescope & string & telescope name & CFHT / PS-1 / SP-1 \\ +class & string & file data class & Cell / Chip / FPA \\ +Nfiles & int & number image files & 64 \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{Raw SCIENCE Exposure\label{tab:RawScienceExp}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +telescope & string & telescope name & CFHT / PS-1 / SP-1 \\ +class & string & file data class & Cell / Chip / FPA \\ +Nfiles & int & number of image files & 1 / 64 \\ +\hline +exptime & float & exposure time & 10.0 (seconds) \\ +filter.ID & string & filter ID (glass) & g.PS1.01 \\ +filter.Band & string & filter bandpass name & g \\ +airmass & float & sec(zenith angle) & 1.35 \\ +RA & float & boresite RA & 125.01 (degrees) \\ +DEC & float & boresite DEC & 35.01 (degrees) \\ +PA & float & FPA position angle & 10.1 (degrees) \\ +obstime.sec & float & observation start & 123423422 (TAI) \\ +obstime.nsec & float & observation start (nsec) & 1234 \\ +telfocus & float & focus distance & 1.50 (mm) \\ +FPAtemp & float & FPA temperature & 90 (K) \\ +dometemp & float & dome air temperature & 290 (K) \\ +airtemp & float & outside air temperature & 285 (K) \\ +mirrortemp & float & primary mirror temp & 286 (K) \\ +teltemp & float & telescope structure temp & 283 (K) \\ +group ID & string & observation group name & target-seq-01 \\ +group seq & int & sequence number in group & 3 \\ +\hline +background & float & average of file skies & 5.0 \\ +dbackground & float & stdev of file skies & 1.0 \\ +FHWM & float & \\ +\hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{Raw Detrend Exposures\label{tab:RawDetrendExp}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +telescope & string & telescope name & CFHT / PS-1 / SP-1 \\ +type & string & exposure type & bias / domeflat / dark \\ +class & string & file data class & Cell / Chip / FPA \\ +Nfiles & int & number of image files & 1 / 64 \\ +\hline +exptime & float & exposure time & 10.0 (seconds) \\ +filter.ID & string & filter ID (glass) & g.PS1.01 \\ +filter.Band & string & filter bandpass name & g \\ +Alt & float & boresite Alt & 125.01 (degrees) \\ +Az & float & boresite Az & 35.01 (degrees) \\ +PA & float & FPA position angle & 10.1 (degrees) \\ +obstime.sec & float & observation start & 123423422 (TAI) \\ +obstime.nsec & float & observation start (nsec) & 1234 \\ +telfocus & float & focus distance & 1.50 (mm) \\ +FPAtemp & float & FPA temperature & 90 (K) \\ +dometemp & float & dome air temperature & 290 (K) \\ +airtemp & float & outside air temperature & 285 (K) \\ +mirrortemp & float & primary mirror temp & 286 (K) \\ +teltemp & float & telescope structure temp & 283 (K) \\ +group ID & string & observation group name & target-seq-01 \\ +group seq & int & sequence number in group & 3 \\ +callamp.state & string & calibration lamp state & on / off \\ +callamp.mode & string & calibration lamp mode & continuum / line / ?? \\ +\hline +background & float & average of file back & 5.0 \\ +dbackground & float & stdev of file back & 1.0 \\ +FHWM & float & \\ +hline +\end{tabular} +\end{center} +\end{table} + +\begin{table}[bh] +\begin{center} +\caption{P1 Exposures\label{tab:P1-Exp}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +exp ID & string & exposure ID & 654321o \\ +camera & string & camera name & MegaPrime / GPC \\ +version & int & P1 version number & 01 \\ +recipe & string & analysis recipe name & basic / flattest \\ +state & string & P1 analysis state & new / done \\ +\hline +output & string & location of output file & rootname.P1.01.smf \\ +P1-log & string & location of P1 logfile & rootname.P1.01.log \\ +Nstars & int & number of astrom stars & 50 \\ +sigma.X & float & astrom scatter in X & 1.0 (arcsec) \\ +sigma.Y & float & astrom scatter in Y & 1.2 (arcsec) \\ +Mcal & float & nominal zeropoint & 25.2 (mag) \\ +Moff & float & measure ZP offset & 0.5 (mag) \\ +dMoff & float & scatter in Moff & 0.1 (mag) \\ + +\begin{table}[bh] +\begin{center} +\caption{P2 Images\label{tab:P2-Images}} +\begin{tabular}{lll} +\hline +\hline +{\bf Field Name} & {\bf Datatype } & {\bf Description} & {\bf Examples} \\ +\hline +exp ID & string & exposure ID & 654321o \\ +class & string & file data class & Cell / Chip / FPA \\ +class ID & string & identify for class & chip00 / cell0102 \\ +version & int & P1 version number & 01 \\ +recipe & string & analysis recipe name & basic / flattest \\ +state & string & P1 analysis state & new / done \\ +input.url & string & file location & rootname.fits \\ +output.image.url & string & file location & rootname.P2.01.fits \\ +output.obj.url & string & file location & rootname.P2.01.smf \\ +output.log.url & string & file location & rootname.P2.01.log \\ +\hline +bias & float & measured bias value & 5.0 \\ +dbias & float & bias residual scatter & 5.0 \\ +Nstars & int & number of astrom stars & 50 \\ +sigma.X & float & astrom scatter in X & 1.0 (arcsec) \\ +sigma.Y & float & astrom scatter in Y & 1.2 (arcsec) \\ +Mcal & float & nominal zeropoint & 25.2 (mag) \\ +Moff & float & measure ZP offset & 0.5 (mag) \\ +dMoff & float & scatter in Moff & 0.1 (mag) \\ +\hline +bias-image +dark-image +flat-image +\hline +\end{tabular} +\end{center} +\end{table} + + + + \end{document} Index: /trunk/doc/psphot/psphot.tex =================================================================== --- /trunk/doc/psphot/psphot.tex (revision 4901) +++ /trunk/doc/psphot/psphot.tex (revision 4902) @@ -18,21 +18,54 @@ \maketitle -\note{the current configuration variables and some of the function - names are not very well chosen. expect these to be modified as the - code base is cleaned.} - \section{Introduction} -\subsection{Background} - -{\bf note: add discussion of the PS project overview} +\subsection{Overview} + +The Institute for Astronomy at the University of Hawaii is developing +a large optical synoptic survey telescope system, the Panoramic Survey +Telescope and Rapid Response System (Pan-STARRS). The science goals, +priorities, top-level concept of operations with associated +operational requirements, and system performance drivers with +associated system performance requirements are described in the +Pan-STARRS Science Goals Statement (SGS). As described in this +document, The system conceptual design for Pan-STARRS utilizes an +array of four 1.8m telescopes each with a 7 degree$^2$ field of view, +giving the system an \'etendue larger than all existing survey +instruments combined (defined as the product of the collecting area +$A$ multiplied by the field-of-view solid angle $\Omega$). Each +telescope will be equipped with a 1.4 billion pixel CCD camera with +low noise and rapid read-out, and the data will be reduced in near +real time to produce both cumulative static sky and difference images +from which transient, moving, and variable objects can be +detected. Pan-STARRS will be able to survey up to $\approx 6,000$ +degree$^{2}$ per night to a detection limit of approximately 24$^{th}$ +magnitude. This unique combination of sensitivity and sky coverage +will open up many new possibilities in time domain astronomy including +a major goal of surveying the Potentially Hazardous Object (PHO) +population down to a diameter of $\approx 300$ meters. In addition, +the Pan-STARRS data will be used to investigate a broad range of +astronomical problems of extreme current interest concerning the Solar +System, the Galaxy, and the Cosmos at large. A prototype single +telescope system, PS-1, is being developed as a preliminary step +before construction of the complete four telescope system. + +\begin{tabular}{ll} +Project sponsor:& AFRL, United States Air Force \\ +Acquirer: & University of Hawaii Institute for Astronomy \\ +User: & Astronomical community \\ +Developer: & University of Hawaii Institute for Astronomy, participating \\ + & institutions, and associated subcontractors +\end{tabular} The Pan-STARRS Image Processing Pipeline is responsible for the basic analysis of images from the Pan-STARRS telescopes Gigapixel Camera. -Among the Pan-STARRS project survey goals is a repeated all-sky survey -in 5 filters, {\it grizy}, beginning with a pre-survey with the -prototype telescope PS-1. The photometric and astrometric precision -goals for the all-sky surveys, as well as the other survey components, -are quite stringent: +The overall goals and requirements of the Image Processing Pipeline +are described in the IPP System/Subsystem Design Description (SSDD; +PSDC-430-XXX) and the IPP System Requirements Specification (SRS; +PSDC-430-XXX). Among the Pan-STARRS project survey goals is a +repeated all-sky survey in 5 filters, {\it grizy}, beginning with a +pre-survey with the prototype telescope PS-1. The photometric and +astrometric precision goals for the all-sky surveys, as well as the +other survey components, are quite stringent: \begin{itemize} @@ -40,10 +73,9 @@ across the sky in the internal photometric system; -\item relative astrometry; 10 milliarcseconds scatter {\bf note: 2-D -radial or 1-D linear?} for individual stars between repeated images. +\item relative astrometry; 10 milliarcseconds scatter for individual +stars between repeated images. \item absolute astrometry: 100 milliarcseconds scatter for all ICRS reference stars (Tycho). - \end{itemize} @@ -68,5 +100,13 @@ astrometry. -\subsubsection{Existing Photometry Analysis Programs} +\subsection{Comparable Programs} + +A variety of astronomical software packages perform the basic object +detection, measurement, and classification tasks needed by the +Pan-STARRS IPP. Each of these programs have their own advantages and +disadvantages. Below we discuss some of the most widely used of these +other packages, highlighting the features of the programs which are +particularly desirable, and noting aspects of the programs which are +problematic for the IPP. \begin{itemize} @@ -75,27 +115,30 @@ pro: well-tested, stable code. con: limited range of models, algorithm converges slowly to a PSF model, limited tests of PSF - validity, inflexible code base, fortran + validity, inflexible code base, fortran (P. Schechter) \item DAOPhot : Pixel-map PSF model with analytical component. pro: well-tested, high-quality photometry. con: Difficult to use in an - automated fashion, does it handle 2D variations well? + automated fashion, does it handle 2D variations well? (P. Stetson) \item Sextractor : pure aperture measurement with rudimentary object subtraction. pro: fast, widely used, easy to automate. con: poor object separation in crowded regions, PSF-modeling is only - beta (psfex), what models are available? + beta (psfex), what models are available? (E. Bertin) \item apphot : IRAF-based aperture photometry. pro: widely used. - con: IRAF-based, aperture photometry. + con: IRAF-based, aperture photometry. (???) \item galfit : detailed galaxy modeling. not a multi-object PSF analysis tool. con: does not provide a PSF model, not easily automated. very detailed results in very slow processing. only a - galaxy analysis program. + galaxy analysis program. (C. Impey) \item SDSS phot : con: tightly integrated into the SDSS software - environment. + environment. (R. Lupton) \end{itemize} + +\note{discussion of these packages is insufficient: flesh out + discussion and add in the references.} The Pan-STARRS IPP team decided that none of the existing packages met @@ -103,12 +146,13 @@ the project. We decided to redesign the photometry analysis from scratch, using the lessons learned from the existing photometry -systems. In addition, the software would be written using the data -analysis C-code library written for the IPP, \code{psLib}, and -integrate the elements of the photometry code into the IPP's mid-level -astronomy data analysis toolkit called \code{psModules}. The result -is 'PSPhot', which can be used either as a stand-alone C program, or -as one of the high-level IPP components of \code{psModules}, available -to programmers either via a C interface or through a SWIG interface in -Perl (or potentially Python). +systems. In the process, the object analysis software would be +written using the data analysis C-code library written for the IPP, +\code{psLib}, and the components of the photometry code would be +integrated into the IPP's mid-level astronomy data analysis toolkit +called \code{psModules}. The result is 'PSPhot', which can be used +either as a stand-alone C program, or as one of the high-level IPP +components of \code{psModules}, available to programmers either via a +C interface or through a SWIG interface in Perl (or potentially +Python). \note{Add discussion of the lessons learned from experience with previous @@ -119,5 +163,5 @@ easily modified. \item PSF variation is fundamental : PSF representation should incorporate 2-D variations. -\item Speed fitting with accurate parameter guesses. +\item Speed fitting with accurate parameter guesses. \item Make good use of moment information to speed analysis. \item Careful definition of PSF validity tests. @@ -127,5 +171,5 @@ \end{itemize} -\section{Description of the PSPhot analysis steps} +\section{PSPhot Analysis Process} \subsection{Overview} @@ -155,4 +199,8 @@ difference image, noise image, etc, as selected. \end{itemize} + +\note{the current configuration variables and some of the function + names are not very well chosen. expect these to be modified as the + code base is cleaned.} \subsection{Image Preparation} @@ -186,10 +234,15 @@ constructed by default from the image by applying three rules: 1) Pixels which are above a specified saturation level are marked as -saturated. 2) Pixels which are below a user-defined value are -considered unresponsive and masked as dead \note{currently -unimplemented and ignored}. 3) Pixels which lie outside of a -user-defined window are considered non-data pixels (eg, overscan) and -are marked as invalid \note{mask values are currently hard-wired -numbers : they should be given named enum values} +saturated (configuration keyword: \code{SATURATE}). 2) Pixels which +are below a user-defined value are considered unresponsive and masked +as dead. 3) Pixels which lie outside of a user-defined window are +considered non-data pixels (eg, overscan) and are marked as invalid. +The valid window is defined by the configuration variables +\code{XMIN}, \code{XMAX}, \code{YMIN}, \code{YMAX}. + +\note{minimum valid data value is currently unimplemented} + +\note{mask values are currently hard-wired numbers : they should be +given named enum values} The noise image, if not supplied is constructed by default from the @@ -204,17 +257,15 @@ represents the noise as a function of position in the image. -\subsubsection{Initial Object Detection} +\subsection{Initial Object Detection} The objects are initially detected by finding the location of local -peaks in the image. \note{In the ideal case, if we were only -interested in detecting PSFs, and we had a good model for the PSF, we -could optimally find the sources by smoothing the image and the noise -image with the PSF model. \em write out the description of Nick's -optimal PSF finding}. The flux image is smoothed with a very small +peaks in the image. The flux image is smoothed with a very small circularly symmetric kernel using a two-pass 1D Gaussian. At this -stage, the goal is to detect only the brighter sources, above a user -defined S/N limit. The detection efficiency for the brighter sources -is not strongly dependent on the form of this smoothing function. -\note{is the smoothing needed?} +stage, the goal is only to detect the brighter sources, above a user +defined S/N limit (configuration keyword: \code{PEAK_NSIGMA}). The +detection efficiency for the brighter sources is not strongly +dependent on the form of this smoothing function. + +\note{Is this smoothing needed?} The local peaks in the smoothed image are found by first detecting @@ -224,28 +275,34 @@ neighboring pixels is kept. Any peak which is lower than any of the 8 neighboring pixels is rejected. Any peak which has the same value as -any of the other 8 pixels is kept if the pixel X and Y coordinates are -greater than or equal to the other equal value pixels. This simple -rule set means that a flat-topped region will maintain peaks at the -maximum X and Y corners of the region. - -\note{the current implementation is ignoring the S/N map in making the -peak detection. This means that we cannot use the same code to find -peaks in a difference image or to re-find peaks in the image after the -modeled objects have been subtracted}. +any of the other 8 pixels is kept if the pixel $X$ and $Y$ coordinates +are greater than or equal to the other equal value pixels. This +simple rule set means that a flat-topped region will maintain peaks at +the maximum $X$ and $Y$ corners of the region. + +\note{The current implementation ignores the S/N map in making the +peak detection. This code must be modified (a la Kaiser) to be used +for a peak-detection pass in a difference image or to re-find peaks in +the image after the modeled objects have been subtracted}. Once a collection of peaks have been identified, basic properties of the objects are measured. First, the local sky flux is measured -(using Median? user-specific method?) within a square annulus with -user-defined dimensions (\code{INNER_RADIUS} and \code{OUTER_RADIUS}). -\note{rejection of some peaks based on the local sky measurement?}. -This local background value is then used to calculate the object first -and second moments within a small user-defined aperture +within a square annulus with user-defined dimensions +(\code{INNER_RADIUS} and \code{OUTER_RADIUS}), using the sample +median. This local background value is then used to calculate the +object first and second moments within a small user-defined aperture (\code{MOMENT_RADIUS}). The first-order moments are a good representation of the object position, while the second-order moments are a measure of the object shape. The second-order moments are somewhat sensitive to the size of the aperture and the accuracy of the -background measurement. \note{discuss object rejection based on the -value of the object moments and the value of the centroid vs peak -pixel coordinates}. +background measurement. The moment calculation is only performed +using pixels which exceed a S/N of 1. If, in the process of +calculating the source moments, the S/N limits reject all but \note{3} +or fewer of the source pixels, the peak is identified as being +suspect, and is not used for further analysis. If the measured +centroid coordinates differ from the peak coordinates be a large +amount (\code{MOMENT_RADIUS}), then the peak is again identified as +being of poor quality and is rejected. In both of these cases, it is +likely that the `peak' was identified in a region of flat flux +distribution or many saturated or edge pixels. \subsection{PSF Determination} @@ -256,13 +313,20 @@ object. An important concept within the PSPhot code is the distinction between a model which describes an object on an image and -a model with describes the point-spread-function across an image. +a model with describes the point-spread-function (PSF) across an +image. Any object in an image may be represented by some analytical model, -for example, a 2-D elliptical Gaussian. The object model will have a -variety of model parameters, in this case the centroid coordinates -($x_o, y_o$), the elliptical shape parameters ($\sigma_x, \sigma_y, -\sigma_{xy}$), the model normalization ($I_o$) and the local value of -the background ($S$). A specific object will have a particular set of -values for these different parameters. +for example, a 2-D elliptical Gaussian: +\begin{eqnarray} +f(x,y) & = & I_o exp (-z) + S \\ + R & = & \frac{(x - x_o)^2}{2\sigma_x^2} + \frac{(y - + y_o)^2}{2\sigma_y^2} + \sigma_{\rm xy}(x - x_o)(y - y_o) +\end{eqnarray} +The object model will have a variety of model parameters, in this case +the centroid coordinates ($x_o, y_o$), the elliptical shape parameters +($\sigma_x, \sigma_y, \sigma_{\rm xy}$), the model normalization +($I_o$) and the local value of the background ($S$). A specific +object will have a particular set of values for these different +parameters. The point-spread-function (PSF) of an image describes the shape of all @@ -276,10 +340,10 @@ independent from object to object. For the case of the elliptical Gaussian model, the PSF parameters would be the shape terms -($\sigma_x, \sigma_y, \sigma_{xy}$) while the independent parameters -would be the centroid, normalization and local sky values ($x_o, y_o, -I_o, S$). PSPhot uses a 2-D polynomial to specify the variation in -the PSF parameters as a function of position in the image. In the -case of the elliptical Gaussian, this implies that the parameters are -each a function of the object centroid coordinates: +($\sigma_x, \sigma_y, \sigma_{\rm xy}$) while the independent +parameters would be the centroid, normalization and local sky values +($x_o, y_o, I_o, S$). PSPhot uses a 2-D polynomial to specify the +variation in the PSF parameters as a function of position in the +image. In the case of the elliptical Gaussian, this implies that the +parameters are each a function of the object centroid coordinates: \begin{eqnarray} \sigma_x & = & f_1(x,y) \\ @@ -309,33 +373,41 @@ 3 for the PSF model. -\note{coding diversion} Throughout PSPhot, there are many places where -it is necessary for the code to refer to an aspect of the object or -PSF model. Often, these quantities are needed deep within other parts -of the code. For example, when attempting to fit the pixel flux -values for an object, it is necessary to generate a guess for the -model parameters. Or, in order to limit the domain of the fit, it is -necessary to determine an isophotal radius for a model. In order to -avoid having the code depend on the specific form of a model, all of -these types of circumstances are abstracted, and a method is provided -to return the necessary function to the higher-level software. For -example, each model type has its own function to define an initial -guess for the model, or a function to determine the radius for a given -flux level. These are then registered as part of the model function -code. Another function is then used to return the appropriate -function for a specific model type. For example, the -\code{psModelLookup_GetFunction} will return the \code{psModelLookup} -function for a given model type. This mechanism makes it very easy to -add new model functions into the PSPhot code base. To add a new model -function, the programmer simply defines a new model name (a string), -the set of all necessary model lookup functions, and places the -reference to the model code at the appropriate location in the -psModelInit.c routine. It is not necessary to specify the PSF model -functions independently or the object model functions. Nor is it -necessary to identify the intended use of a given object model -function (ie, PSF-like object, galaxy, comet, etc). Any model can be -used for the PSF model. The code currently uses a fixed translation -between the object model parameters and the PSF model parameters. It -also defines a specific order for the 4 independent parameters. -\note{it may also require that two of the PSF-like parameters +PSPhot is written so that the object detection, measurement, and +classification code does not depend on the specific form of the +available object model functions. Access to the characteristics of +the models is provided through a simple function abstraction method. +Throughout PSPhot, there are many places where it is necessary for the +code to refer to an aspect of the object or PSF model. Often, these +quantities are needed deep within other parts of the code. For +example, when attempting to fit the pixel flux values for an object, +it is necessary to generate a guess for the model parameters. Or, in +order to limit the domain of the fit, it is necessary to determine an +isophotal radius for a model. + +In order to avoid having the code depend on the specific form of a +model, the function calls needed in these types of circumstances are +abstracted, and a method is provided to return the necessary function +to the higher-level software. For example, each model type has its +own function to define an initial guess for the model, or a function +to determine the radius for a given flux level. These are then +registered as part of the model function code. Another function is +then used to return the appropriate function for a specific model +type. For example, the \code{psModelLookup_GetFunction} will return +the \code{psModelLookup} function for a given model type. This +mechanism makes it very easy to add new model functions into the +PSPhot code base. To add a new model function, the programmer simply +defines a new model name (a string), the set of all necessary model +lookup functions, and places the reference to the model code at the +appropriate location in the psModelInit.c routine. + +When a new model is provided to PSPhot, it is not necessary to specify +the intended use of the object model function (ie, PSF-like object, +galaxy, comet, etc). Any model can be used for the PSF model, or to +describe the flux distributions of the non-PSF objects. The code +currently uses a fixed translation between the object model parameters +and the PSF model parameters. It also defines a specific order for +the 4 independent parameters. + +\note{the code may also require that two of the PSF-like parameters represent the shape in some way}. @@ -346,25 +418,26 @@ to be PSF-like. PSPhot uses the object moments to make the initial guess at a collection of PSF-like objects. At this point, the program -has measured the second order moments for all objects identified their -peaks, as well as an approximate signal-to-noise ratio. All objects -with a S/N ratio greater than a user-defined parameter are selected by -PSPhot, though objects which have more than a certain number of -saturated pixels are excluded at this stage. PSPhot then examines the -2-D plane of $\sigma_x, \sigma_y$ in search of a concentrated clump of -objects. To do this, it constructs an artificial image with pixels -representing the value of $\sigma_x, \sigma_y$, using a user-defined -scale for the size of a pixel in this artificial image (note that the -units of the $\sigma_x, \sigma_y$ plane are the size of the -second-moment in pixels in the original image). A typical value for -the bin size is approximately 0.1 image pixels. The binned $\sigma_x, -\sigma_y$ plane is then examined to find a peak which has a -significance greater than XXX. Unless the image is extremely sparse, -such a peak will be well-defined and should represent the objects -which are all very similar in shape. Other objects in the image will -tend to land in very different locations, failing to produce a single -peak. To avoid detecting a peak from the unresolved cosmic rays, -objects which have second-moments very close to 0 are ignored. The -only danger is if the PSF is very small and too many of these objects -are rejected as cosmic rays. +has measured the second order moments for all objects identified by +their peaks, as well as an approximate signal-to-noise ratio. All +objects with a S/N ratio greater than a user-defined parameter +(\code{PSF_SHAPE_NSIGMA} ???) are selected by PSPhot, though objects +which have more than a certain number of saturated pixels are excluded +at this stage. PSPhot then examines the 2-D plane of $\sigma_x, +\sigma_y$ in search of a concentrated clump of objects. To do this, +it constructs an artificial image with pixels representing the value +of $\sigma_x, \sigma_y$, using a user-defined scale for the size of a +pixel in this artificial image (note that the units of the $\sigma_x, +\sigma_y$ plane are the size of the second-moment in pixels in the +original image). A typical value for the bin size is approximately +0.1 image pixels. The binned $\sigma_x, \sigma_y$ plane is then +examined to find a peak which has a significance greater than XXX. +Unless the image is extremely sparse, such a peak will be well-defined +and should represent the objects which are all very similar in shape. +Other objects in the image will tend to land in very different +locations, failing to produce a single peak. To avoid detecting a +peak from the unresolved cosmic rays, objects which have +second-moments very close to 0 are ignored. The only danger is if the +PSF is very small and too many of these objects are rejected as cosmic +rays. Once a peak has been detected in this plane, the centroid and second @@ -378,18 +451,28 @@ model, allowing all of the parameters (PSF and independent) to vary in the fit. PSPhot uses the Levenberg-Marqardt process for the -non-linear fitting \note{discuss the convergence criteria, model -parameter guesses}. In this process, any objects which fail to -converge in the fit are flagged as invalid. For the resulting -collection of object model parameters, the PSF-dependent parameters of -the models are all fitted as a function of position to a 2-D -polynomial. The order of this polynomial is (should be?) a -user-defined parameter. The fitting process for these polynomials is -iterative, and rejects the $3-\sigma$ outliers in each of three -passes. This fitting technique results in a robust measurement of the -variation of the PSF model parameters as a function of position -without being excessively biased by individual objects which fail -drastically. Objects whose model parameters are rejected by this -iterative fitting technique are also marked as invalid and ignored in -the later PSF model fitting stages. +non-linear fitting. Non-linear fitting can be very computationally +intensive, particularly for if the starting parameters are far from +the minimization values. PSPhot uses a the first and second moments +to make a good guess for the centroid and shape parameters for the PSF +models. In order to minimize the impact of close neighbors, the noise +values used in the fit are enhanced by a fraction of the deviation of +the particular pixel value from the model guess. Any objects which +fail to converge in the fit are flagged as invalid. + +\note{does the noise enhancement introduce too much bias?} + +\note{discuss the convergence criteria, model parameter guesses} + +For the resulting collection of object model parameters, the +PSF-dependent parameters of the models are all fitted as a function of +position to a 2-D polynomial. The order of this polynomial is (should +be?) a user-defined parameter. The fitting process for these +polynomials is iterative, and rejects the $3-\sigma$ outliers in each +of three passes. This fitting technique results in a robust +measurement of the variation of the PSF model parameters as a function +of position without being excessively biased by individual objects +which fail drastically. Objects whose model parameters are rejected +by this iterative fitting technique are also marked as invalid and +ignored in the later PSF model fitting stages. All of the PSF-candidate objects are then re-fitted using the PSF @@ -402,10 +485,10 @@ model for this particular image. -The metric used by PSPhot to assess the PSF model is currently the -scatter in the differences between the aperture and fit magnitudes for -the PSF objects. The difference between the aperture and fit -magnitudes ({\em ApResid}) is a critical parameter for any PSF -modeling software which uses an analytical model to represent the -flux distribution of the objects in an image. +The metric used by PSPhot to assess the PSF model is the scatter in +the differences between the aperture and fit magnitudes for the PSF +objects. The difference between the aperture and fit magnitudes ({\em +ApResid}) is a critical parameter for any PSF modeling software which +uses an analytical model to represent the flux distribution of the +objects in an image. The important concept here is that an analytical model will always @@ -452,15 +535,21 @@ quality are dominant. The brighter is the object, the smaller is the error introduced by the large size of the aperture. However, the -number of very bright stars is limited in any image. - -Consider a typical bright object with a flux of (say) 40,000 counts in -an image of background 1000 counts per pixel, with FWHM of 4 pixels. -In principle, the flux of this object should be measurable with an -accuracy of roughly 0.57\% ($\frac{\sqrt{40000 + 1000 \times -12}}{40000}$). However, the measurement of the sky is limited at some -finite level by Poisson statistics. If we are required to use an -aperture of (say) 25 pixels in radius (eg, 5 arcseconds for an 0.2 -arcsec / pixel detector), and we have an annulus of twice this radius -to measure the local sky, then we will have an error of XXX. +number of very bright stars is limited in any image, and of course the +brighter stars are more likely to suffer from non-linearity or +saturation. + +\note{this discussion sucks: put in some more details of my point: + amplitude of systematic vs random sky errors} + +How important is this effect? Consider a typical bright object with a +flux of (say) 40,000 counts in an image of background 1000 counts per +pixel, with FWHM of 4 pixels. In principle, the flux of this object +should be measurable with an accuracy of roughly 0.57\% +($\frac{\sqrt{40000 + 1000 \times 12}}{40000}$). However, the +measurement of the sky is limited at some finite level by Poisson +statistics. If we are required to use an aperture of (say) 25 pixels +in radius (eg, 5 arcseconds for an 0.2 arcsec / pixel detector), and +we have an annulus of twice this radius to measure the local sky, then +we will have an error of XXX. \note{outline the variation of {\em ApResid} as a function of @@ -469,11 +558,16 @@ PSPhot measures the aperture correction ({\em ApResid}) for every PSF candidate object, then calculates the trend of this correction as a -function of the magnitude. This trend \note{write the correct form} -is fitted with a line. The resulting function can be used to -determine the effective aperture correction for an infinite flux -object and the average bias inherent in the sky measurement for the -image. The scatter of the PSF-candidate object measurements about -this trend is a measure of how well we can measure photometry from the -image by applying the specific PSF model. +function of the magnitude. This trend is fitted with a line. The +resulting function can be used to determine the effective aperture +correction for an infinite flux object and the average bias inherent +in the sky measurement for the image. The scatter of the +PSF-candidate object measurements about this trend is a measure of how +well we can measure photometry from the image by applying the specific +PSF model. The slope of this trend is a measure of the bias in the +local sky measurment for each object. In principal, the measured sky +levels could be modified by this bias. More generally, the measured +bias in a collection of images could be used to improve the model +fitting or sky fitting portion of the software the remove the bias +term. PSPhot allows a collection of PSF model functions to be tried on all @@ -481,5 +575,9 @@ ApResid scatter is measured. The PSF model function with the smallest value for the ApResid scatter is then used by PSPhot as the best PSF -model for this image. +model for this image. The number of models to be tested is specified +by the configuration keyword \code{PSF_MODEL_N}. The configuration +variables \code{PSF_MODEL_0}, \code{PSF_MODEL_1}, through +\code{PSF_MODEL_N - 1} specify the names of the models which should be +tested. \subsubsection{PSF Model applied to detected objects} @@ -561,4 +659,6 @@ \subsubsection{Types of Object / PSF models currently available} +\note{the discussion of the model types needs to be extended} + \begin{itemize} \item Pure elliptical Gaussian (GAUSS) @@ -608,5 +708,5 @@ test in this case}. -\subsection{faint sources} +\subsection{Faint Sources} \note{the following discussion is theoretical : it is not yet coded} @@ -660,4 +760,9 @@ constraints on the quality of the detection (no Chi-Square is measured, for example). + +\note{In the ideal case, if we were only interested in detecting PSFs, +and we had a good model for the PSF, we could optimally find the +sources by smoothing the image and the noise image with the PSF model. +\em write out the description of Nick's optimal PSF finding}. PSPhot allows the user to select between these three options for the @@ -732,3 +837,45 @@ \note{need to discuss failings and holes} +\section{User's Guide} + +\subsection{Configuration Parameters} + +\begin{verbatim} +FAINT_SN_LIM +FIT_MAX_CHI +FIT_MIN_SN +FIT_NSIGMA +FIT_PADDING +FIT_RADIUS +GAIN +GAL_MODEL +GAL_MOMENTS_RADIUS +INNER_RADIUS +INPUT +MASK +NOISE +NSUBSET +OUTER_RADIUS +OUTPUT +OUTPUT_MODE +PEAK_NSIGMA +PSF_MODEL_N +PSF_MOMENTS_RADIUS +PSF_SHAPE_NSIGMA +RDNOISE +SATURATE +SMOOTH_NSIGMA +SMOOTH_SIGMA +XMAX +XMIN +YMAX +YMIN +\end{verbatim} + +\subsection{Command-Line Arguments and Options} + +\subsection{Input \& Output Data Formats} + +\section{Sample Tests} + \end{document}