Index: trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 37892) +++ trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 37893) @@ -105,14 +105,10 @@ \section{INTRODUCTION}\label{sec:intro} +\note{more PS1 background} + The Pan-STARRS Image Processing Pipeline is responsible for the basic analysis of images from the Pan-STARRS telescopes Gigapixel Camera. -The overall goals and requirements of the Image Processing Pipeline -are described in the IPP System/Subsystem Design Description (SSDD; -PSDC-430-XXX) and the IPP System Requirements Specification (SRS; -PSDC-430-XXX). Among the Pan-STARRS project survey goals is a -repeated all-sky survey in 5 filters, {\it grizy}, beginning with a -pre-survey with the prototype telescope PS-1. The photometric and -astrometric precision goals for the all-sky surveys, as well as the -other survey components, are quite stringent: +The photometric and astrometric precision goals for the all-sky +surveys, as well as the other survey components, are quite stringent: \begin{itemize} @@ -128,13 +124,12 @@ An additional constraint on the Pan-STARRS system comes from the high -data rate. The prototype telescope alone is expected to produce -typically $\sim 700$ GB per night of imaging data. These images will -not be limited to high galactic latitudes, so large numbers of -measurable stars can be expected in much of the data. The combination -of the high precision goals of the astrometric and photometric -measurements and the high data rate (and a finite computing budget) -mean that the process of detecting, classifying, and measuring the -astronomical objects in the image data stream will be a significant -challenge. +data rate. PS1 produces typically $\sim 700$ GB per night of imaging +data. These images range from high galactic latitudes to the Galactic +Bulge, so large numbers of measurable stars can be expected in much of +the data. The combination of the high precision goals of the +astrometric and photometric measurements and the high data rate (and a +finite computing budget) mean that the process of detecting, +classifying, and measuring the astronomical objects in the image data +stream in a timely fashion are a significant challenge. In order to achieve these ambitious goals, the object detection, @@ -186,25 +181,23 @@ \end{itemize} -\note{discussion of these packages is insufficient: flesh out - discussion and add in the references.} - -\note{Add discussion of the lessons learned from experience with previous - analysis programs} - -The Pan-STARRS IPP team decided that none of the existing packages met -all of their needs, particularly given the very challenging goals of -the project. We decided to redesign the photometry analysis from -scratch, using the lessons learned from the existing photometry -systems. In the process, the object analysis software would be -written using the data analysis C-code library written for the IPP, -\code{psLib}, and the components of the photometry code would be -integrated into the IPP's mid-level astronomy data analysis toolkit -called \code{psModules}. The result is 'PSPhot', which can be used -either as a stand-alone C program, or as one of the high-level IPP -components of \code{psModules}, available to programmers either via a -C interface or through a SWIG interface in Perl (or potentially -Python). +\note{re-phrase this:} The Pan-STARRS IPP team decided that none of +the existing packages met all of their needs, particularly given the +very challenging goals of the project. We decided to redesign the +photometry analysis from scratch, using the lessons learned from the +existing photometry systems. In the process, the object analysis +software would be written using the data analysis C-code library +written for the IPP, \code{psLib}, and the components of the +photometry code would be integrated into the IPP's mid-level astronomy +data analysis toolkit called \code{psModules}. The result is +'PSPhot', which can be used either as a stand-alone C program, or as +callable set of functions. \note{discuss the psphot program varients} + +\begin{verbatim} +Other Varients: +* psphotStack -- 5 filter simultaneous fitting +* psphotFullForce +\end{verbatim} \section{PSPhot Design Goals} @@ -244,18 +237,4 @@ (since a single data value is used for X or Y). For the $4800^2$ GPC chips, this yields a limit of about 0.25 milliarcsecond.} - -\item {\bf processing time of 45 seconds} This requirement depends - strongly on the hardware organization, the amount of time spent on - other analysis steps, the density of stars per image, and the depth - for a given type of image. For the sources at the faint limit (eg, - $5\sigma$), the average density of sources is expected to be roughly - $3\times10^5$ per square degree, while sources at the 20 $\sigma$ - level may have densities of $\sim 5\times10^4$ per square degree. - Allowing 30 seconds for the PSPhot portion of the analysis, of which - 15 is used for careful analysis of the brighter sources, 10 seconds - is used for PSF modeling and other overheads, and the remaining 5 - seconds is used for the PSF fitting of the faintest source implies - that the detailed modelling may take roughly 3msec per source, and - the basic PSF fitting may be allowed 150 usec per source. \end{itemize} @@ -325,15 +304,9 @@ \end{itemize} -Note that a given run of PSPhot allows the user to perform many of -these stages only if needed. For example, the PSF model may already be -available from external information, in which case the PSF modeling -stage can be skipped. Or, when used as a library function, the image -may have already been loaded and the mask and weight images -constructed. In some implementations, it may be possible to skip the -initial object detection stage because only supplied sources are -measured. These are only some of the possible configurations. The -use of these different configurations depends on the source of the -image, the desired detail and speed of the processing, and the level -of accuracy desired from the analysis. +PSPhot is highly configurable. Users may choose via the configuration +system which of the above analyses are performed. This may be useful +for testing, but may also allow for specialized use cases. For +example, the PSF model may already be available from external +information, in which case the PSF modeling stage can be skipped. \subsection{Image Preparation} @@ -341,28 +314,25 @@ The first step is to prepare the image for detection of the astronomical objects. We need three separate images: the measured -flux, the corresponding variance image, and a mask defining which -pixels are valid and which should be ignored. For the stand-alone -program, the input flux image is a required program argument. When it -is loaded, it is converted by default to 32-bit floating point -representation. In the function-call form of PSPhot, the image must -be supplied by the user in 32-bit floating point format. The noise -and mask images may either be provided by the user, or they may be -automatically generated from the input image, based on -configuration-defined values for the image gain, read-noise, -saturation, and so forth. For the function-call form of the program, -the flux image is provided in the API, and references to the mask and -noise are provided in the configuration information. As in the -stand-alone C-program, the noise and mask may be constructed -automatically by PSPhot. +flux (signal image), the corresponding variance image, and a mask +defining which pixels are valid and which should be ignored. The +signal and variance images are represented internally as 32-bit +floating point values. The noise and mask images may either +be provided by the user, or they may be automatically generated from +the input image, based on configuration-defined values for the image +gain, read-noise, saturation, and so forth. For the function-call +form of the program, the flux image is provided in the API, and +references to the mask and noise are provided in the configuration +information. As in the stand-alone C-program, the noise and mask may +be constructed automatically by PSPhot. \note{describe the use of the covariance image} \note{describe the difference between 'bad' and 'suspect' pixels} -For the mask, we use a 16-bit image in which a value of 0 represents a -valid pixel. We use each of the 16 bits to define different reasons a -pixel should be ignored. This allows us to optionally respect or -ignore the mask depending on the circumstance. For example, in some -cases, we ignore saturated pixels completely while in other -circumstances, it may be useful to know the flux value of the +The mask is represented as 16-bit integer image in which a value of 0 +represents a valid pixel. Each of the 16 bits define different +reasons a pixel should be ignored. This allows us to optionally +respect or ignore the mask depending on the circumstance. For +example, in some cases, we ignore saturated pixels completely while in +other circumstances, it may be useful to know the flux value of the saturated pixel. In addition, the mask pixels are used to define the pixels available during a model fit, and which should be ignored for @@ -372,10 +342,8 @@ saturated (configuration keyword: \code{SATURATE}). 2) Pixels which are below a user-defined value are considered unresponsive and masked -as dead. 3) Pixels which lie outside of a user-defined window are -considered non-data pixels (eg, overscan) and are marked as invalid. -The valid window is defined by the configuration variables +as dead. 3) Pixels which lie outside of a user-defined coordinate +window are considered non-data pixels (eg, overscan) and are marked as +invalid. The valid window is defined by the configuration variables \code{XMIN}, \code{XMAX}, \code{YMIN}, \code{YMAX}. - -\note{discuss the mask.config file, in which the mask meanings are assigned to bit values} The noise image, if not supplied is constructed by default from the @@ -384,9 +352,10 @@ each pixel. In this case, the image is assumed to represent the readout from a single detector, with well-defined gain and read noise -characteristics. In some obvious cases, this assumption will not be -valid. For example, if the input flux image is the result of an image -stack with significantly variable number of input measurements per -pixel, it will be necessary to supply a noise image which accurately -represents the noise as a function of position in the image. +characteristics. This assumption is not always valid. For example, +if the input flux image is the result of an image stack with a +variable number of input measurements per pixel (due to masking and +dithering), it will be necessary to supply a noise image which +accurately represents the noise as a function of position in the +image. \subsection{Initial Object Detection}