Index: trunk/doc/design/ippSSDD.tex =================================================================== --- trunk/doc/design/ippSSDD.tex (revision 8140) +++ trunk/doc/design/ippSSDD.tex (revision 8703) @@ -613,12 +613,13 @@ \paragraph{House keeping} -\subparagraph{Lock sweeping} In the event that a Storage Object operation fails to complete successfully -stale locks will have to be identified and removed from the IPP Pixel -Data Server Database. This should be done periodically by comparing -the entries in the Lock table to the list of active nodes maintained -by the IPP Controller. It should also happen as soon as possible -after a node goes offline (triggered by the IPP Controller marking a -node as offline?). A sweep must be /completed/ before an offline node -can be marked on-line. +\subparagraph{Lock sweeping} +In the event that a Storage Object operation fails to complete +successfully stale locks will have to be identified and removed from +the IPP Pixel Data Server Database. This should be done periodically +by comparing the entries in the Lock table to the list of active nodes +maintained by the IPP Controller. It should also happen as soon as +possible after a node goes offline (triggered by the IPP Controller +marking a node as offline?). A sweep must be /completed/ before an +offline node can be marked on-line. Once a node is determined to be offline all entries in the Lock table @@ -628,8 +629,9 @@ table. -\subparagraph{Consistency sweeping} Periodically the IPP Pixel Data Server meta-data and Storage Object will need -to be checked for sanity. This would be similar to running fsck on a -modern filesystem. Consistency sweeping should include Lock sweeping -and should be considered a super-set. +\subparagraph{Consistency sweeping} +Periodically the IPP Pixel Data Server meta-data and Storage Object +will need to be checked for sanity. This would be similar to running +fsck on a modern filesystem. Consistency sweeping should include Lock +sweeping and should be considered a super-set. \subsubsection{Nebulous Database} @@ -1491,4 +1493,7 @@ photometrically corrected flats (-grid option). +\tbd{fill out this discussion in the analysis section on the +astrometric and photometric reference catalog}. + \subsubsection{Uniphot : Zero Point Analysis} @@ -1496,5 +1501,5 @@ points for images and the spatial overlap information to determine a best set of image zero points which have a specific time scale for the -atmospheric stability. This analysis would be used after relative +atmospheric stability. This analysis is used after relative photometry has been determined for data in DVO. This analysis currently is defined to unify the zero points of a collection of @@ -1503,11 +1508,17 @@ photometry corrections for a collection of images distributed over a large range in space and time, but still with significant -overlap. distritions with subustanailaccount for the c - -\subsubsection{Global Astrometry Analysis} - -This operation uses the reference and image detections to determine an -optical distortion model for the camera and static astrometry model -components. The astrometry model includes: (1) field distortion +overlap. + +\tbd{fill out this discussion in the analysis section on the +astrometric and photometric reference catalog}. + +\subsubsection{relastro : Global Astrometry Analysis} + +This operation uses the reference and image detections to improve the +astrometric reference catalog. It determines an improved optical +distortion model for the camera and static astrometry model +components, and then applies the improved astrometric solutions to the +observations to yield high-quality astrometry for the average object +positions. The astrometry model includes: (1) field distortion introduced by the telescope optics, which is a smoothly-varying function of the field position relative to the center of the telescope @@ -1516,5 +1527,8 @@ along with chip-dependent plate-scale modifications needed to represent tilts or warps of the individual detectors relative to the -ideal flat focal plane. . +ideal flat focal plane. + +\tbd{fill out this discussion in the analysis section on the +astrometric and photometric reference catalog}. \subsubsection{DVO shell} @@ -2835,4 +2849,9 @@ to an error upstream in the processing). +\tbd{add discussion of the choices to be made in generating the + static sky image stacks: interpolation methods, selection of input + images by IQ, smoothing of input images by their PSF, weighting and + clipping of input pixels} + Object analysis of the static sky images is {\em not} a part of the Phase 4 analysis. This processing is envisioned to take place @@ -2840,4 +2859,7 @@ scheduled as a separate analysis task, probably run during the day at a time when the computing infrastructure is not under significant load. + +\tbd{add discussion of the multiple image analysis and object + analysis without the static sky (ie, on all input images at once)} \subsubsection{Magic and Phase 4 Modifications} @@ -3161,5 +3183,5 @@ parameter $\nu$, and a collection of annular aperture flux measurements, all of which are also measured for the P4$\Sigma$ -images. In addition, the galaxy-shape parameters $Gamma_1 \& +images. In addition, the galaxy-shape parameters $\Gamma_1 \& \Gamma_2$, along with the complete `polarization' terms are measured, as well as a collection of annular aperture flux and variance @@ -3173,5 +3195,21 @@ per second), it is only necessary to process the complete sky in a year, or an average rate of $\sim$2 Mpix per second, or $< 1$\% of the -object analysis in the other analysis stages. +object analysis in the other analysis stages. These operations are +all functions which will be performed within the PSPhot program using +recipe options. + +\subsection{Astrometric and Photometric Reference Catalog} + +The IPP is responsible for generating the Astrometric and Photometric +(AP) Reference Catalog. The IPP provides several tools for performing +this analysis. The DVO programs \code{relphot}, \code{uniphot}, and +\code{relastro} perform most of the operations required to generate +the AP Reference Catalog. These include the determination of the +image zero-points, the identification of objects with significant +variability, the detection of individual outlier measurements, the +detection of objects with substantial astrometric error, the analysis +of parallax and proper-motions, etc. In addition, the DVO shell +program will be used to generate the color transformations from the +observed data and to perform other tests of the catalog quality. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -3191,6 +3229,5 @@ functions in the operational system, the IPP will make use of Perl as the scripting language to provide the required flow-control to tie the -modules together. \tbd{note that we use C only, not perl for -scripting}. +modules together. This approach satisfies the requirement that complicated low-level @@ -3551,9 +3588,9 @@ from the selected reference catalog. The observed sources are matched to the reference sources, using either a two-point grid search or -optionally a \tbd{triangle match}. Once an approximate match is -found, a linear fit between detector coordinates an projected -celestial coordinates is attempted. The projected coordinate system -may optionally make use of the default telescope distortion model, if -it is known. The radius of the match between observed and reference +optionally a triangle match. Once an approximate match is found, a +linear fit between detector coordinates an projected celestial +coordinates is attempted. The projected coordinate system may +optionally make use of the default telescope distortion model, if it +is known. The radius of the match between observed and reference sources is reduced to improve the statistics of the match. This anaysis mode is used in the Phase 2 processing. @@ -3610,6 +3647,6 @@ \subsection{poisub} -Poisub is the image difference analysis program. \tbd{Paul: please - flesh this out!}. +Poisub is the image difference analysis program. \tbd{finish this +discussion}. \subsection{stac} @@ -3618,5 +3655,5 @@ same region of the sky. It consists of two major stages: the warping stage and the image combination stage with robust outlier rejection. -\tbd{Paul: flesh this out!} +\tbd{update / finish this discussion} \subsection{Command Sequences} @@ -4035,4 +4072,7 @@ \subsection{IPP Pipelines Overview} +\tbd{add the use of Q/A measurements from the IPP CDR Response +document} + The IPP as a whole performs all of the image analysis functions required by the Pan-STARRS telescopes, including images from the full @@ -4380,10 +4420,10 @@ header or new exp table?), the exposure is added to the `raw exposure' table for images of that type. The allowed types are `detrend', (all -bias, dark, flat images), `object', `focus'(??), etc. (** The -different tables represent different analysis modes. This process -also adds an entry to the exp ID / image file match **). This process -also adds all science (OBJECT) exposures to the P1 exposure table (for -mosaic data) or the P2 chip table (for single detector data). These -tables are used to trigger the Phase 1 and Phase 2 analysis stages. +bias, dark, flat images), `object', `focus'(??), etc. The different +tables represent different analysis modes. This process also adds an +entry to the exp ID / image file match. This process also adds all +science (OBJECT) exposures to the P1 exposure table (for mosaic data) +or the P2 chip table (for single detector data). These tables are +used to trigger the Phase 1 and Phase 2 analysis stages. \subsection{Phase 1} @@ -4559,9 +4599,9 @@ rules. -\note{Phase 4 run can be defined by selecting an observation group, a +\tbd{Phase 4 run can be defined by selecting an observation group, a set of exposures, or a set of rules related to a spatial region (eg, region, time range, and filter}. -\note{Phase 4 discussion (and diagram) needs more work} +\tbd{Phase 4 discussion (and diagram) needs more work} \subsection{Analysis Version and Recipes}