Index: trunk/doc/design/ippSSDD.tex =================================================================== --- trunk/doc/design/ippSSDD.tex (revision 6167) +++ trunk/doc/design/ippSSDD.tex (revision 6434) @@ -1,3 +1,2 @@ -%%% $Id: ippSSDD.tex,v 1.7 2006-01-22 09:54:47 eugene Exp $ \documentclass[panstarrs]{panstarrs} @@ -11,6 +10,6 @@ \project{Pan-STARRS Image Processing Pipeline} \organization{Institute for Astronomy} -\version{00} -\docnumber{PSDC-430-011} +\version{DR.02} +\docnumber{PSDC-430-014} % allow paragraphs to be listed in TOC for now @@ -23,10 +22,6 @@ \RevisionsStart % version Date Description -DR.01 & 2004.01.01 & First draft \\ \hline -DR.02 & 2004.03.05 & Second draft \\ \hline -DR.03 & 2004.03.25 & Section reorganization \\ \hline -DR.04 & 2004.04.13 & Most sections fleshed out \\ \hline -DR.05 & 2004.04.29 & Reorganization for consistency \\ \hline -DR.06 & 2004.10.21 & Major revision in prep of PDR \\ \hline +DR.01 & 2006.01.19 & First draft release to CDR committee \\ \hline +DR.02 & 2006.01.22 & Second draft release to CDR committee \\ \hline \RevisionsEnd @@ -96,6 +91,6 @@ achieve the requirements specified by the Pan-STARRS PS-1 IPP Software Requirements Specification (SRS; PSDC-430-005). The requirements flow -begun in the SGS and SCD and continued in the SRS is used to guide the -design presented here. +which began in the SGS and Pan-STARRS system SCD, and continued in the +IPP SRS and the IPP SCD, is used to guide the design presented here. \subsection{Requirements Definitions} @@ -135,4 +130,5 @@ PSDC-430-007 & Pan-STARRS IPP PSLib Supplementary Design Requirements Specification \\ \hline PSDC-430-010 & Pan-STARRS IPP Perl Code Conventions \\ \hline +PSDC-430-011 & Pan-STARRS IPP System Concept Definition \\ \hline PSDC-430-012 & Pan-STARRS IPP Modules Supplementary Design Requirements Specification \\ \hline PSDC-430-014 & Pan-STARRS IPP PS-1 Cluster Support \\ \hline @@ -140,6 +136,4 @@ Posix Standard & Open Group Based Specifications Issue 6, IEEE Std 1003.1, 2003 \\ \DocumentsEnd - -\tbd{add the other subsystem SDDs} \section{Subsystem Overview} @@ -232,6 +226,4 @@ \subsection{System Design Decisions} -\tbd{add discussion of the AP survey and other survey projects} - Since Pan-STARRS is a survey project, all data from the telescopes will be uniformly analyzed by the Pan-STARRS Image Processing Pipeline @@ -263,22 +255,34 @@ elements. -\subsection{Analysis Tasks and Stages} - -\tbd{this discussion is confusing, and even more so in the context of - the PanTasks definition of a 'task'}. - -Specific programs are required to perform the processing steps listed -above. These can be divided into well-defined analysis stages, each -of which operates on a particular unit of data, such as a single OTA -image or a collection of astronomical objects. Analysis tasks -representing the different analysis stages are performed on the IPP -computer cluster. Note the distinction between the generic analysis -{\em stage} and a specific analysis {\em task}. An analysis stage -represents a type of analysis which is performed, such as the basic -image calibration and object detection analysis. An analysis task is -a particular realization of an analysis stage, e.g., the analysis of -OTA number 61 from exposure 654321 to produce a specific set of output -data products. The analysis stages are discussed in detail in -Section~\ref{sec:AnalysisStages}. +\subsection{Analysis Stages, Programs, and Libraries} + +The processing steps listed above can be divided into well-defined +analysis stages, each of which operates on a particular unit of data, +such as a single OTA image or a collection of astronomical objects. +The different analysis stages defined by the IPP are discussed in +Sections~\ref{sec:AnalysisStages} - \ref{sec:MiscStages}. The +analysis defined by these analysis stages is further divided into +specific programs which perform the particular operation. Some +analysis stages are primarily defined by a single programs; others +require multiple programs and substages. It is also possible for a +single program to be involved in multiple analysis stages. The major +IPP analysis programs are discussed in some detail in +Section~\ref{sec:AnalysisPrograms}. Other important programs in the +collection called \code{ippTools} are used to define the analysis +stages and their interrelationships (Section~\ref{sec:ipptools}). The +IPP software hierarchy is further divided into \code{psModules} +(representing complex analysis operations, often astronomical in +nature) and \code{psLib}, the data and analysis foundation library +(Section~\ref{sec:SoftwareHierarchy}). + +Analysis tasks representing the different analysis stages are +performed on the IPP computer cluster. Note the distinction between +the generic analysis {\em stage} and a specific analysis {\em task}. +An analysis stage represents a type of analysis which is performed, +such as the basic image calibration and object detection analysis. An +analysis task is a particular realization of an analysis stage, e.g., +the analysis of OTA number 61 from exposure 654321 to produce a +specific set of output data products. The analysis stages are +discussed in detail in Section~\ref{sec:AnalysisStages}. A particular stage may process individual images, collections of @@ -507,5 +511,5 @@ monitor the age of images and delete them on some schedule. This functionality currently resides in the IPP Scheduler -(Section~\ref{sec:scheduler}). +(Section~\ref{sec:pantasks}). As shown in Figure~\ref{fig:Nebulous}, Nebulous consists of the @@ -521,6 +525,6 @@ \begin{center} \resizebox{4.5in}{!}{\includegraphics{pics/ImageServer}} -\caption{The components of the IPP Image Server.} -\label{fig:ImageServer} +\caption{The components of Nebulous, the IPP Image/File Server.} +\label{fig:Nebulous} \end{center} \end{figure} @@ -1106,5 +1110,5 @@ detection data, the {\tt Regions} table allows for multiple computers to serve the database tables. The region file specifies the machine -which stores the specific table. Figure~\ref{fig:APDBRegions} +which stores the specific table. Figure~\ref{fig:DVOskyregions} illustrates schematically the subdivision of the sky and the association between different levels of the hierarchy with different @@ -1582,4 +1586,5 @@ \subsection{PanTasks : the IPP Scheduler} +\label{sec:pantasks} PanTasks is the IPP tool which manages the sequencing of data analysis @@ -2148,5 +2153,5 @@ \begin{center} \includegraphics[scale=0.85,angle=-90]{pics/pantasks.05.ps} -\caption{\label{queues} pcontrol job states. Transitions labeled Ux +\caption{\label{JobStates} pcontrol job states. Transitions labeled Ux are issued by the pcontrol user (including PanTasks). Transitions labeled Px are initiated by pcontrol. Transitions labeled Tx are @@ -2250,5 +2255,5 @@ \begin{center} \includegraphics[scale=0.85,angle=-90]{pics/pantasks.07.ps} -\caption{\label{queues} PanTasks queues and MDDB tables} +\caption{\label{Pclient} Pclient monitor loop.} \end{center} \end{figure} @@ -3126,5 +3131,6 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{System Design : Miscellaneous Tasks} +\section{System Design : Miscellaneous Analysis Stages} +\label{sec:MiscStages} This section discusses additional operations which are performed by @@ -3171,4 +3177,5 @@ \section{Software Hierarchy} +\label{sec:SoftwareHierarchy} In order to facilitate testing and development, and to encourage @@ -3229,6 +3236,4 @@ \subsection{IPP Analysis Programs} -\tbd{clean this up} - The major IPP processing tasks are organized into stages, which consist of multiple modules. Each stage represents a collection of @@ -3240,5 +3245,6 @@ images from multiple telescopes and search for transients). -\section{Top-Level IPP Analysis Routines} +\section{Top-Level IPP Analysis Programs} +\label{sec:AnalysisPrograms} The IPP uses a handful of high-level analysis routines which perform @@ -3904,4 +3910,5 @@ \section{IPPTools} +\label{sec:ipptools} PanTasks is the IPP tool which manages the sequencing of data analysis @@ -5401,5 +5408,5 @@ \section{AP Database Table Contents} -\label{sec:APDBTableContents} +\label{sec:DVOTableContents} \begin{table}[bh] @@ -5700,5 +5707,5 @@ \begin{table}[bh] \begin{center} -\caption{Database Hosts\label{tab:APDBHosts}} +\caption{Database Hosts\label{tab:DVOHosts}} \begin{tabular}{lll} \hline