Changeset 40715

Timestamp:

May 6, 2019, 10:05:31 AM (7 years ago)

Author:

eugene

Message:

tweak table positions

File:

• trunk/doc/release.2015/ps1.datasystem/datasystem.tex (modified) (5 diffs)

trunk/doc/release.2015/ps1.datasystem/datasystem.tex

@@ -1100 +1100 @@
 eventual public released.
 
-When a \ippstage{diff} processing is defined, an entry is added to the
-\ippdbtable{diffRun} table, and the appropriate input images are added
-to the \ippdbtable{diffInputSkyfile} table, with one entry for each
-skycell that is covered by the images.  For a \ippstage{diff}
-generated from two \ippstage{warp} stage products, the input images
-have their \ippdbcolumn{warp_id} values recorded in the
-\ippdbcolumn{warp1} and \ippdbcolumn{warp2} for each skycell that
-overlaps.  If two \ippstage{stack} stages are to be used in the
-difference, their \ippdbcolumn{stack_id} entries are recorded in the
-\ippdbcolumn{stack1} and \ippdbcolumn{stack2} fields.  As each
-\ippstage{stack} only covers a single skycell, the \ippstage{diff} is
-usually defined indirectly, using other information from the
-\ippdbtable{stackRun} table to select appropriate
-\ippdbcolumn{stack_id} values.  Similarly, \ippstage{diff} processing
-is defined for the mixed case by creating entries that populate one of
-\ippdbcolumn{warp1} and \ippdbcolumn{stack1} and populating one of
-\ippdbcolumn{warp2} and \ippdbcolumn{stack2}.  In all cases, the
-minuend of the subtraction to be performed is the ``1'' entry, and the
-subtrahend is the ``2'' entry.
-
-Jobs are created based on the entries of
-\ippdbtable{diffInputSkyfile}, with the appropriate images and
-catalogs passed to the \ippprog{ppSub} program.  This does the
-subtraction, as well as the photometry of any sources detected in the
-\ippstage{diff} image.  Sources may be detected as a positive source
-(flux in the minuend is higher than the subtrahend) or as a negative
-source (flux in the subtrahend is higher).  The algorithm used for PSF
-matching is described in Paper III.  Upon completion of these
-jobs, statistics about the processing are written to an entry in the
-\ippdbtable{diffSkyfile} table.  An \ippmisc{advance} checks for the
-completion of all of the components listed in
-\ippdbtable{diffInputSkyfile}, and marks the \ippdbtable{diffRun}
-entry as such.
-
-\section{Post-Processing : Database Ingest and Calibration}
-\label{sec:postprocessing}
-
-\subsection{DVO}
-\label{sec:DVO}
-
-\subsubsection{Overview}
-
-% intro
-The Pan-STARRS IPP uses an internal database system, distinct from the
-publicly visible database system, to determine the association
-between multiple detections of the same astronomical object and as
-part of the astrometric and photometric calibration process.  This
-database system, called the ``Desktop Virtual Observatory'' (DVO) was
-developed originally for the LONEOS project
-\citep{1995DPS....27.0110B}, and used as part of the CFHT Elixir
-system \citep{2004PASP..116..449M}.  The capabilities of this
-databasing system have been somewhat expanded for the Pan-STARRS
-context.
-
-% overview
-DVO tracks three main classes of information: 1) average properties of
-astronomical objects; 2) measurements of those objects (from which the
-average properties are derived); 3) properties of the images which
-provided some or all of the measurements.  In addition, certain
-metadata tables define general features of the database.
-Table~\ref{tab:DVO_schema} lists the full collection of database
-tables used by DVO.
-
-In the most basic implementation, a collection of measurements for
-detections from a set of images is loaded into DVO along with the
-metadata describing the images.  The latter includes properties such
-as the exposure time, airmass, filter, time \& date of the exposure,
-etc.  Critically, the image metadata includes an astrometric
-transformation relating the detection coordinate on the image to the
-coordinate on the sky.  As the collection of measurements are loaded
-into DVO, the software constructs astronomical objects based on those
-detections.  If images overlap, multiple observations of the same
-astronomical object are grouped together.  Thus, a single DVO database
-will contain a one-to-many relationship between the images and the
-measurements and a many-to-one relationship between the measurements
-and the derived astronomical objects.
-
-\subsubsection{DVO Schema}
-
-\begin{table*}[hb]
+\begin{table*}
 \begin{center}
 \caption{DVO Database Tables\label{tab:DVO_schema}}
@@ -1202 +1123 @@
 \end{center}
 \end{table*}
+
+When a \ippstage{diff} processing is defined, an entry is added to the
+\ippdbtable{diffRun} table, and the appropriate input images are added
+to the \ippdbtable{diffInputSkyfile} table, with one entry for each
+skycell that is covered by the images.  For a \ippstage{diff}
+generated from two \ippstage{warp} stage products, the input images
+have their \ippdbcolumn{warp_id} values recorded in the
+\ippdbcolumn{warp1} and \ippdbcolumn{warp2} for each skycell that
+overlaps.  If two \ippstage{stack} stages are to be used in the
+difference, their \ippdbcolumn{stack_id} entries are recorded in the
+\ippdbcolumn{stack1} and \ippdbcolumn{stack2} fields.  As each
+\ippstage{stack} only covers a single skycell, the \ippstage{diff} is
+usually defined indirectly, using other information from the
+\ippdbtable{stackRun} table to select appropriate
+\ippdbcolumn{stack_id} values.  Similarly, \ippstage{diff} processing
+is defined for the mixed case by creating entries that populate one of
+\ippdbcolumn{warp1} and \ippdbcolumn{stack1} and populating one of
+\ippdbcolumn{warp2} and \ippdbcolumn{stack2}.  In all cases, the
+minuend of the subtraction to be performed is the ``1'' entry, and the
+subtrahend is the ``2'' entry.
+
+Jobs are created based on the entries of
+\ippdbtable{diffInputSkyfile}, with the appropriate images and
+catalogs passed to the \ippprog{ppSub} program.  This does the
+subtraction, as well as the photometry of any sources detected in the
+\ippstage{diff} image.  Sources may be detected as a positive source
+(flux in the minuend is higher than the subtrahend) or as a negative
+source (flux in the subtrahend is higher).  The algorithm used for PSF
+matching is described in Paper III.  Upon completion of these
+jobs, statistics about the processing are written to an entry in the
+\ippdbtable{diffSkyfile} table.  An \ippmisc{advance} checks for the
+completion of all of the components listed in
+\ippdbtable{diffInputSkyfile}, and marks the \ippdbtable{diffRun}
+entry as such.
+
+\section{Database Ingest and Calibration}
+\label{sec:postprocessing}
+
+\subsection{DVO}
+\label{sec:DVO}
+
+\subsubsection{Overview}
+
+% intro
+The Pan-STARRS IPP uses an internal database system, distinct from the
+publicly visible database system, to determine the association
+between multiple detections of the same astronomical object and as
+part of the astrometric and photometric calibration process.  This
+database system, called the ``Desktop Virtual Observatory'' (DVO) was
+developed originally for the LONEOS project
+\citep{1995DPS....27.0110B}, and used as part of the CFHT Elixir
+system \citep{2004PASP..116..449M}.  The capabilities of this
+databasing system have been somewhat expanded for the Pan-STARRS
+context.
+
+% overview
+DVO tracks three main classes of information: 1) average properties of
+astronomical objects; 2) measurements of those objects (from which the
+average properties are derived); 3) properties of the images which
+provided some or all of the measurements.  In addition, certain
+metadata tables define general features of the database.
+Table~\ref{tab:DVO_schema} lists the full collection of database
+tables used by DVO.
+
+In the most basic implementation, a collection of measurements for
+detections from a set of images is loaded into DVO along with the
+metadata describing the images.  The latter includes properties such
+as the exposure time, airmass, filter, time \& date of the exposure,
+etc.  Critically, the image metadata includes an astrometric
+transformation relating the detection coordinate on the image to the
+coordinate on the sky.  As the collection of measurements are loaded
+into DVO, the software constructs astronomical objects based on those
+detections.  If images overlap, multiple observations of the same
+astronomical object are grouped together.  Thus, a single DVO database
+will contain a one-to-many relationship between the images and the
+measurements and a many-to-one relationship between the measurements
+and the derived astronomical objects.
+
+\subsubsection{DVO Schema}
 
 \paragraph{Photcodes}
@@ -2594 +2594 @@
 the Moab job control file was constructed.
 
+\begin{table}
+\caption{\label{tab:SC_processing_parameters} Cost values for remote processing}
+\begin{center}
+\begin{tabular}{lcc}
+\hline
+\hline
+{\bf IPP Stage} & {\bf $t_\mathrm{task}$ (s)} & {\bf $S_\mathrm{task}$} \\
+\hline
+  \ippstage{chip} & 150 & 2 \\
+  \ippstage{camera} & 1700 & 2 \\
+  \ippstage{warp} & 110 & 2 \\
+  \ippstage{stack} & 1500 & 6 \\
+  \ippstage{staticsky} & 7200 & 6 \\
+%  \ippstage{diff} & 300 & 2 \\
+  \ippstage{fullforce} & 300 & 2 \\
+\hline
+\end{tabular}
+\end{center}
+\end{table}
+
 The control file contains the resource requests for the job, as well
 as the commands to be executed to complete it.  The resource request
@@ -2612 +2632 @@
 values used for the various IPP processing stages.
 
-\begin{table*}
-\caption{\label{tab:SC_processing_parameters} Cost values for remote processing}
-\begin{center}
-\begin{tabular}{lcc}
-\hline
-\hline
-{\bf IPP Stage} & {\bf $t_\mathrm{task}$ (s)} & {\bf $S_\mathrm{task}$} \\
-\hline
-  \ippstage{chip} & 150 & 2 \\
-  \ippstage{camera} & 1700 & 2 \\
-  \ippstage{warp} & 110 & 2 \\
-  \ippstage{stack} & 1500 & 6 \\
-  \ippstage{staticsky} & 7200 & 6 \\
-%  \ippstage{diff} & 300 & 2 \\
-  \ippstage{fullforce} & 300 & 2 \\
-\hline
-\end{tabular}
-\end{center}
-\end{table*}
-
 Once the preparation for the job is complete, the input and output
 file lists, the task list, and the job control file are transferred
@@ -2690 +2690 @@
 cluster as other users started to utilize the system, with 168,685 /
 994,890 runs processed there.
-
-%% add a discussion of lessons-learned?
 
 \section{Conclusion}