Changeset 40009 for trunk

Timestamp:

Apr 4, 2017, 12:28:13 PM (9 years ago)

Author:

eugene

Message:

updates to data system

File:

• trunk/doc/release.2015/ps1.datasystem/datasystem.tex (modified) (4 diffs)

trunk/doc/release.2015/ps1.datasystem/datasystem.tex

@@ -177 +177 @@
 
 \section{Overview of Pan-STARRS Data Processing}
+\label{sec:overview}
 
 The Pan-STARRS Data Analysis system contains many features to support
@@ -291 +292 @@
 
 \subsection{Processing Database}
-\label{sec:database}
+\label{sec:processing.database}
 
 A critical element in the Pan-STARRS IPP infrastructure is the
@@ -1022 +1023 @@
 
 \section{Post-Processing : Database Ingest \& Calibration}
+\label{sec:postprocessing}
 
 \subsection{DVO}
@@ -1395 +1397 @@
 \subsection{Pantasks \& Parallel Processing}
 \label{sec:pantasks}
+
+Sections~\ref{sec:subsystesm} \& \ref{sec:postprocessing} describe the
+analysis steps which take place in the Pan-STARRS data analysis
+systems.  Individually, these steps appear as commands which could be
+run by a user within the UNIX environment of the PS1 data system.  The
+processing database (Section~\ref{sec:processing.database}) provides
+the logical links to relate the results of one analysis stage to
+another.  In order to make a complete system which can run
+automatically, it is necessary to have a process which can use the
+contents of the processing database to generate the commands
+corresponding to the analysis stages.  This system needs to (1)
+regularly examine the database to find items from stages which are
+ready to be processed, (2) to have rules which define how to construct
+the appropriate commands, (3) to cause those commands to be executed
+within the processing system, (4) to monitor the active processing
+jobs for completion, and (5) to check on the results of those
+commands and update the processing database as needed.  Within the
+Pan-STARRS IPP, the top-level management of these operations is
+performed with the program called \ippprog{pantasks}.  
+
+The core capability of the \ippprog{pantasks} program is to take a
+collection of ``tasks'' which describe the concept of a command which
+might be run and to regularly generate new commands based on that
+concept.  These commands may the be run by the pantasks program
+directly, or they may be passed to another program which 
+
+* task vs job
 
 \note{this section needs to be re-written : pclient vs pcontrol vs pantasks}