Index: /trunk/doc/design/design.tex
===================================================================
--- /trunk/doc/design/design.tex	(revision 547)
+++ /trunk/doc/design/design.tex	(revision 548)
@@ -1,3 +1,3 @@
-%%% $Id: design.tex,v 1.8 2004-04-23 04:06:00 price Exp $
+%%% $Id: design.tex,v 1.9 2004-04-29 21:30:37 price Exp $
 \documentclass[panstarrs]{panstarrs}
 
@@ -35,7 +35,14 @@
 \pagenumbering{arabic}
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{Scope}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Identification}
@@ -46,4 +53,8 @@
 Pan-STARRS 1 (PS-1), the initial demonstration telescope to be
 constructed on Haleakala by Jan 2006.  
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{System Overview}
@@ -60,4 +71,8 @@
 roughly 2 years.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{Document Overview}
 
@@ -70,9 +85,14 @@
 type with surrounding square brackets}.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \section{Referenced Documents}
 
 This section lists documents referred to by this specification.\\
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \DocumentsInternalSection
@@ -85,19 +105,22 @@
 \DocumentsEnd
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{System Design Decisions}
 
-Since \PS{} is a survey project, all data from the telescopes
-will be uniformly analysed by the \PS{} Image Processing Pipeline
-(IPP) and the appropriate resulting data products made available to
-internal and external science analysis systems as they become
-available.  The processing performed by the IPP on the science images
-will consist of detrending and object detection for the individual
-images, combination of multiple overlapping images and further object
-detection, subtraction of a reference (static-sky) image and detection
-of residual objects, update of the static sky images, and detailed
-object analysis of the static sky images.  In addition, the IPP will
-produce improved astrometric and photometric reference catalogs on an
+Since \PS{} is a survey project, all data from the telescopes will be
+uniformly analysed by the \PS{} Image Processing Pipeline (IPP) and
+the appropriate resulting data products made available to internal and
+external science analysis systems as they become available.  The
+processing performed by the IPP on the science images will consist of
+detrending and object detection for the individual images, combination
+of multiple overlapping images and further object detection,
+subtraction of a reference (static-sky) image and detection of
+residual objects, update of the static sky images, and detailed object
+analysis of the static sky images.  In addition, the IPP will produce
+improved astrometric and photometric reference catalogs on an
 occasional basis as needed.  The output data products from the IPP
 consist of the calibration images, reduced images from the individual
@@ -105,21 +128,25 @@
 object photometry, and reference astrometry and photometry.
 
-The IPP interacts closely with other \PS{} systems responsible
-for other aspects of the \PS{} operation, including the summit
-systems (OATS), the science object database, the Moving Object
-Processing System (MOPS), and potentially other client science
-pipelines.
+The IPP interacts closely with other \PS{} systems responsible for
+other aspects of the \PS{} operation, including the summit systems
+(OATS), the science object database, the Moving Object Processing
+System (MOPS), and potentially other client science pipelines.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{System Overview}
 
-The \PS{} Image Processing Pipeline (IPP) consists of a
-collection of computer hardware and software organized to perform the
-tasks required to process images from the \PS{} telescopes.  The
-primary goal of the IPP is to process the science images from the
-\PS{} telescopes and make the results available to other systems
-within \PS{}.  To achieve this goal, the IPP must also perform
-other analysis functions to generate the calibrations needed in the
-science image processing and to occasionally use the derived data to
-generate improved astrometric and photometric reference catalogs.
+The \PS{} Image Processing Pipeline (IPP) consists of a collection of
+computer hardware and software organized to perform the tasks required
+to process images from the \PS{} telescopes.  The primary goal of the
+IPP is to process the science images from the \PS{} telescopes and
+make the results available to other systems within \PS{}.  To achieve
+this goal, the IPP must also perform other analysis functions to
+generate the calibrations needed in the science image processing and
+to occasionally use the derived data to generate improved astrometric
+and photometric reference catalogs.
 
 In order to meet these broad goals, the IPP must have the following
@@ -130,6 +157,6 @@
 \item Provide access mechanisms to these data products (both to the
 subsystems of the IPP and in some cases to external users);
-\item Continuously accept new image data and
-metadata from the telescope system;
+\item Continuously accept new image data and metadata from the
+telescope system;
 \item Execute various analysis processes using these data products;
 and
@@ -152,63 +179,283 @@
 requirements.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{System Architecture}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Architectural Components}
 
-The IPP is organised into several different architectural components,
-as follows:
+In Figure~\ref{fig:functionalities} we show the functionality of the
+IPP.
+
+The Observatory and Telescope System (\textbf{OATS}) system at the
+summit periodically produces metadata (e.g.\ weather measurements,
+observations completed) and pixel data (the image pixels from the
+cameras).  The \textbf{Pollster} regularly (e.g., twice per minute)
+polls OATS for the existence of new data.  If new data exists, the
+Pollster writes it to the \textbf{Metadata DB}, which maintains a
+table of observations that have been obtained and whether these
+observations are reduced, not reduced, or being reduced.  The
+\textbf{Scheduler} regularly (e.g., twice per minute) polls the
+Metadata DB for observations that match predefined criteria that are
+required to run reduction processes.  For example, the Phase 1
+processing requires that Phase 0 has been run on a focal plane
+metadata, and also requires that the observations are available and
+have not yet been processed.  If the criteria are met, the appropriate
+stage is passed to the \textbf{Localiser} which, checks the
+\textbf{Pixel DB} to determine if the stage should be performed on a
+particular node.  The Localiser passes the reduction stage to be
+processed, along with the preferred (or mandatory) node that should
+execute the reduction stage, to the \textbf{Controller}.  It is the
+Controller's responsibility to maintain the list of reduction stages
+to be processed and execute these stages on the \textbf{Nodes}.  The
+Nodes may retrieve the pixel data from OATS, they write to the Pixel
+DB the location of the products of the reduction and report their
+completion to the Controller.
+
+External systems, such as the Moving Object Processing System
+(\textbf{MOPS}) and other Client Science Pipelines (\textbf{CSPs})
+read the Metadata DB and the Object DB.  They may also write to the
+Object DB the classification of particular objects (e.g., identify an
+object as an asteroid).  Also, the MOPS and CSPs may also query the
+Pixel DB for the location of pixel data and copies data from the
+Nodes.
+
+\begin{figure}
+\psfig{file=pics/IPPfunctionalities,width=15cm,angle=0}
+\caption{The functionalities of the architectural design.  See the text
+for further explanation.}
+\label{fig:functionalities}
+\end{figure}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{OATS}
+
+The Observatory And Telescope System (OATS) is not a part of the IPP,
+but interfaces are required with it in order to allow the Pollster to
+get the list of observations not in the Metadata DB, and the nodes to
+retrieve pixel data.  Also, the Scheduler may report the need for new
+calibration data.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Pollster}
+
+The Pollster is a program that polls OATS at regular intervals for the
+existence of observations not contained in the Metadata DB.  New
+weather and image metadata are written to the Metadata DB.
+
+There is no reason why this architectural component cannot be
+contained within another (such as the Scheduler), but it is shown here
+as separate for simplicity.
+
+A polling model is adopted so that OATS' interface may be kept as
+simple as possible --- OATS should not be concerned with whether the
+IPP has received notifications.  Under this polling model, it is
+specifically the responsibility of the IPP to retrieve from OATS the
+metadata that is not not already in the Metadata DB.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Metadata DB}
+
+The Metadata DB stores and maintains the metadata\footnote{Note that
+metadata is any data which is not pixel data or object data.},
+including the list of images taken by the telescope system and whether
+these images have been processed.  The Metadata DB is regularly polled
+by the Scheduler to determine what images are ready to be processed.
+
+Both the Scheduler and the Pollster update the status of the Metadata
+DB --- the Pollster as new images become available at the Summit, and
+the Scheduler as images are processed.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Scheduler}
+
+The Scheduler is responsible for determining the processing stages
+that are required to be run on any data.  Examples of these processing
+stages are ``Copy the pixels from the summit'' and ``Run Phase 2
+processing on chip 12 of exposure 123''.
+
+Processing stages to be executed are passed to the Localiser, which
+returns to the Scheduler the list of processing stages with node
+assignments to each of the stages.  This list of processing stages
+with node assignments is passed to the Controller for execution.
+
+Processing stages which have executed are reported by the Controller,
+which updates the Metadata DB appropriately.
+
+The Scheduler may also interact with OATS to inform it of the need
+for new calibration data.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Localiser}
+
+It is the duty of the Localiser to assign processing stages to
+particular nodes.  This may be in order to optimise performance by
+distributing the stages across the nodes, or in the simplest possible
+case, it may make no recommendation upon the node which performs a
+particular stage.
+
+The Localiser may query the Pixel DB in order to identify the location
+of calibration data that may be needed for the processing stage to run
+(and in all likelihood, assign the processing stage to the same node as
+that which holds the calibration data).
+
+The Localiser may either demand or request that a stage is performed on
+a particular node, or make no recommendation, and passes the processing
+stage back to the Scheduler.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Controller}
+
+The Controller's job is to control the execution of the processing
+stages on the nodes.  It is passed stages by the Localiser, and
+executes them on the appropriate nodes.  It must detect whether a node
+executing a processing stage has died, and re-execute the stage on an
+alternate node.
+
+The completed stages are reported back to the Scheduler.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Pixel DB}
+
+The Pixel DB is responsible for storing and maintaining the location
+of pixel data in the IPP, including the raw images from the telescope,
+the master calibration images, the reference static-sky images, and
+any temporary image data products produced by the IPP.  It provides
+this information upon request to the Localiser.  
+
+Note that this design assumes that the pixel data will be stored on
+the same nodes that will be doing the processing.
+
+The Pixel DB will be periodically ``published'' as the quality of the
+data is assured.  The external world will only have access to the
+published version of the Pixel DB.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Nodes}
+
+The Nodes perform the grunt work of executing the processing stages as
+directed by the Controller.  When the processing stage has completed,
+they report back to the Controller.
+
+They may retrieve pixel data from OATS (the Summit) and write it to
+local disk when directed to do so by the Controller.  They also may
+access the Metadata DB to read configurations, weather information
+etc, and to write summary statistics etc.  They may also access the
+Object DB to read objects of interest, and to write objects from the
+processing stage.
+
+As they write products, the Nodes register with the Pixel DB that they
+have written the requested output (so that the Pixel DB is aware that
+the data has been written and is not merely scheduled to be written).
+The Nodes do not need to read from the Pixel DB, since everything
+(where to read input pixels from, where to write output pixels to) is
+specified by the Localiser.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Object DB}
+
+The Object DB is a facility to store all of the information about
+astronomical objects, including individual measurements of objects on
+the images, the summary information about those objects, and reference
+object data\footnote{Note that this is (possibly) a separate entity
+from the object database being developed by SAIC.}.
+
+The Nodes, CSPs and MOPS may read objects from the Object DB, and the
+Nodes may write objects (either new objects or updates), and the CSPs
+and MOPS may write certain fields of objects (e.g., the external
+identifiers and class of object).
+
+The Object DB will be periodically ``published'' as the quality of the
+data is assured.  The external world will only have access to the
+published version of the Object DB.  The published version of the
+Object DB will likely be the DB being developed by SAIC.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{CSPs and MOPS}
+
+The Client Science Programs (CSPs) and the Moving Object Processing
+System (MOPS) are not a part of the IPP, but are external systems.  We
+include them here to show the required interfaces.
+
+The CSPs and MOPS may query the Pixel DB, the Metadata DB and the
+Object DB.  In addition, they may write certain fields to the object
+DB (e.g., the external identifiers and class of object) as they
+process objects, and they may retrieve pixel data from the Nodes.
+
+Since ``CSPs'' is a vague term, we now give some examples which may
+help to illustrate the functionality.
+
+One example of a CSP is a web front-end to retrieve (published) images
+and objects from the Pixel DB and Object DB.
+
+Another example would be a program interested in searching for
+transiting extrasolar planets.  Such a program may periodically poll
+the Metadata DB for new processed observations in its region of
+interest (such as the Galactic Plane), retrieve the object photometry
+of all high signal-to-noise stars in the processed observations, and
+attempt to identify a planetary transit in progress.
+
+Yet another example would be a Stationary Transient Object Pipeline,
+which would periodically poll the Metadata DB for new processed
+observations, and query the Object DB for variable sources which were
+identified twice (so that they are not moving objects).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Related/Connected components}
+
+The Pollster may be contained within the Scheduler (i.e., the
+Scheduler may initiate and/or schedule as a processing stage the
+Pollster), but this is not assumed to be so in this document; this
+decision is left to the implementation.
+
+The Localiser is strongly coupled to the Pixel DB, and throughout this
+document, these are both referred to as components of the ``IPP Pixel
+Server''.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsubsection{Responsibility}
+
+The IPP team will develop and have responsibility for maintaining
+these systems.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsection{Processing Stages}
+
+We now consider the collection of IPP processing stages which are
+executed by the Controller on the Nodes.  We define a ``stage'' to be
+the largest complete task which may be performed in serial without
+interation between parallel threads.
+
+Depending on the particular stage, it may process individual images,
+collections of images, or on derived data products.  Because of the
+nature of the image data, many of the analysis stages can be run in
+parallel because, for example, the analysis of a chip in one image
+does not depend on the results from another chip.
+
+The data analysis stages are divided into several categories as follows:
 
 \begin{enumerate}
-\item IPP Pixel Server (IPS) --- a respository for all image pixel
-data, including the raw images from the telescope, the master
-calibration images, the reference static-sky images, and any temporary
-image data products produced by the IPP.
-\item IPP Object Database (IOD) --- a facility to store all of the
-information about astronomical objects, including individual
-measurements of objects on the images, the summary information about
-those objects, and reference object data\footnote{Note that this is
-(possibly) a separate entity from the object database being developed
-by SAIC.}.
-\item IPP Metadata Database (IMD) --- a storage element for all data
-which is neither image pixel data or astronomical object data.
-\item Analysis Pipelines --- all of the top-level analysis processes
-which are performed on images or collections of object data.
-\item Controller --- a system which manages the process of executing
-in parallel analysis pipelines on specific datasets on the cluster of
-computers.
-\item Scheduler --- a system which evaluates the current state of data
-in the various repositories and makes decisions about which analysis
-processes should be executed at any given time.
-\end{enumerate}
-
-The relationship between these software elements is shown in
-Figure~\ref{overview}.  This figure also shows the interactions
-between the IPP and other \PS{} systems.
-
-The IPP team will develop and have responsibility for these systems.
-
-\begin{figure}
-\begin{center}
-\resizebox{8cm}{!}{\includegraphics{pics/overview}}
-\caption{ \label{overview} IPP System Overview. \tbd{``Processing
-Jobs'' should be renamed ``Analysis Stages''.} }
-\end{center}
-\end{figure}
-
-\subsubsection{Analysis Stages}
-
-We now consider the collection of IPP analysis stages.  We define an
-analysis stage to be the largest complete task which may be performed
-in serial without interation between parallel threads.
-
-Depending on the particular analysis stage, it may process individual
-images, collections of images, or on derived data products.  Because
-of the nature of the image data, many of the analysis stages can be
-run in parallel because, for example, the analysis of a chip in one
-image does not depend on the results from another chip.
-
-The data analysis stages are divided into three categories as follows:
-
-\begin{enumerate}
-\item Science Image Analysis Stages
+\item Retrieval Stage --- pixel data are retrieved from OATS (the
+  Summit).
+\item Science Image Processing Stages
   \begin{enumerate}
   \item Phase 1: image processing preparation --- estimates
@@ -223,14 +470,17 @@
     by a major frame.
   \end{enumerate}
-\item Calibration Image Analysis Stages
+\item Calibration Image Processing Stages
   \begin{enumerate}
   \item Calibration 1: Basic master-detrend creation --- combination
-    of simple detrend images.
+    of simple detrend images (e.g., bias, dome flat etc).
   \item Calibration 2: Sky-model/fringe-mode generation ---
-    combination of more-complicated detrend images.
+    combination of more-complicated detrend images (e.g., fringe,
+    scattered light etc).
   \item Calibration 3: Flat-field correction image creation ---
     analysis of photometry from multiple dithered FPAs.
   \end{enumerate}
-\item Reference Catalog Analysis Stages
+\item Calibration Test Processing Stage --- tests whether new
+  calibration data are required.
+\item Reference Catalog Processing Stages
   \begin{enumerate}
   \item Astrometry reference catalog generation --- processing of the
@@ -243,20 +493,6 @@
 \end{enumerate}
 
-Figure~\ref{system} shows the flow of data between the various IPP
-software systems and the different analysis stages, each managed by
-the controller.  The thick lines represent the flow of pixel data, the
-thin lines represent the flow of metadata and object data, and the
-grey lines represent the flow of commands.  The hatched systems
-represent external \PS{} systems (OATS, the Sky Server, the SAIC
-Object Database, the Moving Object Processing System, and other Client
-Science Pipelines).
-
-\begin{figure}
-\begin{center}
-\resizebox{8cm}{!}{\includegraphics{pics/pipelines}}
-\caption{ \label{system} IPP System Overview. \tbd{Small part at
-top is missing.} }
-\end{center}
-\end{figure}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Hardware Systems}
@@ -278,4 +514,8 @@
 \end{center}
 \end{figure}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Software Hierarchy}
@@ -300,4 +540,7 @@
 stringent.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{External Libraries}
 
@@ -307,4 +550,7 @@
 implementation details of the external libraries.  Examples of the
 external libraries are FFTW and SLALib.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{\PS{} Library}
@@ -318,4 +564,7 @@
 and celestial coordinates.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Modules}
 
@@ -327,4 +576,7 @@
 modules are overscan subtraction and image combination.  Some modules
 (e.g.\ find objects on an image) will be used by multiple stages.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Stages}
@@ -339,4 +591,7 @@
 multiple telescopes and search for transients).
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Controllers}
 
@@ -345,4 +600,7 @@
 of the controller functionality is ``Run the phase 2 processing on
 exposure number 1234 using machines 1,3,5,7,9''.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Scheduler}
@@ -354,13 +612,26 @@
 controllers on exposure 1234''.
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{System Interfaces}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+ 
 \section{System Architectural Design}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{Architectural Components}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Pixel Server}
 
@@ -384,4 +655,6 @@
 making it available for processing by the IPP System.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Pixel Server Components}
 
@@ -395,4 +668,6 @@
 \item IPP Pixel Server I/O Library (IPSIOL)
 \end{enumerate}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{IPP Pixel Server Scheduler (IPSS)}
@@ -430,4 +705,5 @@
 operator.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{IPP Pixel Server Data Locality Optimizer (IPPDLO)}
@@ -437,4 +713,6 @@
 the data to be optimized so that it resides on the node which will
 process it.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{IPP Pixel Server Database (IPSD)}
@@ -458,4 +736,6 @@
 \end{itemize}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subparagraph{IPP Pixel Server Node Agent (IPSNA)}
 
@@ -473,4 +753,6 @@
 
 \tbd{The Agent does not wear a suit, nor does it know kung fu.}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{IPP Pixel Server I/O Library (IPSIOL)}
@@ -491,4 +773,5 @@
 \end{itemize}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Pixel Data Flow}
@@ -496,4 +779,6 @@
 Below we sketch out the intended sequence of events for common
 operations.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{Acquisition of data from the Summit Pixel Server}
@@ -521,4 +806,6 @@
 \end{figure}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subparagraph{Processing Reads}
 
@@ -532,4 +819,6 @@
 \end{enumerate}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subparagraph{Processing Writes}
 
@@ -541,4 +830,6 @@
 \item The processing stage uses the IPSIOL to write the image.
 \end{enumerate}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subparagraph{Processing Updates}
@@ -563,5 +854,6 @@
 \end{figure}
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Metadata Database}
@@ -582,4 +874,6 @@
 dedicated process or analysis pipeline collection of processes.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Metadata Tables}
 
@@ -587,5 +881,5 @@
 Database.
 
-\begin{tabular}{l}
+\begin{tabular}{ll}
 \hline
 \multicolumn{2}{l}{\bf Metadata Tables} \\
@@ -604,5 +898,5 @@
 Science Chip stats & Details on processed chips. \\
 Science Cell stats & Details on processed cells. \\
-Science FPA stats & Details on processed FPAs.
+Science FPA stats & Details on processed FPAs. \\
 Sky-Detector overlaps & List of overlaps between sky cells and detectors. \\
 Processed Sky-Cell stats & Details on sky cells. \\
@@ -615,4 +909,6 @@
 \hline
 \end{tabular}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Metadata Table Contents}
@@ -719,5 +1015,5 @@
 Number of chips & The number of chips that comprise the FPA. \\
 NX, NY & \tbd{Assuming the chips are laid out rectilinearly,} the number of chips in each dimension. \\
-Astrometry & The astrometry used for the FPA.
+Astrometry & The astrometry used for the FPA. \\
 \hline
 \end{tabular}
@@ -726,5 +1022,5 @@
 \hline
 \multicolumn{2}{l}{\bf Raw Chips} \\
-i, j & \tbd{Assuming a rectilinear FPA,} the chip number in each dimension.
+i, j & \tbd{Assuming a rectilinear FPA,} the chip number in each dimension. \\
 ID & Chip identification number. \\
 temps & The chip temperature. \\
@@ -783,9 +1079,9 @@
 Flat-field image & The flat-field image that was applied. \\
 Kernel convolution parameters & A description of the OT kernel. \\
-Flat-field stats & Summary statistics for flat-field (sigma of sky). & \\
+Flat-field stats & Summary statistics for flat-field (sigma of sky). \\
 Mask image & The mask image that was applied. \\
 Masking algorithm & \tbd{The algorithm used to mask the bad pixels.} \\
 Fringe images & The fringe model images that were used. \\
-Fringe stats & Summary statistics for fringes (fringe amplitude, sky sigma) & \\
+Fringe stats & Summary statistics for fringes (fringe amplitude, sky sigma) \\
 Object detection stats & Summary statistics for object detection (number of objects, depth, other
 input parameters). \\
@@ -795,5 +1091,5 @@
 Updated photometry parameters & The parameters used to update the photometry: magnitude zero point
 and other corrections. \\
-Photometry stats & Summary statistics for the photometry (number of stars, $sigma_m$) & \\
+Photometry stats & Summary statistics for the photometry (number of stars, $sigma_m$) \\
 Reference catalog & The reference catalog that was used for the photometry. \\
 PSF stats & Summary statistics of the PSF. \\
@@ -848,5 +1144,5 @@
 \begin{tabular}{ll}
 \hline
-\multicolumn{1}{l}{\bf Processed Sky-Cell stats} \\
+\multicolumn{2}{l}{\bf Processed Sky-Cell stats} \\
 Input Chips & Identification numbers of the chips used to produce the sky cell. \\
 PSF adjustments & \tbd{Adjustments to the PSF.} \\
@@ -934,12 +1230,12 @@
 \begin{tabular}{ll}
 \hline
-\multicolumn{1}{l}{\bf Calibration 3 output metadata } \\
+\multicolumn{2}{l}{\bf Calibration 3 output metadata } \\
 Input images & Identification numbers of the input chips. \\
 Input image stats & Summary statistics of the input chips. \\
-Input object summary stats & Summary statistics of the objects on the input chips (number, density, etc) & \\
+Input object summary stats & Summary statistics of the objects on the input chips (number, density, etc) \\
 Object rejection criteria & Parameters of the rejection step. \\
-Phot stats & Summary statistics of the relative photometry (Mcal, dMcal, Klam, etc, bin size) & \\
+Phot stats & Summary statistics of the relative photometry (Mcal, dMcal, Klam, etc, bin size) \\
 Residual stats & Summary statistics of the residuals. \\
-Output image params & Parameters of the output image (size, etc) & \\
+Output image params & Parameters of the output image (size, etc) \\
 \hline
 \end{tabular}
@@ -947,5 +1243,5 @@
 \begin{tabular}{ll}
 \hline
-\multicolumn{1}{l}{\bf Astrometric Reference Generation output metadata } \\
+\multicolumn{2}{l}{\bf Astrometric Reference Generation output metadata } \\
 \hline
 \end{tabular}
@@ -969,5 +1265,10 @@
 \end{tabular}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Metadata Queries}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Object Database}
@@ -987,7 +1288,9 @@
 etc?
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Object DB Tables}
 
-\begin{tabular}{l}
+\begin{tabular}{ll}
 \hline
 \multicolumn{2}{l}{\bf Object DB Tables} \\
@@ -1006,7 +1309,14 @@
 \end{tabular}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Object DB Table Contents}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Object DB Queries}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Controller}
@@ -1022,4 +1332,6 @@
 be distributed to the available machines in the cluster.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Components}
 
@@ -1040,4 +1352,6 @@
 error state.  The Controller daemon monitors the collection of remote
 clients and sends them new pending jobs when they become free.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Remote Clients}
@@ -1087,4 +1401,6 @@
 backgrounding.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{User Clients}
 
@@ -1111,4 +1427,7 @@
 the controller, including the list of pending, active, and completed
 jobs and the status of the individual jobs.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Scheduler}
@@ -1128,9 +1447,16 @@
 client) to send new jobs}.
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Analysis Stages}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Overview}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Science Image Pipelines}
@@ -1148,4 +1474,6 @@
 failure of the job.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Calibration Image Pipelines}
 
@@ -1159,4 +1487,6 @@
 pipeline, and the sky foreground pattern generation pipeline.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Reference Catalog Pipelines}
 
@@ -1166,4 +1496,6 @@
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Phase 1 : image processing preparation}
 
@@ -1204,4 +1536,6 @@
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Phase 2 : image reduction : new version}
 
@@ -1221,4 +1555,6 @@
 \end{center}
 \end{figure}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Phase 2 Concept}
@@ -1252,4 +1588,6 @@
 These modules are each explained below.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Form OT Kernel}
 
@@ -1259,4 +1597,5 @@
 used to convolve by.  The output is the OT convolution kernel.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Convolve de-trend images}
@@ -1284,4 +1623,5 @@
 Each of these will be used for a later module.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Overscan Subtraction}
@@ -1311,4 +1651,6 @@
 These will be used for a subsequent module.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Trim}
 
@@ -1330,4 +1672,6 @@
 modules.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Non-Linearity Correction}
 
@@ -1343,4 +1687,6 @@
 a polynomial correction, with the specified coefficients.  The output
 is the corrected object image, which is used for a later module.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Flat field}
@@ -1362,4 +1708,6 @@
 \end{enumerate}
 Both of these will be used in later modules.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Subtract sky}
@@ -1383,4 +1731,6 @@
 which is used for the next module.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Identify CRs by morphology}
 
@@ -1401,4 +1751,6 @@
 which is sent to the IPP Pixel Server.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Find objects}
 
@@ -1415,4 +1767,6 @@
 the image, which is sent to the metadata database, associated with the
 object image.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Bright object postage stamps}
@@ -1431,4 +1785,6 @@
 outputs are these postage stamps and pixel masks, which are sent to
 the IPP Pixel Server.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Metadata}
@@ -1450,4 +1806,6 @@
 \end{itemize}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Pixel Masks}
 \label{ap:masks}
@@ -1471,4 +1829,6 @@
 affect the flux in neighbouring pixels
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Object Catalogs}
 \label{ap:catalogs}
@@ -1494,4 +1854,6 @@
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Phase 3 : exposure analysis}
 
@@ -1552,4 +1914,6 @@
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Phase 4 : image combination}
 
@@ -1560,4 +1924,6 @@
 \end{center}
 \end{figure}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Phase 4 Concept}
@@ -1585,4 +1951,6 @@
 These modules are each explained below.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Combine Images}
 
@@ -1627,4 +1995,5 @@
 \end{enumerate}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Identify Sources}
@@ -1639,4 +2008,5 @@
 the IPP Object Database.
  
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Transient Identification}
@@ -1687,4 +2057,5 @@
 \end{enumerate}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Add to Static Sky}
@@ -1719,4 +2090,6 @@
 \end{enumerate}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Notes}
 
@@ -1732,5 +2105,7 @@
 \end{itemize}
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Basic detrend image creation}
 
@@ -1742,5 +2117,7 @@
 iteratively rejected.
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Fringe pattern and sky foreground model creation}
 
@@ -1754,5 +2131,7 @@
 parameters.  
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Photometric flat correction image creation}
 
@@ -1764,5 +2143,7 @@
 stage.  
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Astrometric Reference Catalog}
 
@@ -1771,5 +2152,7 @@
 For PS4, this shall be the PS1 catalog.
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Photometric Reference Catalog}
 
@@ -1781,8 +2164,19 @@
 For PS4, the PS1 catalogue shall be used.
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{Modules}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{\PS{} Library}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Internal Interfaces}
@@ -1808,5 +2202,7 @@
 C:DB interactions
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{External Interfaces}
@@ -1820,4 +2216,7 @@
 or the science processing pipelines.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{OATS}
 
@@ -1833,4 +2232,7 @@
 the PTS (i.e.\ calibration needs).
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Published Static Sky Server}
 
@@ -1840,4 +2242,7 @@
 provides updated static sky images to the SIS when available.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Object Database}
 
@@ -1847,4 +2252,7 @@
 timescale.  Is this a function of the IOD?}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Moving Object Processing System}
 
@@ -1853,4 +2261,7 @@
 The MOPS may interface with the IMD as needed.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Other Client Science Pipelines}
 
@@ -1859,5 +2270,12 @@
 much data?}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{Computer Hardware}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Overview}
@@ -1903,4 +2321,7 @@
 impact and will be evaluated along with the needed hardware at a later
 date.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Scenarios}
@@ -2004,4 +2425,7 @@
 \end{table}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Existing Hardware Throughput}
 
@@ -2038,4 +2462,7 @@
 \end{table}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Data Storage Requirements}
 
@@ -2048,4 +2475,6 @@
 requirements specifically for PS-1.  Table~\ref{storage} summarizes
 the data storage requirements in the different scenarios. 
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Raw Data Storage}
@@ -2066,7 +2495,10 @@
 number is simply scaled down by a factor of 4.  The choice of the
 minimal data volume does not affect these numbers because the raw data
-is already stored with 16 bit pixels.  ({\bf note: the PS-1 design
-reference may now require storage of the entire first year of data,
-calculated to be 200 TB}).
+is already stored with 16 bit pixels.
+
+\tbd{The PS-1 design reference may now require storage of the entire
+first year of data, calculated to be 200 TB.}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Static Sky Data Storage}
@@ -2082,6 +2514,10 @@
 while in PS-1, the reduction is a factor of roughly 8 because we only
 intend to store the static sky for the ecliptic plane survey and the
-small IPP verification program ({\bf note: this last point is no
-longer valid - the PS-1 static sky may require the entire 3pi}).
+small IPP verification program.
+
+\tbd{This last point is no longer valid - the PS-1 static sky may
+require the entire 3pi.}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Calibration Frame Storage}
@@ -2103,4 +2539,6 @@
 need to regenerate all master calibration frames on a weekly
 time-scale.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Metadata Database Storage}
@@ -2142,4 +2580,6 @@
 standard data volume choice.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Object Database Storage}
 
@@ -2160,4 +2600,7 @@
 limiting the depth of object detections.  Again, the minimal data
 volume scenario is irrelevant to the object database volume.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{CPU Requirements}
@@ -2252,4 +2695,7 @@
 \end{table}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Per-Node I/O Requirements}
 
@@ -2272,4 +2718,6 @@
 these assumptions, Table~\ref{throughput} lists the time allocations
 for the complete set of scenarios for the case of PS-4.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Random / Standard Data Scenario}
@@ -2288,4 +2736,6 @@
 of 15.2 seconds.  Note that the disk I/O is parallel with the network
 I/O and substantially underfills the disk bandwidth.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Random / Minimal Data Scenario}
@@ -2305,4 +2755,6 @@
 seconds.  Again, note that the disk I/O is parallel with the network
 I/O and substantially underfills the disk bandwidth.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Optimal / Standard Data Scenario}
@@ -2326,4 +2778,6 @@
 sequential with the disk I/O.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Optimal / Minimal Data Scenario}
 
@@ -2334,4 +2788,6 @@
 bandwidths, the data volumes imply a total I/O period of 4.6 seconds.
 Again, the network I/O is presumed to be sequential with the disk I/O.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Phase 4 Node I/O Requirements / Standard Data Volume}
@@ -2360,4 +2816,6 @@
 bandwidth, this implies an I/O period of 16 seconds for Phase 4.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Phase 4 Node I/O Requirements / Minimal Data Volume}
 
@@ -2397,4 +2855,7 @@
 \end{table}
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsubsection{Switch I/O Requirements}
 
@@ -2405,4 +2866,6 @@
 scenarios discussed above: Random Data Distribution, Random / Minimal,
 Optimal Data Distribution, and Optimal / Minimal.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Random / Standard Data Scenario}
@@ -2432,4 +2895,6 @@
 summit.)
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Random / Minimal Data Scenario}
 
@@ -2441,4 +2906,6 @@
 or 560 MB/sec.  
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \paragraph{Optimal / Standard Data Scenario}
 
@@ -2448,4 +2915,6 @@
 750 MB/sec.  The inter-switch communication also remains the same at
 1.12 GB/sec.  
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \paragraph{Optimal / Minimal Data Scenario}
@@ -2504,4 +2973,7 @@
 \end{center}
 \end{table}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsubsection{Conclusions}
@@ -2541,7 +3013,14 @@
 about I/O load on the processor during analysis.  
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{Notes}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Cell vs Chip vs FPA vs Major Frame} 
@@ -2586,4 +3065,8 @@
 either on the IPP side or on the PTS/TCS side.
 
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \subsection{Identifying ghosts, spikes, etc}
 
@@ -2598,10 +3081,7 @@
 addition of data.
 
-\subsection{Delete Phase 1?}
-
-except for the moving objects, phase 1 jobs are very light: include as
-part of phase 2 steps?  How long will the moving object ephemeris
-likely take?  The output of this analysis will not be required until
-Phase 4.  
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Pending Sky-cell / Detector table}
@@ -2610,4 +3090,9 @@
 give something which the scheduler can query to decide when to
 initiate phase 4. 
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{Appendices}