IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 1084


Ignore:
Timestamp:
Jun 24, 2004, 10:24:27 AM (22 years ago)
Author:
eugene
Message:

minor cleanups

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/design/ippSRS.tex

    r1067 r1084  
    1 %%% $Id: ippSRS.tex,v 1.5 2004-06-22 08:35:17 eugene Exp $
     1%%% $Id: ippSRS.tex,v 1.6 2004-06-24 20:24:27 eugene Exp $
    22\documentclass[panstarrs]{panstarrs}
    33
     
    1414
    1515% allow paragraphs to be listed in TOC for now
    16 \setcounter{tocdepth}{4}
     16\setcounter{tocdepth}{3}
    1717
    1818\begin{document}
     
    2828DR.05 & 2003.05.24 & Incorporating comments from internal review \\ \hline
    2929\RevisionsEnd
     30
     31\TBDsStart
     32% section     page      TBR number    Description
     33section & page & TBR & description \\ \hline
     34\TBDsEnd
    3035
    3136\tableofcontents
     
    99104\subsection{Science Requirements}
    100105\label{req:system-capabilities}
    101 
    102 \tbd{distinguish data products in commissioning, during AP survey,
    103 after AP survey}
    104 
    105 \tbd{define astronomy concepts: cleaned, pre-processed, significant,
    106 outlier}
    107106
    108107The IPP must perform the following tasks:
     
    171170 variations in the atmosphere across the field, and the reference
    172171 catalogs.  The photometric scatter in photometric weather must be
    173  better than \tbd{5 millimag} for relative photometry (relative to the
    174  internal filter system) and \tbd{10 millimag} for absolute photometry
     172 better than \tbr{5 millimag} for relative photometry (relative to the
     173 internal filter system) and \tbr{10 millimag} for absolute photometry
    175174 (relative to other filter systems such as the SDSS filters).
    176175
     
    218217\label{req:active-state}
    219218
    220 In active state, the IPP must accept images and metadata from the
    221 external sources (i.e., the summit) and automatically perform the
    222 complete set of image processing tasks, including both calibration and
    223 science image processing.  The IPP must respond to requests for data
    224 from client science pipelines.  In the active state, the IPP must
    225 respond to analysis priority requests issued by the IPP users.
     219In active state, the IPP must:
     220
     221\begin{enumerate}
     222\item Accept images and metadata from the external sources (i.e., the
     223  summit)
     224
     225\item Automatically perform the complete set of image processing
     226  tasks, including both calibration and science image processing.
     227
     228\item Respond to requests for data from client science pipelines.
     229
     230\item Respond to analysis priority requests issued by the IPP users.
     231\end{enumerate}
    226232
    227233\subsubsection{Paused State}
     
    234240\label{req:interactive-state}
    235241
    236 In interactive state, the IPP must accept incoming data and metadata,
    237 but must not automatically process the data.  The IPP must respond to
    238 user commands to initiate portions of the data analysis.
     242In interactive state, the IPP must:
     243
     244\begin{enumerate}
     245\item Accept incoming data and metadata from the external sources.
     246\item Not automatically process the data
     247\item Respond to user commands to initiate portions of the data
     248  analysis.
     249\end{enumerate}
    239250
    240251\subsection{Software Coding Requirements}
     
    243254\label{req:languages}
    244255
    245 Source code must be in C.  All source code must be compiled with `gcc'
    246 version v2.95 or higher, and the tested compiler version must be
    247 defined for the delivered software product.
    248 
    249 Scripting language must be \tbd{Python, version TBD}.
     256\begin{enumerate}
     257\item Source code must be in C. 
     258\item All source code must be compiled with `gcc' version v2.95 or higher.
     259\item The tested compiler version must be defined for the delivered software product.
     260\item Scripting language must be \tbd{Python}, version X.X.
     261\end{enumerate}
    250262
    251263\subsubsection{Interfaces}
    252 We require the following types of interfaces:
    253264\begin{enumerate}
    254265\item Access to low-level Library functions must be provided via C
     
    278289\subsubsection{Naming Conventions}
    279290
    280 Header files must have names starting \code{ps} or \code{p_ps} for
    281 private interface definitions. The latter must appear in a
     291\begin{enumerate}
     292\item Header files must have names starting \code{ps} or \code{p_ps}
     293for private interface definitions. The latter must appear in a
    282294subdirectory \code{private} of whichever directory is being searched
    283295for the public header files.
    284296
    285 Functions visible at global scope that are part of the public API must
    286 have names beginning with \code{ps} and follow the naming conventions
    287 in the coding standard.  Functions visible at global scope but which
    288 are not part of the public interface must have names beginning with
    289 \code{p_ps}.  Functions that are local to a file must \textit{not}
    290 start with \code{ps} or \code{p_ps}.
     297\item Functions visible at global scope that are part of the public
     298API must have names beginning with \code{ps} and follow the naming
     299conventions in the coding standard. 
     300
     301\item Functions visible at global scope but which are not part of the
     302public interface must have names beginning with \code{p_ps}.
    291303 
    292 Variables visible at global scope which are part of the public API
    293 must have names beginning with \code{ps}, and follow the naming
    294 conventions in the coding standard.  Variables that are visible at
    295 global scope but which are not part of the public interface must have
    296 names beginning with \code{p_ps}.  Variables that are local to a file
    297 must \textit{not} start with \code{ps} (or \code{p_ps}).
    298 
    299 The names of all enumerated types and C-preprocessor symbols (but not
    300 variables declared \code{const}) must start with \code{PS_}, in the
    301 case of public symbols, or \code{P_PS_}, for private symbols.  The
    302 rest of the name must be uppercase with words separated by underscores
    303 (\code{_}). An exception is the case of system utilities implemented
    304 as macros, in which case the names must conform to the convention for
    305 function names.
    306 
    307 When defining a function to convert from one type to another, the name
    308 must be of the form \code{psOldToNew}, e.g.\hfil\break
    309 \code{psEquatorialToEcliptic} (\emph{not}
     304\item Functions that are local to a file must \textit{not} start with
     305\code{ps} or \code{p_ps}.
     306 
     307\item Variables visible at global scope which are part of the public
     308API must have names beginning with \code{ps}, and follow the naming
     309conventions in the coding standard. 
     310
     311\item Variables that are visible at global scope but which are not
     312part of the public interface must have names beginning with
     313\code{p_ps}.
     314
     315\item Variables that are local to a file must \textit{not} start with
     316\code{ps} (or \code{p_ps}).
     317
     318\item The names of all enumerated types and C-preprocessor symbols
     319(but not variables declared \code{const}) must start with \code{PS_},
     320in the case of public symbols, or \code{P_PS_}, for private symbols.
     321The rest of the name must be uppercase with words separated by
     322underscores (\code{_}). An exception is the case of system utilities
     323implemented as macros, in which case the names must conform to the
     324convention for function names.
     325
     326\item When defining a function to convert from one type to another,
     327the name must be of the form \code{psOldToNew},
     328e.g.\code{psEquatorialToEcliptic} (\emph{not}
    310329\code{psEquatorial2Ecliptic}).
     330\end{enumerate}
    311331
    312332\subsubsection{C Programming Guidelines}
    313333
    314 Functions that assign to a variable must list that argument
    315 \textit{first}, following the pattern of \code{strcpy}; e.g.
    316 \begin{verbatim}
    317 void psVectorCopy(restrict psVector *out, const restrict psVector *in);
    318 \end{verbatim}
    319 
    320 Type definitions should always be accompanied by prototypes for their
    321 constructors and destructors, following these guidelines:
    322 
    323 \begin{itemize}
    324 \item The constructor name should consist of the type name followed by
    325 \code{Alloc}; e.g. a type \code{psImage} would be created by a
    326 function \code{psImage *psImageAlloc();}.
    327 
    328 \item The type should be freed with a destructor named
    329 \code{typeFree}, e.g.  \code{void psImageFree(psImage *image);}.
    330 
    331 \item The constructor must never return \code{NULL}, and no code calling the
    332 constructor should ever check the return value.
    333 
    334 \item The destructor must not return a value.
    335 
    336 \item The destructor must handle being passed \code{NULL} by simply
    337 returning immediately. This must not be treated as an error
    338 condition.
    339 
    340 \item Constructors and Destructors should use the memory reference
     334\begin{enumerate}
     335\item Functions that assign to a variable must list that argument
     336  \textit{first}, following the pattern of \code{strcpy}.  For
     337  example:
     338  \begin{verbatim}
     339  void psVectorCopy(restrict psVector *out, const restrict psVector *in);
     340  \end{verbatim}
     341
     342\item Type definitions should always be accompanied by prototypes for
     343  their constructors and destructors, following these guidelines:
     344
     345\begin{enumerate}
     346  \item The constructor name should consist of the type name followed
     347  by \code{Alloc}; e.g. a type \code{psImage} would be created by a
     348  function \code{psImage *psImageAlloc();}.
     349
     350  \item The type should be freed with a destructor named
     351  \code{typeFree}, e.g.  \code{void psImageFree(psImage *image);}.
     352
     353  \item The constructor must never return \code{NULL}, and no code
     354  calling the constructor should ever check the return value.
     355
     356  \item The destructor must not return a value.
     357
     358  \item The destructor must handle being passed \code{NULL} by simply
     359  returning immediately. This must not be treated as an error
     360  condition.
     361
     362  \item Constructors and Destructors should use the memory reference
    341363  counter facilities of the PSLib memory management system.
    342 
    343 \end{itemize}
     364\end{enumerate}
     365\end{enumerate}
    344366
    345367\subsubsection{Commenting and Documentation}
    346368
    347 Commenting of delivered C and Python code must follow the C and
    348 Python coding standards and must provide tags for Doxygen
    349 interpretation of the comments and program structures.
    350 
    351 Documentation for the IPP consists of source code documentation and
    352 user documentation.  Source code documentation must be generated with
    353 Doxygen from the in-line comments and must be provided as HTML,
    354 Latex, and man pages.  User documentation includes the API usage for
    355 the modules and library functions as well as user interface
    356 description for the higher-level architectural systems.  User
    357 documentation must be delivered as PDF documents.
     369\begin{enumerate}
     370\item Commenting of delivered C code must follow the C coding
     371  standards and provide tags for Doxygen interpretation of the
     372  comments and program structures.
     373
     374\item Commenting of delivered Python code must follow the Python
     375  coding standards.
     376
     377\item Source code documentation must be generated with Doxygen from
     378  the in-line comments and must be provided as HTML, Latex, and man
     379  pages. 
     380
     381\item User documentation includes the API usage for the modules and
     382  library functions as well as user interface description for the
     383  higher-level architectural systems.  User documentation must be
     384  delivered as PDF documents.
     385\end{enumerate}
    358386
    359387\subsubsection{Version Control}
     
    409437The architectural components consist of:
    410438
    411 \begin{enumerate}
     439\begin{itemize}
    412440
    413441\item {\bf Image Server:} This component is a large data store for all
     
    444472 to assign the analysis tasks to the Controller.
    445473
    446 \end{enumerate}
     474\end{itemize}
    447475
    448476The relationship between these software elements is shown in
     
    461489\subsubsection{Image Server}
    462490
    463 The IPP Image Server must store images on a distributed collection of
    464 computer disks.  Individual instances of a file are only required to
    465 be stored on a single machine (striping across computers is not a
    466 requirement). 
    467 
    468 The IPP Image Server must be capable of honoring requests to store an
    469 image on a specific machine.  If such a request cannot be honored (ie,
    470 the machine is down), the IPP Image Server must select an appropriate
    471 machine and notify the requesting agent of the new location. 
    472 
    473 The IPP Image Server must store multiple copies of each image upon
    474 request, the number of copies specified independently for each file by
    475 the user.
    476 
    477 The IPP Image Server must maintain a record of all image copies
    478 currently available in the repository.  This record must include the
    479 image name, location (which machine), the image size, and the state of
    480 the image (available, locked, deleted).
    481 
    482 The IPP Image Server must lock images in the repository on request.
    483 Both read (shared) and write (exclusive) locks must be provided.  A
    484 read lock must prevent write access to the file; a write lock must
    485 prevent both read and write access.
    486 
    487 The IPP Image Server must return the image location (the computer on
    488 which it resides) upon request.
    489 
    490 The IPP Image Server must provide a specified image upon request.
    491 
    492 The IPP Image Server must delete images in the repository on request.
    493 
    494 The IPP Image Server must accept images from the summit at the maximum
    495 rate of 1 full-camera image every 30 seconds.  The IPP Image Server
    496 must therefore accept new images into the repository at a rate of 64
    497 raw OTAs in 30 seconds and a total input data volume rate of 75
    498 MB/sec.
     491\begin{enumerate}
     492\item The IPP Image Server must store images on a distributed
     493  collection of computer disks.  Individual instances of a file are
     494  only required to be stored on a single machine (striping across
     495  computers is not a requirement).
     496
     497\item The IPP Image Server must be capable of honoring requests to
     498  store an image on a specific machine. 
     499
     500\item If such a request cannot be honored (ie, the machine is down),
     501  the IPP Image Server must select an appropriate machine and notify
     502  the requesting agent of the new location.
     503
     504\item The IPP Image Server must store multiple copies of each image
     505  upon request, the number of copies specified independently for each
     506  file by the user.
     507
     508\item The IPP Image Server must maintain a record of all image copies
     509  currently available in the repository.  This record must include the
     510  image name, location (which machine), the image size, and the state
     511  of the image (available, locked, deleted).
     512
     513\item The IPP Image Server must lock images in the repository on
     514  request.  Both read (shared) and write (exclusive) locks must be
     515  provided.  A read lock must prevent write access to the file; a
     516  write lock must prevent both read and write access.
     517
     518\item The IPP Image Server must return the image location (the
     519  computer on which it resides) upon request.
     520
     521\item The IPP Image Server must provide a specified image upon request.
     522
     523\item The IPP Image Server must delete images in the repository on request.
     524
     525\item The IPP Image Server must accept images from the summit at the
     526  maximum rate of 1 full-camera image every 30 seconds.  The IPP Image
     527  Server must therefore accept new images into the repository at a
     528  rate of 64 raw OTAs in 30 seconds and a total input data volume rate
     529  of 75 MB/sec.
     530\end{enumerate}
    499531
    500532\tbd{archive lifetime}
     
    532564\end{table}
    533565
    534 The AP Database must accept and store individual detections and
    535 collections of detections along with information about the image which
    536 provided the detections.
    537 
    538 Detections must be saved as one of several detection classes (P2, P4S,
    539 P4D, SS) and the AP Database must store the appropriate parameters,
    540 listed in Table~\ref{APdetections}, for each class.
    541 
    542 The AP Database must identify the image which provided the detection,
    543 or in the case of external references, an identifier specific to the
    544 reference source.
    545 
    546 The AP Database must group detections into objects and measure average
    547 parameters of those objects. 
    548 
    549 The AP Database must store parallax and proper motion parameters for a
    550 subset of the average objects.
    551 
    552 The AP Database must store image and filter calibration information
    553 necessary to convert between instrumental magnitudes and calibrated
    554 magnitudes in standard systems.
    555 
    556 The AP Database must perform at least the follow queries, with
    557 constraints on the output based on at least time ranges, magnitude
    558 limits, error limits:
    559 \begin{enumerate}
    560 \item given $(RA,DEC)$ and a Radius, return all objects and/or
    561 detections in the region.
    562 
    563 \item given $(RA,DEC)_0$ to $(RA,DEC)_1$, return all objects and/or
    564   detections in the region.
    565 
    566 \item given $(RA,DEC)$, return closest object.
    567 
    568 \item given object ID, return all detections
    569 
    570 \item given detection, return source image data.
    571 
    572 \item given detection, return object.
    573 
    574 \item given $(RA,DEC)$, return all images overlapping coordinate.
    575 
    576 \item given $(RA,DEC)$ and a Radius, return all images overlapping region.
    577 
    578 \item given $(RA,DEC)_0$ to $(RA,DEC)_1$, return all images overlapping
    579   region.
    580 
    581 \item given detection instrumental magnitude, return derived
    582   magnitudes based on calibration information.
    583 
    584 \item given a collection of detections in a filter, determine the
    585   object average magnitude in that filter.
    586 
    587 \item given a collection of objects and detections, determine the
    588   individual image zero-points.
    589 
    590 \item given a region, return all possible combinations of the object
    591   or detection magnitudes $(M_1 - M_2)$.
    592 
    593 \item given a list of $(RA,DEC)$ entries, return all nearest objects. 
    594 
    595 \item given a filter, telescope, or detector, return all calibration
    596   terms and history.
    597 
    598 \item given a detection, return all non-detections from images which
    599   overlapped the detection coordinates.
    600 
    601 \end{enumerate}
    602 
    603 The AP Database must accept detection IDs of moving objects and label
    604 the detections with the identified object.
     566\begin{enumerate}
     567\item The AP Database must accept and store individual detections and
     568  collections of detections along with information about the image
     569  which provided the detections.
     570
     571\item Detections must be saved as one of several detection classes
     572  (P2, P4S, P4D, SS) and the AP Database must store the appropriate
     573  parameters, listed in Table~\ref{APdetections}, for each class.
     574
     575\item The AP Database must identify the image which provided the
     576  detection, or in the case of external references, an identifier
     577  specific to the reference source.
     578
     579\item The AP Database must group detections into objects and measure
     580  average parameters of those objects.
     581
     582\item The AP Database must store parallax and proper motion parameters
     583  for a subset of the average objects.
     584
     585\item The AP Database must store image and filter calibration
     586  information necessary to convert between instrumental magnitudes and
     587  calibrated magnitudes in standard systems.
     588
     589\item The AP Database must perform at least the follow queries, with
     590  constraints on the output based on at least time ranges, magnitude
     591  limits, error limits:
     592
     593 \begin{enumerate}
     594 \item given $(RA,DEC)$ and a Radius, return all objects and/or
     595 detections in the region.
     596
     597 \item given $(RA,DEC)_0$ to $(RA,DEC)_1$, return all objects and/or
     598   detections in the region.
     599
     600 \item given $(RA,DEC)$, return closest object.
     601
     602 \item given object ID, return all detections
     603
     604 \item given detection, return source image data.
     605
     606 \item given detection, return object.
     607
     608 \item given $(RA,DEC)$, return all images overlapping coordinate.
     609
     610 \item given $(RA,DEC)$ and a Radius, return all images overlapping region.
     611
     612 \item given $(RA,DEC)_0$ to $(RA,DEC)_1$, return all images overlapping
     613   region.
     614
     615 \item given detection instrumental magnitude, return derived
     616   magnitudes based on calibration information.
     617
     618 \item given a collection of detections in a filter, determine the
     619   object average magnitude in that filter.
     620
     621 \item given a collection of objects and detections, determine the
     622   individual image zero-points.
     623
     624 \item given a region, return all possible combinations of the object
     625   or detection magnitudes $(M_1 - M_2)$.
     626
     627 \item given a list of $(RA,DEC)$ entries, return all nearest objects. 
     628
     629 \item given a filter, telescope, or detector, return all calibration
     630   terms and history.
     631
     632 \item given a detection, return all non-detections from images which
     633   overlapped the detection coordinates.
     634 \end{enumerate}
     635
     636\item The AP Database must accept detection IDs of moving objects and
     637  label the detections with the identified object.
     638
     639\item The AP Database must accept new detections at the rate generated
     640  by the telescope from the Phase 2 and Phase 4 analysis.  Except
     641  within 10 degrees of the galactic plane, the AP Database must keep
     642  up with the incoming rates.  The expected rates are listed in
     643  Table~\ref{APrates}, along with the total data volume required for
     644  storage space over the PS-1 lifetime.
     645
     646\end{enumerate}
     647
     648\tbd{archive lifetime}
     649
     650\tbd{reliability}
     651
     652\tbd{backups}
    605653
    606654\begin{table}
     
    625673\end{table}
    626674
    627 The AP Database must accept new detections at the rate generated by
    628 the telescope from the Phase 2 and Phase 4 analysis.  Except within 10
    629 degrees of the galactic plane, the AP Database must keep up with the
    630 incoming rates.  The expected rates are listed in Table~\ref{APrates},
    631 along with the total data volume required for storage space over the
    632 PS-1 lifetime. 
    633 
    634 \tbd{archive lifetime}
    635 
    636 \tbd{reliability}
    637 
    638 \tbd{backups}
    639 
    640675\subsubsection{Metadata Database}
    641676
    642677\begin{table}
    643678\begin{center}
    644 \caption{Metadata Classes\label{, and the while
    645 the metadata}}
     679\caption{Metadata Classes\label{metadata}}
    646680\begin{tabular}{l}
    647681\hline
     
    673707stages
    674708
    675 The Metadata Database must store and provide metadata for all raw
    676 images, for processed images, for the calibration images (both raw and
    677 master), for the extracted object lists.  Metadata describing the
    678 environmental conditions at the telescope must also be stored and
    679 provided as needed.  Table~\ref{metadata} lists the classes of
    680 metadata which must be stored by the Metadata Database.
    681 
    682 If analysis results are exchanged between analysis stages via the
    683 Metadata Database, it must provide access to the queried data on
    684 timescales of $<2$ seconds to avoid slowing down the analysis systems.
    685 
    686 The Metadata Database must store the metadata for the lifetime of the
    687 project.  The Metadata Database must be capable of accepting a total
    688 data volume after 2 years of operation of 128 GB.
    689 
    690 The Metadata Database must respond to simple queries which return the
    691 data in the categories listed in Table~\ref{metadata} based on the
    692 primary data key and with basic constraints of time ranges and other
    693 simple conditional constraints.
    694 
    695 The Metadata must store descriptive information about the raw images
    696 received from the summit and the current state of the data processing.
    697 The Metadata must also store descriptive information for each of the
    698 static sky images currently available. 
    699 
    700 The IPP requires configuration information defining the organization
    701 and configuration of the IPP itself.  The Metadata database must store
    702 the configuration information with restricted access so that only
    703 specific people may change the information.  Examples of configuration
    704 data include the default parameters for the various analysis programs,
    705 the description of the computing environment, and the process status
    706 information, etc.  The Metadata Database must restrict access to the
    707 scientific parameters to a different group from the software and
    708 hardware configuration parameters.
     709\begin{enumerate}
     710\item The Metadata Database must store and provide metadata for all
     711  raw images, for processed images, for the calibration images (both
     712  raw and master), for the extracted object lists.  Metadata
     713  describing the environmental conditions at the telescope must also
     714  be stored and provided as needed.  Table~\ref{metadata} lists the
     715  classes of metadata which must be stored by the Metadata Database.
     716
     717\item If analysis results are exchanged between analysis stages via
     718  the Metadata Database, it must provide access to the queried data on
     719  timescales of $<2$ seconds to avoid slowing down the analysis
     720  systems.
     721
     722\item The Metadata Database must store the metadata for the lifetime
     723  of the project.
     724
     725\item The Metadata Database must be capable of accepting a total data
     726  volume after 2 years of operation of 128 GB.
     727
     728\item The Metadata Database must respond to simple queries which
     729  return the data in the categories listed in Table~\ref{metadata}
     730  based on the primary data key and with basic constraints of time
     731  ranges and other simple conditional constraints.
     732
     733\item The Metadata must store descriptive information about the raw
     734  images received from the summit and the current state of the data
     735  processing.
     736
     737\item The Metadata must also store descriptive information for each of
     738  the static sky images currently available.
     739
     740\item The IPP requires configuration information defining the
     741  organization and configuration of the IPP itself.  The Metadata
     742  database must store the configuration information with restricted
     743  access so that only specific people may change the information.
     744  Examples of configuration data include the default parameters for
     745  the various analysis programs, the description of the computing
     746  environment, and the process status information, etc.
     747
     748\item The Metadata Database must restrict access to the scientific
     749  parameters to a different group from the software and hardware
     750  configuration parameters.
     751
     752\item In the discussion of the Analysis Stages below, various steps
     753  specify that the values are user-configurable parameters.  These
     754  parameters must be stored in and extracted from the Metadata
     755  Database.
     756\end{enumerate}
    709757
    710758\subsubsection{Controller}
    711 
    712 The IPP Controller must manage tasks on a cluster of up to 128
    713 computers. 
    714 
    715 On startup, the IPP Controller must attempt to establish communication
    716 with all of its computers and set their state to be {\tt alive} or
    717 {\tt dead} based on the success of the connection.
    718 
    719 The IPP Controller must detect computers which crash or stop
    720 responding and set their state to {\tt dead}.
    721 
    722 The IPP Controller must attempt to re-establish communication with
    723 {\tt dead} computers. 
    724 
    725 The IPP Controller must accept tasks from external users and systems,
    726 which may specify a desired CPU (node) and priority in addition to the
    727 task command.
    728 
    729 The IPP Controller must attempt to run pending tasks on the desired
    730 node, if available (not {\tt dead} or {\tt off}).  If the node is
    731 unavailable, the IPP Controller must attempt to run the task on
    732 another node.  If the node is available, the IPP Controller must
    733 attempt to run a given task only if no higher-priority tasks are
    734 available and no task is currently being executed.
    735 
    736 The IPP Controller must monitor the output from the task and write it
    737 to an associated log destination.
    738 
    739 The IPP Controller must monitor the execution status of each task
    740 currently executing on a node and perform the following actions:
    741 \begin{enumerate}
    742 \item identify the task as successful if it has a valid exit status.
    743 \item identify the task as unsuccessful if it has an error exit
    744   status.
    745 \item identify the task as unattempted if the computer crashed.
    746 \end{enumerate}
    747 
    748 The IPP Controller must accept and perform the following external
    749 commands:
    750 \begin{enumerate}
    751 \item add a task to the pending task list.
    752 \item delete a specific task from the pending task list.
    753 \item return the current status of a specific task.
    754 \item return a list of all pending and non-pending tasks.
    755 \item set a specified computer state to {\tt off} or {\tt dead}.
    756 \item restrict a specified CPU to a class of tasks.
    757 \item halt execution of a specified task.
    758 \item set the IPP Controller state to {\tt finish}, {\tt abort}, or
    759   {\tt stop}.
     759\begin{enumerate}
     760
     761\item The IPP Controller must manage tasks on a cluster of up to 128
     762  computers.
     763
     764\item On startup, the IPP Controller must attempt to establish
     765  communication with all of its computers and set their state to be
     766  {\tt alive} or {\tt dead} based on the success of the connection.
     767
     768\item The IPP Controller must detect computers which crash or stop
     769  responding and set their state to {\tt dead}.
     770
     771\item The IPP Controller must attempt to re-establish communication
     772  with {\tt dead} computers.
     773
     774\item The IPP Controller must accept tasks from external users and
     775  systems, which may specify a desired CPU (node) and priority in
     776  addition to the task command.
     777
     778\item The IPP Controller must attempt to run pending tasks on the
     779  desired node, if available (not {\tt dead} or {\tt off}).
     780
     781\item If the node is unavailable, the IPP Controller must attempt to
     782  run the task on another node.
     783
     784\item If the node is available, the IPP Controller must attempt to run
     785  a given task only if no higher-priority tasks are available and no
     786  task is currently being executed.
     787
     788\item The IPP Controller must monitor the output from the task and
     789  write it to an associated log destination.
     790
     791\item The IPP Controller must monitor the execution status of each
     792  task currently executing on a node and perform the following
     793  actions:
     794
     795  \begin{enumerate}
     796  \item identify the task as successful if it has a valid exit status.
     797  \item identify the task as unsuccessful if it has an error exit
     798    status.
     799  \item identify the task as unattempted if the computer crashed.
     800  \end{enumerate}
     801
     802\item The IPP Controller must accept and perform the following
     803  external commands:
     804  \begin{enumerate}
     805  \item add a task to the pending task list.
     806  \item delete a specific task from the pending task list.
     807  \item return the current status of a specific task.
     808  \item return a list of all pending and non-pending tasks.
     809  \item set a specified computer state to {\tt off} or {\tt dead}.
     810  \item restrict a specified CPU to a class of tasks.
     811  \item halt execution of a specified task.
     812  \item set the IPP Controller state to {\tt finish}, {\tt abort}, or
     813    {\tt stop}.
     814  \end{enumerate}
    760815\end{enumerate}
    761816
    762817\subsubsection{Scheduler}
    763 
    764 The IPP Scheduler initiates analysis tasks which it must send to the
    765 IPP Controller.
    766 
    767 All analysis tasks sent by the IPP Scheduler must include a complete
    768 UNIX command with necessary arguments, the priority of the task, and
    769 optionally the desired processing node.
    770 
    771 The IPP Scheduler must refer to several input data sources to decide
    772 what tasks to initiate.  These data sources include the IPP Metadata
    773 Database, the Summit Metadata Database, and User requests. 
    774 
    775 The IPP Scheduler must query the Databases on a regular basis to check
    776 for new input information.  These queries must take place at least
    777 once every \tbr{5 seconds}.
    778 
    779 The IPP Scheduler must accept new User input in real-time (within 0.1
    780 seconds of the request).
    781 
    782 The IPP Scheduler must construct new tasks on the basis of the inputs
    783 and a task dependency table. 
    784 
    785 When the IPP Scheduler is placed in the {\em paused state}, it must
    786 only initiate User-requested tasks.
    787 
    788 When the IPP Scheduler is placed in the {\em interactive state}, it
    789 must initiate User-requested tasks as well as data transfer tasks.
    790 
    791 When the IPP Scheduler is placed in the {\em automatic state}, it must
    792 initiate the most appropriate task based on the inputs.
    793 
    794 The IPP Scheduler must receive the exit status of tasks from the IPP
    795 Controller.
    796 
    797 The IPP Scheduler must send the exit status of the analysis tasks to
    798 the appropriate destination as defined by the task dependency table.
     818\begin{enumerate}
     819\item The IPP Scheduler must send the analysis tasks which it
     820  initiates to the IPP Controller.
     821
     822\item All analysis tasks sent by the IPP Scheduler must include a
     823  complete UNIX command with necessary arguments, the priority of the
     824  task, and optionally the desired processing node.
     825
     826\item The IPP Scheduler must refer to several input data sources to
     827  decide what tasks to initiate.  These data sources include the IPP
     828  Metadata Database, the Summit Metadata Database, and User requests.
     829
     830\item The IPP Scheduler must query the Databases on a regular basis to
     831  check for new input information.  These queries must take place at
     832  least once every \tbr{5 seconds}.
     833
     834\item The IPP Scheduler must accept new User input in real-time
     835  (within 0.1 seconds of the request).
     836
     837\item The IPP Scheduler must construct new tasks on the basis of the
     838  inputs and a task dependency table.
     839
     840\item When the IPP Scheduler is placed in the {\em paused state}, it
     841  must only initiate User-requested tasks.
     842
     843\item When the IPP Scheduler is placed in the {\em interactive state},
     844  it must initiate User-requested tasks as well as data transfer
     845  tasks.
     846
     847\item When the IPP Scheduler is placed in the {\em automatic state},
     848  it must initiate the most appropriate task based on the inputs.
     849
     850\item The IPP Scheduler must receive the exit status of tasks from the
     851  IPP Controller.
     852
     853\item The IPP Scheduler must send the exit status of the analysis
     854  tasks to the appropriate destination as defined by the task
     855  dependency table.
     856\end{enumerate}
    799857
    800858\subsection{Analysis Stages}
     
    813871group.
    814872
    815 The science image analysis stages must perform their analysis quickly
    816 enough to keep up with the incoming data stream.  The required
    817 processing time is derived from the rate at which science images are
    818 obtained by PS-1.  At a minimum, the Science Image Analysis must keep
    819 up with the average image rate over the course of 1 day.  In order to
    820 provide a sufficient buffer for variations in the processing speed,
    821 the Science Image Analysis must be able to process all images from a
    822 night within 12 hours. 
    823 
    824 The maximum latency between the acquisition of an image and the
    825 completion of the science image analysis is set by the science
    826 requirements of the fast transient recovery programs.  The science
    827 image analysis must process images from these observing programs
    828 within \tbr{5 min} of their arrival time in the IPP Image Server.
    829 
    830 The science image analysis stages must processes up to 1000 science
    831 images per night. 
     873\begin{enumerate}
     874\item The science image analysis stages must perform their analysis
     875  quickly enough to keep up with the incoming data stream.  The
     876  required processing time is derived from the rate at which science
     877  images are obtained by PS-1. 
     878
     879\item At a minimum, the Science Image Analysis must keep up with the
     880  average image rate over the course of 1 day. 
     881
     882\item In order to provide a sufficient buffer for variations in the
     883  processing speed, the Science Image Analysis must be able to process
     884  all images from a night within 12 hours.
     885
     886\item The maximum latency between the acquisition of an image and the
     887  completion of the science image analysis is set by the science
     888  requirements of the fast transient recovery programs.  The science
     889  image analysis must process images from these observing programs
     890  within \tbr{5 min} of their arrival time in the IPP Image Server.
     891
     892\item The science image analysis stages must processes up to 1000
     893  science images per night.
     894
     895\end{enumerate}
    832896
    833897\subsubsection{Phase 1 : image processing preparation}
    834898
    835 The Phase 1 analysis stage must determine the astrometric solution of
    836 the complete camera (FPA image) with an accuracy of \tbr{1 arcsec}
    837 peak-to-peak deviation. 
    838 
    839 The Phase 1 analysis stage must load the guide star pixel and
    840 celestial coordinates from the \tbd{IPP Metadata Database}\comment{or
    841 from the image header?}.
    842 
    843 If guide stars are not available, the Phase 1 analysis stage must
    844 extract bright stars from the image.  This extraction must be done in
    845 less than \tbr{1 second}.  The total number of stars and size of the
    846 bright-star acquisition box must be a user-configurable parameter.
    847 
    848 In order for blind astrometry of an image to succeed, it is necessary
    849 that approximate image coordinates be known.  The Phase 1 analysis
    850 must be able to succeed despite initial coordinate errors as large as
    851 \tbr{20\arcsec}.
    852 
    853 The Phase 1 analysis stage must construct a table of the overlaps
    854 between the science image to be processed and the static sky images.
    855 
    856 The overlaps must be overestimated by a small amount so that errors in
    857 astrometry at Phase 1 will not cause any valid static sky / science
    858 image pairs to be missed.  The amount of overlap must be a
    859 user-configurable parameter.
    860 
    861 Sky cells which do not have sufficient science image overlap \tbd{$<
    862 5\%$} must be excluded from the overlap table.
    863 
    864 It is not unusual for an image to be obtained with invalid coordinates
    865 or without any valid stars.  For example, the telescope control system
    866 may make an error and report the wrong time or coordinates.  Or, the
    867 image may be obtained in exceptionally poor conditions with no
    868 detected stars.  Phase 1 must return a descriptive error message in
    869 these conditions.
     899\begin{enumerate}
     900\item The Phase 1 analysis stage must determine the astrometric
     901  solution of the complete camera (FPA image) with an accuracy of
     902  \tbr{1 arcsec} peak-to-peak deviation.
     903
     904\item The Phase 1 analysis stage must load the guide star pixel and
     905  celestial coordinates from the \tbd{IPP Metadata
     906  Database}\comment{or from the image header?}.
     907
     908\item If guide stars are not available, the Phase 1 analysis stage
     909  must extract bright stars from the image.
     910
     911\item This extraction must be done in less than \tbr{1 second}. 
     912 
     913\item The total number of stars and size of the bright-star
     914  acquisition box must be a user-configurable parameter.
     915
     916\item In order for blind astrometry of an image to succeed, it is
     917  necessary that approximate image coordinates be known.  The Phase 1
     918  analysis must be able to succeed despite initial coordinate errors
     919  as large as \tbr{20\arcsec}.
     920
     921\item The Phase 1 analysis stage must construct a table of the
     922  overlaps between the science image to be processed and the static
     923  sky images.
     924
     925\item The overlaps must be overestimated by a small amount so that
     926  errors in astrometry at Phase 1 will not cause any valid static sky
     927  / science image pairs to be missed.
     928
     929\item The amount of overlap must be a user-configurable parameter.
     930 
     931\item Sky cells which do not have sufficient science image overlap
     932  \tbd{$< 5\%$} must be excluded from the overlap table.
     933
     934\item It is not unusual for an image to be obtained with invalid
     935  coordinates or without any valid stars.  For example, the telescope
     936  control system may make an error and report the wrong time or
     937  coordinates.  Or, the image may be obtained in exceptionally poor
     938  conditions with no detected stars.  Phase 1 must return a
     939  descriptive error message in these conditions.
     940\end{enumerate}
    870941
    871942\subsubsection{Phase 2 : image reduction}
     
    874945the detector are processed to remove instrumental signatures. 
    875946
    876 The Phase 2 analysis stage must consult the processing recipe to
    877 define the necessary analysis steps performed by the Phase 2 stage.
    878 
    879 Phase 2 must perform the analysis steps only if required by the
    880 processing recipe.  The processing recipe must define the stages to be
    881 executed with optional exposure time and background flux limits to
    882 require or exclude select certain stages.
    883 
    884 In the discussion below, various steps specify that the values are
    885 user-configurable parameters.  These parameters must be stored in and
    886 extracted from the Metadata Database.
     947\paragraph{Processing Recipe}
     948\begin{enumerate}
     949\item The Phase 2 analysis stage must consult the processing recipe to
     950  define the necessary analysis steps performed by the Phase 2 stage.
     951
     952\item Phase 2 must perform the analysis steps only if required by the
     953  processing recipe.
     954
     955\item The processing recipe must define the stages to be executed with
     956  optional exposure time and background flux limits to require or
     957  exclude select certain stages.
     958\end{enumerate}
    887959
    888960\paragraph{Detrend Image Convolutions}
    889 
    890 The Phase 2 analysis stage must determine the OT kernel from the IPP
    891 Metadata Database\comment{or image header}.
    892 
    893 The Phase 2 analysis stage must convolve the flat-field and
    894 high-spatial-frequency fringe images with the OT kernel.  If no OT
    895 kernel exists, this step must be silently skipped.
     961\begin{enumerate}
     962
     963\item The Phase 2 analysis stage must determine the OT kernel from the
     964  IPP Metadata Database\comment{or image header}.
     965
     966\item The Phase 2 analysis stage must convolve the flat-field and
     967  high-spatial-frequency fringe images with the OT kernel.
     968
     969\item If no OT kernel exists, this step must be silently skipped.
     970\end{enumerate}
    896971
    897972\paragraph{Flag bad and saturated pixels}
    898 
    899 The Phase 2 analysis must load the basic bad pixel map appropriate to
    900 the detector of interest. 
    901 
    902 The Phase 2 analysis must use the OT kernel to grow the traps in the
     973\begin{enumerate}
     974
     975\item The Phase 2 analysis must load the basic bad pixel map appropriate to
     976the detector of interest.
     977
     978\item The Phase 2 analysis must use the OT kernel to grow the traps in the
    903979raw bad pixel map. 
    904980
    905 The Phase 2 analysis must mask saturated pixels and a user-specified
     981\item The Phase 2 analysis must mask saturated pixels and a user-specified
    906982number of surrounding pixels.
    907983
    908 Different bits must be set to identify different reasons for masking
     984\item Different bits must be set to identify different reasons for masking
    909985the pixels.
     986\end{enumerate}
    910987
    911988\paragraph{Bias correction via overscan subtraction}
    912 
    913 Phase 2 must perform bias subtraction on the image.
    914 
    915 Phase 2 must choose the bias subtraction method and analysis statistic
     989\begin{enumerate}
     990
     991\item Phase 2 must perform bias subtraction on the image.
     992
     993\item Phase 2 must choose the bias subtraction method and analysis statistic
    916994based on the user-configured parameters.
    917995
    918 The bias correction must be measured from the image overscan region.
    919 
    920 The overscan region must be determined from the image
     996\item The bias correction must be measured from the image overscan region.
     997
     998\item The overscan region must be determined from the image
    921999header\comment{or Metadata DB}.
    9221000
    923 The bias subtraction must apply one of the following bias corrections,
     1001\item The bias subtraction must apply one of the following bias corrections,
    9241002depending on the user parameters:
     1003
    9251004\begin{enumerate}
    9261005\item subtract a single constant from the image. 
     
    9341013\end{enumerate}
    9351014
    936 The statistic used to calculate the overscan constant or the inputs to
    937 the spline and polynomial fits must be derived from groups of pixels
    938 on the basis of one of several possible statistics, as specified by
    939 the user parameters.  The choice of statistics must include the sample
    940 and robust mean, median, and modes.
    941 
    942 In the case of a single constant, all of the overscan pixel values are
    943 used in the calculation of this statistic.  In the case of the 1D
    944 functional representation, the input values to the fit must represent
    945 the coordinate along the overscan, with the statistic derived from the
    946 pixels in the perpendicular direction at each location. 
    947 
    948 If specified in the user parameters, sigma-clipping must be performed
    949 on the input data values. 
     1015\item The statistic used to calculate the overscan constant or the
     1016inputs to the spline and polynomial fits must be derived from groups
     1017of pixels on the basis of one of several possible statistics, as
     1018specified by the user parameters.
     1019
     1020\item The choice of statistics must include the sample and robust
     1021mean, median, and modes.
     1022
     1023\item In the case of a single constant, all of the overscan pixel
     1024values are used in the calculation of this statistic.
     1025
     1026\item In the case of the 1D functional representation, the input
     1027values to the fit must represent the coordinate along the overscan,
     1028with the statistic derived from the pixels in the perpendicular
     1029direction at each location.
     1030
     1031\item If specified in the user parameters, sigma-clipping must be
     1032performed on the input data values.
    9501033
    9511034The bias subtraction must leave no residuals greater than \tbr{1 DN}
    9521035peak-to-peak.
     1036\end{enumerate}
    9531037
    9541038\paragraph{Trim object image}
    955 
    956 The Phase 2 analysis must trim the non-imaging pixels from the image.
    957 
    958 The definition of the imaging area must be determined from the
     1039\begin{enumerate}
     1040
     1041\item The Phase 2 analysis must trim the non-imaging pixels from the
     1042image.
     1043
     1044\item The definition of the imaging area must be determined from the
    9591045Metadata Database\comment{or image header?}.
    9601046
    961 Phase 2 must trim pixel near the edges that have been compromised due
    962 to OT operation.
     1047\item Phase 2 must trim pixel near the edges that have been
     1048compromised due to OT operation.
     1049\end{enumerate}
    9631050
    9641051\paragraph{Correct for non-linearity}
    9651052
    966 If required, the science image must be corrected for the effects of
    967 non-linearity.  The correction must be a function of chip.
     1053If required by the recipe, each chip must be independently corrected for the
     1054effects of non-linearity.
    9681055
    9691056\paragraph{Flat-field correction}
    970 
    971 The Phase 2 analysis must divide the science image by the provided
    972 flat-field image.
    973 
    974 The division must handle zero-valued pixels in the flat-field image
    975 without raising floating point exceptions, setting the corresponding
    976 bit value in the mask.
    977 
    978 The flat-field images must be appropriately normalized (see section
    979 \ref{mkcal}).
    980 
    981 The flat-fielded image must have a consistent photometric zero-point
    982 across the chip, and across the full FPA, to within 0.2\% with
    983 peak-to-peak deviations of \tbr{0.5\%}.
     1057\begin{enumerate}
     1058
     1059\item The Phase 2 analysis must divide the science image by the
     1060  provided flat-field image.
     1061
     1062\item The division must handle zero-valued pixels in the flat-field
     1063  image without raising floating point exceptions, setting the
     1064  corresponding bit value in the mask.
     1065
     1066\item The flat-field images must be appropriately normalized (see
     1067  section \ref{mkcal}).
     1068
     1069\item The flat-fielded image must have a consistent photometric
     1070  zero-point across the chip, and across the full FPA, to within 0.2\%
     1071  with peak-to-peak deviations of \tbr{0.5\%}.
     1072\end{enumerate}
    9841073
    9851074\tbd{color of stars in flat-field correction?}
    9861075
    9871076\paragraph{Sky \& Fringe subtraction}
    988 
    989 The Phase 2 analysis must subtract the sky (and fringe where needed)
    990 contributions from the images.
    991 
    992 The residual after the background subtraction must have an average
    993 offset of 0 counts, excluding the signal from astronomical features. 
    994 
    995 The background residuals must have peak-to-peak variations of less
    996 than \tbr{1\%} of the input background amplitude. 
    997 
    998 The background residuals must have a scatter of less than \tbr{1\%} of
    999 the input background amplitude for apertures of less than
    1000 \tbr{10~arcsec}.\comment{derived from the need for systematic errors
    1001 of better than 0.5\% and known background ranges.}
     1077\begin{enumerate}
     1078
     1079\item The Phase 2 analysis must subtract the sky (and fringe where
     1080  needed) contributions from the images.
     1081
     1082\item The residual after the background subtraction must have an
     1083  average offset of 0 counts, excluding the signal from astronomical
     1084  features.
     1085
     1086\item The background residuals must have peak-to-peak variations of
     1087  less than \tbr{1\%} of the input background amplitude.
     1088
     1089\item The background residuals must have a scatter of less than
     1090  \tbr{1\%} of the input background amplitude for apertures of less
     1091  than \tbr{10~arcsec}.\comment{derived from the need for systematic
     1092  errors of better than 0.5\% and known background ranges.}
     1093\end{enumerate}
    10021094
    10031095\paragraph{Identify `cosmic rays'}
    1004 
    1005 The Phase 2 analysis must detect cosmic rays in single images which
    1006 are brighter than a user-configurable threshold. 
    1007 
    1008 The Phase 2 analysis must mask detected cosmic rays with a unique
    1009 bit value in the mask.
    1010 
    1011 The Phase 2 analysis must extend the masked region by a
    1012 user-configurable growth factor.
    1013 
    1014 The Phase 2 analysis must perform the cosmic ray detection only if it
    1015 is required by the analysis recipe.
     1096\begin{enumerate}
     1097
     1098\item The Phase 2 analysis must detect cosmic rays in single images
     1099  which are brighter than a user-configurable threshold.
     1100
     1101\item The Phase 2 analysis must mask detected cosmic rays with a
     1102  unique bit value in the mask.
     1103
     1104\item The Phase 2 analysis must extend the masked region by a
     1105  user-configurable growth factor.
     1106
     1107\item The Phase 2 analysis must perform the cosmic ray detection only
     1108  if it is required by the analysis recipe.
     1109\end{enumerate}
    10161110
    10171111\paragraph{Find objects in the image}
    1018 
    1019 The Phase 2 analysis must perform object detection on the processed
    1020 images.
    1021 
    1022 The object detection must detect all objects above a user-configured
    1023 threshold. The threshold must be a positive value; negative values
    1024 must invoke an error.  The detection threshold must optionally be a
    1025 function of the average background flux or the local noise level.
    1026 
    1027 The object detection must measure the following object parameters:
    1028 \begin{enumerate}
    1029 \item object centroid and position errors
    1030 \item an extended object position ($x_g, y_g$)
    1031 \item instrumental PSF magnitude and error
    1032 \item local background level and error
    1033 \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) of the object
    1034   and their covariance matrix
    1035 \end{enumerate}
    1036 
    1037 Minimal object classification must be performed to distinguish objects
    1038 which are consistent with a single PSF, objects which are
    1039 inconsistently large, objects which are inconsistently small, and
    1040 objects which are saturated.
    1041 
    1042 The resulting collection of detected objects must be saved along with
    1043 the relevant image metadata (\ie filter, exposure time, etc).
     1112\begin{enumerate}
     1113
     1114\item The Phase 2 analysis must perform object detection on the
     1115  processed images.
     1116
     1117\item The object detection process must detect all objects above a
     1118  user-configured threshold.
     1119
     1120\item The threshold must be a positive value; negative values must
     1121  invoke an error.
     1122
     1123\item The detection threshold must optionally be a function of the
     1124  average background flux or the local noise level.
     1125
     1126\item The object detection must measure the following object
     1127  parameters:
     1128  \begin{enumerate}
     1129  \item object centroid and position errors
     1130  \item an extended object position ($x_g, y_g$)
     1131  \item instrumental PSF magnitude and error
     1132  \item local background level and error
     1133  \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) of the object
     1134    and their covariance matrix
     1135  \end{enumerate}
     1136
     1137\item Minimal object classification must be performed to distinguish
     1138  objects which are consistent with a single PSF, objects which are
     1139  inconsistently large, objects which are inconsistently small, and
     1140  objects which are saturated.
     1141
     1142\item The resulting collection of detected objects must be saved along
     1143  with the relevant image metadata (\ie filter, exposure time, etc).
     1144\end{enumerate}
    10441145
    10451146\paragraph{Astrometry}
    1046 
    1047 The Phase 2 analysis must match the detected objects with known
    1048 astrometric reference objects.
    1049 
    1050 The astrometric reference object coordinates must be adjusted for
    1051 proper motion.
    1052 
    1053 The reference and detected object coordinates must be fit to determine
    1054 astrometric parameters for the individual OTAs. 
    1055 
    1056 The OTA astrometric parameters must include Chebychev polynomials of the
    1057 coordinates up to 3rd order.
    1058 
    1059 The fitted number of polynomial orders must be a user-configured
    1060 parameter. 
    1061 
    1062 The Cell astrometric parameters must not be allowed to vary in the
    1063 fit. 
    1064 
    1065 The fit must be robust, rejecting outlier matches (either stars with
    1066 poorly determined proper motion or spurious matches). 
    1067 
    1068 The resulting astrometric solution must be consistent across the OTA
    1069 field to within \tbr{300 milli-arcsec}.
     1147\begin{enumerate}
     1148
     1149\item The Phase 2 analysis must match the detected objects with known
     1150  astrometric reference objects.
     1151
     1152\item The astrometric reference object coordinates must be adjusted
     1153  for proper motion.
     1154
     1155\item The reference and detected object coordinates must be fit to
     1156  determine astrometric parameters for the individual OTAs.
     1157
     1158\item The OTA astrometric parameters must include Chebychev
     1159  polynomials of the coordinates up to 3rd order.
     1160
     1161\item The fitted number of polynomial orders must be a user-configured
     1162  parameter.
     1163
     1164\item The Cell astrometric parameters must not be allowed to vary in
     1165  the fit.
     1166
     1167\item The fit must be robust, rejecting outlier matches (either stars
     1168  with poorly determined proper motion or spurious matches).
     1169
     1170\item The resulting astrometric solution must be consistent across the
     1171  OTA field to within \tbr{300 milli-arcsec}.
     1172\end{enumerate}
    10701173
    10711174\paragraph{Postage Stamps}
    1072 
    1073 The Phase 2 analysis must extract subrasters (`postage stamps')
    1074 surrounding a user-specified list of coordinates from the flattened
    1075 images.
    1076 
    1077 The postage stamp images must be saved in the IPP Image Server.
    1078 
     1175\begin{enumerate}
     1176
     1177\item The Phase 2 analysis must extract subrasters (`postage stamps')
     1178  surrounding a user-specified list of coordinates from the flattened
     1179  images.
     1180
     1181\item The postage stamp images must be saved in the IPP Image Server.
     1182\end{enumerate}
     1183 
    10791184\subsubsection{Phase 3 : exposure analysis}
    1080 
    1081 The Phase 3 analysis must use the objects detected in Phase 2, matched
    1082 with a user-specified reference photometry catalog, to determine the
    1083 image photometric zero point and zero-point variations across the
    1084 field. 
    1085 
    1086 If zero-point variations are significant \tbd{level TBD}, the
    1087 zero-point variations must be modeled with a Chebychev polynomial
    1088 correction of order 3 or less.
    1089 
    1090 The photometric nature of the FPA image must be categorized
    1091 \tbd{numerical scale?} on the basis of the zero-point consistency, the
    1092 transparency compared with recent long-term measurements in the
    1093 filter, and the external indicators of photometricity.
    1094 
    1095 The Phase 3 analysis must use the objects detected in Phase 2, matched
    1096 with an appropriate astrometric reference catalog, to improve the
    1097 distortion model used for the image.
    1098 
    1099 The resulting astrometric accuracy must be limited by the astrometric
    1100 reference catalog, ie, 250 mas for USNO-B1.0.
     1185\begin{enumerate}
     1186
     1187\item The Phase 3 analysis must use the objects detected in Phase 2,
     1188  matched with a user-specified reference photometry catalog, to
     1189  determine the image photometric zero point and zero-point variations
     1190  across the field.
     1191
     1192\item If zero-point variations are significant \tbd{level TBD}, the
     1193  zero-point variations must be modeled with a Chebychev polynomial
     1194  correction of order 3 or less.
     1195
     1196\item The photometric nature of the FPA image must be categorized
     1197  \tbd{numerical scale?} on the basis of the zero-point consistency,
     1198  the transparency compared with recent long-term measurements in the
     1199  filter, and the external indicators of photometricity.
     1200
     1201\item The Phase 3 analysis must use the objects detected in Phase 2,
     1202  matched with an appropriate astrometric reference catalog, to
     1203  improve the distortion model used for the image.
     1204
     1205\item The resulting astrometric accuracy must be limited by the
     1206  astrometric reference catalog, ie, 250 mas for USNO-B1.0.
     1207\end{enumerate}
    11011208
    11021209\subsubsection{Phase 4 : image combination}
     
    11081215
    11091216\paragraph{Extract image pixels}
    1110 
    1111 The Phase 4 analysis must determine the corresponding set of image
    1112 pixels for a given sky cell.
    1113 
    1114 The corresponding image pixels must be extracted from the input
    1115 images, using the astrometric information for each OTA and Cell to
    1116 determine the exact overlaps.
    1117 
    1118 The Phase 4 analysis must not miss any pixels in this match, and it
    1119 must read no more than 20\% more pixels than necessary from the input
    1120 images.
    1121 
    1122 The Phase 4 analysis must skip any sky cells with fewer than 5\% of
    1123 their pixels overlapping the input images.
     1217\begin{enumerate}
     1218
     1219\item The Phase 4 analysis must determine the corresponding set of
     1220  image pixels for a given sky cell.
     1221
     1222\item The corresponding image pixels must be extracted from the input
     1223  images, using the astrometric information for each OTA and Cell to
     1224  determine the exact overlaps.
     1225
     1226\item The Phase 4 analysis must not miss any pixels in this match, and
     1227  it must read no more than 20\% more pixels than necessary from the
     1228  input images.
     1229
     1230\item The Phase 4 analysis must skip any sky cells with fewer than 5\%
     1231  of their pixels overlapping the input images.
     1232\end{enumerate}
    11241233
    11251234\paragraph{Transform pixel coordinates}
    1126 
    1127 Pixels which have been extracted from the input images must be mapped
    1128 to the corresponding pixels in the sky image.
    1129 
    1130 The transformation must be based on the measured astrometric solution
    1131 for the input images relative to the reference catalog used to
    1132 generate the static sky image.
    1133 
    1134 This warping must use a locally-linear astrometric solution.
    1135 
    1136 The output image must maintain photometric consistency with the input
    1137 image to within 0.2\%. 
     1235\begin{enumerate}
     1236
     1237\item Pixels which have been extracted from the input images must be
     1238  mapped to the corresponding pixels in the sky image.
     1239
     1240\item The transformation must be based on the measured astrometric
     1241  solution for the input images relative to the reference catalog used
     1242  to generate the static sky image.
     1243
     1244\item This warping must use a locally-linear astrometric solution.
     1245 
     1246\item The output image must maintain photometric consistency with the
     1247  input image to within 0.2\%.
     1248\end{enumerate}
    11381249
    11391250\tbd{interpolation?  does interpolation method choice risk losing flux?}
     
    11471258
    11481259\paragraph{Image outlier pixel rejection}
    1149 
    1150 When multiple images are combined, the group of input pixels which
    1151 contribute to an output pixel must be examined and pixels from the
    1152 group of images which are inconsistent with the ensemble \tbd{how
    1153 much?} must be identified and flagged. 
    1154 
    1155 This outlier rejection must be performed optionally.
     1260\begin{enumerate}
     1261
     1262\item When multiple images are combined, the group of input pixels
     1263  which contribute to an output pixel must be examined and pixels from
     1264  the group of images which are inconsistent with the ensemble
     1265  \tbd{how much?} must be identified and flagged.
     1266
     1267\item This outlier rejection must be performed optionally.
    11561268
    11571269\tbd{for moving objects and images which are not simultaneous, do we
     
    11601272\tbd{use the spatial information?  fit a 2-D Nth order polynomial to
    11611273  the collection of pixels and then look for outliers}
     1274\end{enumerate}
    11621275
    11631276\paragraph{Initial cleaned image}
     
    11781291
    11791292\paragraph{Find objects in the image}
    1180 
    1181 The Phase 4 analysis must perform object detection on the difference
    1182 images.
    1183 
    1184 All objects in the difference image must be detected and the pixels
    1185 belonging to variable sources flagged in the input image. 
    1186 
    1187 The object detection must detect all objects above a user-configured
    1188 threshold.  Both positive and negative objects must be detected; the
    1189 specified threshold must define the absolute value of the detection
    1190 thresholds.  The detection threshold must optionally be a function of
    1191 the average background flux or the local noise level.
    1192 
    1193 The object detection must measure the following object parameters:
    1194 \begin{enumerate}
    1195 \item object centroid and position errors
    1196 \item instrumental PSF magnitude and error
    1197 \item local background level and error
    1198 \item streak L, $\phi$, $\sigma_L$, $\sigma_\phi$
    1199 \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) and their covariance matrix
    1200 \end{enumerate}
    1201 
    1202 Minimal object classification must be performed to distinguish objects
    1203 which are consistent with a single PSF, objects which are
    1204 inconsistent, and objects which are saturated. 
    1205 
    1206 The resulting collection of detected objects must be saved along with
    1207 the relevant image metadata (\ie filter, exposure time, etc).
     1293\begin{enumerate}
     1294
     1295\item The Phase 4 analysis must perform object detection on the
     1296  difference images.
     1297
     1298\item All objects in the difference image must be detected and the
     1299  pixels belonging to variable sources flagged in the input image.
     1300
     1301\item The object detection must detect all objects above a
     1302  user-configured threshold.
     1303
     1304\item Both positive and negative objects must be detected: the
     1305  specified threshold must define the absolute value of the detection
     1306  thresholds.
     1307
     1308\item The detection threshold must optionally be a function of the
     1309  average background flux or the local noise level.
     1310
     1311\item The object detection must measure the following object parameters:
     1312  \begin{enumerate}
     1313  \item object centroid and position errors
     1314  \item instrumental PSF magnitude and error
     1315  \item local background level and error
     1316  \item streak L, $\phi$, $\sigma_L$, $\sigma_\phi$
     1317  \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) and their covariance matrix
     1318  \end{enumerate}
     1319
     1320\item Minimal object classification must be performed to distinguish
     1321  objects which are consistent with a single PSF, objects which are
     1322  inconsistent, and objects which are saturated.
     1323
     1324\item The resulting collection of detected objects must be saved along
     1325  with the relevant image metadata (\ie filter, exposure time, etc).
     1326\end{enumerate}
    12081327
    12091328\paragraph{Cleaned Input Image}
    1210 
    1211 The pixels flagged as being from the difference image sources must be
    1212 masked in the input images. 
    1213 
    1214 A new, cleaned image must be constructed from the masked input images.
     1329\begin{enumerate}
     1330
     1331\item The pixels flagged as being from the difference image sources
     1332  must be masked in the input images.
     1333
     1334\item A new, cleaned image must be constructed from the masked input
     1335  images.
     1336
     1337\end{enumerate}
    12151338
    12161339\tbd{how to handle variable stars?}
    12171340
    12181341\paragraph{Find objects in the image}
    1219 
    1220 The Phase 4 analysis must perform object detection on the cleaned,
    1221 summed image.
    1222 
    1223 The object detection must detect all objects above a user-configured
    1224 threshold. The threshold must be a positive value; negative values
    1225 must invoke an error.  The detection threshold optionally must be a
    1226 function of the average background flux or the local noise level.
    1227 
    1228 The object detection must measure the following object parameters:
    1229 \begin{enumerate}
    1230 \item object centroid and position errors
    1231 \item an extended object position ($x_g, y_g$)
    1232 \item instrumental PSF magnitude and error
    1233 \item local background level and error
    1234 \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) and their
    1235   covariance matrix
    1236 \item the Petrosian radius, magnitude, axis ratio, and angle
    1237 \item the S\'ersic radius, magnitude, axis ratio, angle, and parameter $\nu$.
    1238 \end{enumerate}
    1239 
    1240 Minimal object classification must be performed to distinguish objects
    1241 which are consistent with a single PSF, objects which are
    1242 inconsistent, and objects which are saturated. 
    1243 
    1244 The resulting collection of detected objects must be saved along with
    1245 the relevant image metadata (\ie filter, exposure time, etc).
     1342\begin{enumerate}
     1343
     1344\item The Phase 4 analysis must perform object detection on the
     1345  cleaned, summed image.
     1346
     1347\item The object detection must detect all objects above a
     1348  user-configured threshold.
     1349
     1350\item The threshold must be a positive value; negative values must
     1351  invoke an error.
     1352
     1353\item The detection threshold optionally must be a function of the
     1354  average background flux or the local noise level.
     1355
     1356\item The object detection must measure the following object parameters:
     1357  \begin{enumerate}
     1358  \item object centroid and position errors
     1359  \item an extended object position ($x_g, y_g$)
     1360  \item instrumental PSF magnitude and error
     1361  \item local background level and error
     1362  \item second moments ($\sigma_{\rm min}, \sigma_{maj}$) and their
     1363    covariance matrix
     1364  \item the Petrosian radius, magnitude, axis ratio, and angle
     1365  \item the S\'ersic radius, magnitude, axis ratio, angle, and parameter $\nu$.
     1366  \end{enumerate}
     1367
     1368\item Minimal object classification must be performed to distinguish
     1369  objects which are consistent with a single PSF, objects which are
     1370  inconsistent, and objects which are saturated.
     1371
     1372\item The resulting collection of detected objects must be saved along
     1373  with the relevant image metadata (\ie filter, exposure time, etc).
     1374\end{enumerate}
    12461375
    12471376\paragraph{Image Processing Q/A}
     
    12811410\label{mkcal}
    12821411
    1283 tbd{Requirements on the speed of processing the calibration images.}
     1412\tbd{Requirements on the speed of processing the calibration images.}
    12841413
    12851414The Calibration analysis stages must construct the various types of
     
    12881417
    12891418\paragraph{bias images}
    1290 
    1291 The \code{bias} calibration stage must construct a master bias image
    1292 from a collection of raw bias images.
    1293 
    1294 The \code{bias} calibration stage must correct the input images based
    1295 on the overscan region.
    1296 
    1297 The \code{bias} calibration stage must combine the input images using
    1298 the statistic specified by the user, selected from one of the
    1299 following: sample mean, median, and mode, robust mean, median, and
    1300 mode, and the clipped mean and median.
    1301 
    1302 The \code{bias} calibration stage must construct residual images, in
    1303 which the master bias is applied to the input images.
    1304 
    1305 Outlier residual images, those for which the residual bias and
    1306 variance in the bias image are excessive ($> 1DN$), must be excluded
    1307 from the input image stack the the bias image reconstructed.
     1419\begin{enumerate}
     1420
     1421\item The \code{bias} calibration stage must construct a master bias
     1422  image from a collection of raw bias images.
     1423
     1424\item The \code{bias} calibration stage must correct the input images
     1425  based on the overscan region.
     1426
     1427\item The \code{bias} calibration stage must combine the input images
     1428  using the statistic specified by the user, selected from one of the
     1429  following: sample mean, median, and mode, robust mean, median, and
     1430  mode, and the clipped mean and median.
     1431
     1432\item The \code{bias} calibration stage must construct residual
     1433  images, in which the master bias is applied to the input images.
     1434
     1435\item Outlier residual images, those for which the residual bias and
     1436  variance in the bias image are excessive ($> 1DN$), must be excluded
     1437  from the input image stack the the bias image reconstructed.
     1438\end{enumerate}
    13081439
    13091440\paragraph{dark images}
    1310 
    1311 The \code{dark} calibration stage must construct a master dark image
    1312 from a collection of raw dark images.
    1313 
    1314 The \code{dark} calibration stage must raise an error if the input
    1315 images have exposure time which deviate by more than \tbr{2\%}.
    1316 
    1317 The \code{dark} calibration stage must correct the input dark images
    1318 for the bias.
    1319 
    1320 The \code{dark} calibration stage must combine the input images using
    1321 the statistic specified by the user, selected from one of the
    1322 following: sample mean, median, and mode, robust mean, median, and
    1323 mode, and the clipped mean and median.
    1324 
    1325 The \code{dark} calibration stage must construct residual images, in
    1326 which the master dark is applied to the input images.
    1327 
    1328 Outlier residual images, those for which the residual level and
    1329 variance are excessive ($> 1DN$), must be excluded from the input
    1330 image stack the the dark image reconstructed.
     1441\begin{enumerate}
     1442
     1443\item The \code{dark} calibration stage must construct a master dark
     1444  image from a collection of raw dark images.
     1445
     1446\item The \code{dark} calibration stage must raise an error if the
     1447  input images have exposure time which deviate by more than
     1448  \tbr{2\%}.
     1449
     1450\item The \code{dark} calibration stage must correct the input dark
     1451  images for the bias.
     1452
     1453\item The \code{dark} calibration stage must combine the input images
     1454  using the statistic specified by the user, selected from one of the
     1455  following: sample mean, median, and mode, robust mean, median, and
     1456  mode, and the clipped mean and median.
     1457
     1458\item The \code{dark} calibration stage must construct residual
     1459  images, in which the master dark is applied to the input images.
     1460
     1461\item Outlier residual images, those for which the residual level and
     1462  variance are excessive ($> 1DN$), must be excluded from the input
     1463  image stack the the dark image reconstructed.
     1464\end{enumerate}
    13311465
    13321466\paragraph{flat-field images}
    1333 
    1334 The \code{flat-field} calibration stage must construct a master
    1335 flat-field image from a collection of raw flat-field images. 
    1336 
    1337 The \code{flat-field} calibration stage must accept a group of images
    1338 from one of the following flat-field sources: dome, twilight,
    1339 night-sky.
    1340 
    1341 The \code{flat-field} calibration stage must raise an error if the
    1342 input images in a single stack used more than one of the above
    1343 flat-field sources or multiple filters.
    1344 
    1345 The \code{flat-field} calibration stage must correct the input
    1346 flat-field images for the bias and dark.
    1347 
    1348 The \code{flat-field} calibration stage must combine the input images
    1349 using the statistic specified by the user, selected from one of the
    1350 following: sample mean, median, and mode, robust mean, median, and
    1351 mode, and the clipped mean and median.
    1352 
    1353 The \code{flat-field} calibration stage must construct residual
    1354 images, in which the master flat-field is applied to the input images.
    1355 
    1356 Outlier residual images, those for which the residual level and
    1357 variance are excessive ($> 0.1$\%, or 1.02 times the Poisson limit of
    1358 the flat-field image), must be excluded from the input image stack the
    1359 the flat-field image reconstructed.
     1467\begin{enumerate}
     1468
     1469\item The \code{flat-field} calibration stage must construct a master
     1470  flat-field image from a collection of raw flat-field images.
     1471
     1472\item The \code{flat-field} calibration stage must accept a group of
     1473  images from one of the following flat-field sources: dome, twilight,
     1474  night-sky.
     1475
     1476\item The \code{flat-field} calibration stage must raise an error if
     1477  the input images in a single stack used more than one of the above
     1478  flat-field sources or multiple filters.
     1479
     1480\item The \code{flat-field} calibration stage must correct the input
     1481  flat-field images for the bias and dark.
     1482
     1483\item The \code{flat-field} calibration stage must combine the input
     1484  images using the statistic specified by the user, selected from one
     1485  of the following: sample mean, median, and mode, robust mean,
     1486  median, and mode, and the clipped mean and median.
     1487
     1488\item The \code{flat-field} calibration stage must construct residual
     1489  images, in which the master flat-field is applied to the input
     1490  images.
     1491
     1492\item Outlier residual images, those for which the residual level and
     1493  variance are excessive ($> 0.1$\%, or 1.02 times the Poisson limit
     1494  of the flat-field image), must be excluded from the input image
     1495  stack the the flat-field image reconstructed.
     1496\end{enumerate}
    13601497
    13611498\paragraph{mask images}
    1362 
    1363 The \code{mask} calibration stage must construct a bad-pixel mask from
    1364 a stack of raw flat-field images and a master flat-field image.
    1365 
    1366 The \code{mask} calibration stage must mask the pixels which are
    1367 inconsistent in the input flats by more than \tbr{1\%}, given
    1368 sufficient signal-to-noise in the input flats.
    1369 
    1370 The \code{mask} calibration stage must mask the pixels which are
    1371 consistently low or high in the input flats by more than a factor of
    1372 \tbr{3} beyond the typical pixel.
    1373 
    1374 The \code{mask} calibration stage must mask the pixels identified in a
    1375 table of bad pixels generated externally to the calibration stage.
    1376 
    1377 The \code{mask} calibration stage must use multiple bit values to
    1378 identify the different types of masked pixels.
    1379 
    1380 The \code{mask} calibration stage must raise an error if the input
    1381 images generate too many bad pixels in the mask.
     1499\begin{enumerate}
     1500
     1501\item The \code{mask} calibration stage must construct a bad-pixel
     1502  mask from a stack of raw flat-field images and a master flat-field
     1503  image.
     1504
     1505\item The \code{mask} calibration stage must mask the pixels which are
     1506  inconsistent in the input flats by more than \tbr{1\%}, given
     1507  sufficient signal-to-noise in the input flats.
     1508
     1509\item The \code{mask} calibration stage must mask the pixels which are
     1510  consistently low or high in the input flats by more than a factor of
     1511  \tbr{3} beyond the typical pixel.
     1512
     1513\item The \code{mask} calibration stage must mask the pixels
     1514  identified in a table of bad pixels generated externally to the
     1515  calibration stage.
     1516
     1517\item The \code{mask} calibration stage must use multiple bit values
     1518  to identify the different types of masked pixels.
     1519
     1520\item The \code{mask} calibration stage must raise an error if the
     1521  input images generate too many bad pixels in the mask.
     1522\end{enumerate}
    13821523
    13831524\paragraph{fringe frames}
    1384 
    1385 The \code{fringe} calibration stage must construct a master fringe
     1525\begin{enumerate}
     1526
     1527\item The \code{fringe} calibration stage must construct a master fringe
    13861528frame from a stack of raw night-time sky images or from a stack of
    13871529dome fringe frames.
    13881530
    1389 The \code{fringe} calibration stage must raise an error if the input
     1531\item The \code{fringe} calibration stage must raise an error if the input
    13901532stack consists is images generated with more than one type of fringe
    13911533source or with multiple filters.
    13921534
    1393 The \code{fringe} calibration stage must flatten the input images
     1535\item The \code{fringe} calibration stage must flatten the input images
    13941536to remove the pixel-to-pixel sensitivity variations of the detector.
    13951537
    1396 The \code{fringe} calibration stage must measure the fringe amplitude
     1538\item The \code{fringe} calibration stage must measure the fringe amplitude
    13971539on the input fringe images.
    13981540
    1399 The \code{fringe} calibration stage must scale the input fringe images
     1541\item The \code{fringe} calibration stage must scale the input fringe images
    14001542based on the fringe amplitude.
    14011543
    1402 The \code{fringe} calibration stage must combine the scaled input
     1544\item The \code{fringe} calibration stage must combine the scaled input
    14031545images using the statistic specified by the user, selected from one of
    14041546the following: sample mean, median, and mode, robust mean, median, and
    14051547mode, and the clipped mean and median.
    14061548
    1407 The \code{fringe} calibration stage must construct residual images, in
     1549\item The \code{fringe} calibration stage must construct residual images, in
    14081550which the master fringe image is applied to the input images, along
    14091551with all necessary preceding calibration images.
    14101552
    1411 The \code{fringe} calibration stage must measure the residual fringe
     1553\item The \code{fringe} calibration stage must measure the residual fringe
    14121554amplitude on the residual images.
     1555\end{enumerate}
    14131556
    14141557\paragraph{low-spatial-frequency sky models}
     
    14201563
    14211564\paragraph{Flat-field correction frame}
    1422 
    1423 The \code{flat-field correction} calibration stage must construct a
     1565\begin{enumerate}
     1566
     1567\item The \code{flat-field correction} calibration stage must construct a
    14241568flat-field correction image from dithered observations of a stellar
    14251569field.
    14261570
    1427 The \code{flat-field correction} calibration stage must construct a
     1571\item The \code{flat-field correction} calibration stage must construct a
    14281572flat-field correction image for each filter and camera independently.
    14291573
    1430 The \code{flat-field correction} calibration stage must construct a
     1574\item The \code{flat-field correction} calibration stage must construct a
    14311575correction image which makes the photometry of multiple observations
    14321576of the same stellar source consistent at different locations in the
    14331577focal plane.
    14341578
    1435 The \code{flat-field correction} calibration stage must construct
     1579\item The \code{flat-field correction} calibration stage must construct
    14361580corrected flat-field images using the measured correction.
    14371581
    1438 The \code{flat-field correction} calibration stage must determine the
     1582\item The \code{flat-field correction} calibration stage must determine the
    14391583consistency of the corrected flat-field images using the dithered
    14401584stellar field observations flattened with the corrected flat-field
    1441 image..
     1585image.
     1586\end{enumerate}
    14421587
    14431588\paragraph{Non-linearity correction frames}
    1444 
    1445 The \code{non-linear correction} calibration stage must construct a
     1589\begin{enumerate}
     1590
     1591\item The \code{non-linear correction} calibration stage must construct a
    14461592non-linear correction from a collection of images of a constant source
    14471593with varying exposure times.
    14481594
    1449 The \code{non-linear correction} calibration stage must construct a
     1595\item The \code{non-linear correction} calibration stage must construct a
    14501596non-linear correction which linearizes the detector fluxes $<0.5\%$.
    14511597
    1452 The \code{non-linear correction} calibration stage must determine the
     1598\item The \code{non-linear correction} calibration stage must determine the
    14531599saturation regime, in which the non-linear correction is no longer
    14541600consistent to $<0.5\%$.
     1601\end{enumerate}
    14551602
    14561603\subsubsection{Reference Catalog Creation}
     
    14601607future Pan-STARRS calibration.  The generation of these catalogs is
    14611608inherently a research project, and will require human control and
    1462 intervention.  The IPP is required to provide the data access,
    1463 manipulation and visualization tools needed to construct these
    1464 reference catalogs and to assess their quality.  In this section, we
    1465 list the requirements of the tools needed for this effort.
    1466 
    1467 \subsubsection{Astrometry Reference Creation}
     1609intervention.  The IPP must provide the data access, manipulation and
     1610visualization tools needed to construct these reference catalogs and
     1611to assess their quality.  In this section, we list the requirements of
     1612the tools needed for this effort.
     1613
     1614\paragraph{Astrometry Reference Creation}
    14681615
    14691616\begin{table}
     
    14871634\end{table}
    14881635
    1489 The IPP must have the capability of generating an astrometric
    1490 reference on the basis of the observations obtained by the AP survey.
    1491 The IPP must provide the analysis tools needed to generate the master
    1492 astrometric reference catalog.  Much of the required functionality is
    1493 covered by the AP Database.
    1494 
    1495 The Astrometry Reference creation tools must return the match between
     1636The IPP will generate an astrometric reference on the basis of the
     1637observations obtained by the AP survey.  The IPP must provide the
     1638analysis tools needed to generate the master astrometric reference
     1639catalog.  Much of the required functionality is covered by the AP
     1640Database.  The specific requirements for the Astrometric Reference
     1641creation are listed below:
     1642
     1643\begin{enumerate}
     1644\item The Astrometry Reference creation tools must return the match between
    14961645stars observed with Pan-STARRS and any of several astrometric
    14971646reference catalogs listed in Table~\ref{AstroRefs}.
    14981647
    1499 The tools must convert the reference catalog object coordinates to all
     1648\item The tools must convert the reference catalog object coordinates to all
    15001649of the coordinate frames of relevance in the telescope and camera
    15011650system:
    1502 \begin{itemize}
     1651\begin{enumerate}
    15031652\item Catalog to mean positions
    15041653\item Mean to apparent positions
     
    15071656\item focal plane to specific detector (OTA)
    15081657\item specific detector to detector cell
    1509 \end{itemize}
    1510 
    1511 The tools must provide the necessary calibration data: the telescope
     1658\end{enumerate}
     1659
     1660\item The tools must provide the necessary calibration data: the telescope
    15121661and camera optical distortion models and the variation of the image
    15131662PSF across the camera field, as a function of color.
    15141663
    1515 The tools must fit the observed stellar coordinates to the astrometric
     1664\item The tools must fit the observed stellar coordinates to the astrometric
    15161665reference catalog coordinates to determine improved astrometric
    15171666solutions for both the stars and the detectors. 
    15181667
    1519 The tools must determine improved telescope optical distortion models
     1668\item The tools must determine improved telescope optical distortion models
    15201669based on the astrometric solutions.
    15211670
    1522 The tools must optionally determine the fit coefficients as a function
     1671\item The tools must optionally determine the fit coefficients as a function
    15231672of position along, or with combinations of magnitude or color. 
    15241673
    1525 The fitting method must include robust outlier rejection. 
    1526 
    1527 The tools must identify objects which are detected in the catalog, but
     1674\item The fitting method must include robust outlier rejection. 
     1675
     1676\item The tools must identify objects which are detected in the catalog, but
    15281677not the science image or vice-versa.
    15291678
    1530 The tools must determine average centroiding errors for each object.
    1531 
    1532 The tools must plot the fit residuals against a wide variety of
     1679\item The tools must determine average centroiding errors for each object.
     1680
     1681\item The tools must plot the fit residuals against a wide variety of
    15331682parameters: the object positions, magnitudes, colors, etc.
    15341683
    1535 The tools must allow the fit to exclude subsets of objects from the
     1684\item The tools must allow the fit to exclude subsets of objects from the
    15361685fits on the basis of these parameters.  .
    15371686
    1538 The tools must provide coordinates of the guide stars in the same frame
     1687\item The tools must provide coordinates of the guide stars in the same frame
    15391688of reference as the normal image data.
    15401689
    1541 The tools must perform the various fitting steps for the guide stars
     1690\item The tools must perform the various fitting steps for the guide stars
    15421691rather than for the normal image data.
    1543 
    1544 \subsubsection{Photometry Reference Creation}
     1692\end{enumerate}
     1693
     1694\paragraph{Photometry Reference Creation}
    15451695
    15461696\begin{table}
     
    15611711\end{table}
    15621712
    1563 The IPP must have the capability of generating a photometric reference
    1564 on the basis of the observations obtained by the AP survey.  The IPP
    1565 must provide the analysis tools needed to generate a master
    1566 photometric reference catalog.  Much of the required functionality is
    1567 covered by the AP Database.
    1568 
    1569 The Photometry Reference creation tools must return the match between
     1713The IPP will generate a photometric reference catalog on the basis of
     1714the observations obtained by the AP survey.  The IPP must provide the
     1715analysis tools needed to generate a master photometric reference
     1716catalog.  Much of the required functionality is covered by the AP
     1717Database.  The specific requirements for the Photometric Reference
     1718creation are listed below:
     1719
     1720\begin{enumerate}
     1721\item The Photometry Reference creation tools must return the match between
    15701722stars observed with Pan-STARRS and any of several photometric
    15711723reference catalogs listed in Table~\ref{PhotoRefs}.
    15721724
    1573 The tools must convert between different photometric systems, including:
    1574 \begin{itemize}
     1725\item The tools must convert between different photometric systems, including:
     1726\begin{enumerate}
    15751727\item instrumental to nominal detector magnitude
    15761728\item nominal detector magnitude to average filter system
    15771729\item average filter system to reference photometry system
    1578 \end{itemize}
    1579 
    1580 These transformations must account for color and airmass terms. 
    1581 
    1582 The tools must measure and apply relative photometry corrections
     1730\end{enumerate}
     1731
     1732\item These transformations must account for color and airmass terms. 
     1733
     1734\item The tools must measure and apply relative photometry corrections
    15831735between images.
    15841736
    1585 The tools must determine photometric transformation fit coefficients
     1737\item The tools must determine photometric transformation fit coefficients
    15861738as a function of airmass, magnitude, color and detector coordinates,
    15871739or with combinations of the above.
    15881740
    1589 The fitting method must include robust outlier rejection.
    1590 
    1591 The tools must reject specific objects from the fit on the basis of
     1741\item The fitting method must include robust outlier rejection.
     1742
     1743\item The tools must reject specific objects from the fit on the basis of
    15921744object locations, instrumental magnitudes, observed and reference
    15931745errors, and in particular time of the observations.
    15941746
    1595 The tools must plot the fit residuals against a wide variety of
     1747\item The tools must plot the fit residuals against a wide variety of
    15961748parameters, including the object positions, magnitudes, colors, etc.
    15971749
    1598 The tools must provide photometry from the guide stars in the same
     1750\item The tools must provide photometry from the guide stars in the same
    15991751system as observations of stars from the normal imaging data.
    16001752
    1601 The tools must perform the above fitting steps for the guide stars
     1753\item The tools must perform the above fitting steps for the guide stars
    16021754rather than for the normal image data.
     1755\end{enumerate}
    16031756
    16041757\subsection{Modules}
     
    16401793\subsubsection{Image Formats}
    16411794
    1642 Certain IPP programs must be able to read and write standard FITS images.
    1643 
    1644 Certain IPP programs must be able to read and write files in modified
    1645 FITS format with Pan-STARRS definitions for non-square pixel arrays.
     1795\begin{enumerate}
     1796\item Certain IPP programs must be able to read and write standard
     1797  FITS images.
     1798
     1799\item Certain IPP programs must be able to read and write files in
     1800  modified FITS format with Pan-STARRS definitions for non-square
     1801  pixel arrays.
     1802\end{enumerate}
    16461803
    16471804\subsubsection{Table Formats}
     
    16581815externally provided reference catalogs:
    16591816
    1660 \begin{itemize}
     1817\begin{enumerate}
    16611818\item Hipparcos
    16621819\item Tycho2
     
    16671824\item USNO-Bx
    16681825\item YBx
    1669 \end{itemize}
     1826\end{enumerate}
    16701827
    16711828\subsubsection{Analysis Reference Data}
     
    16731830The IPP must store reference data describing the following entities:
    16741831
    1675 \begin{itemize}
     1832\begin{enumerate}
    16761833\item Telescopes
    16771834\item Cameras
     
    16801837\item software basic parameters
    16811838\item computer configuration
    1682 \end{itemize}
     1839\end{enumerate}
    16831840
    16841841\subsection{External Interfaces}
     
    17091866hardware requirements addressed in this section consist of:
    17101867
    1711 \begin{itemize}
     1868\begin{enumerate}
    17121869\item Total Disk Volume
    17131870\item Total Processing Power
     
    17151872\item Sustained Node Network I/O
    17161873\item Sustained Disk I/O
    1717 \end{itemize}
     1874\end{enumerate}
    17181875
    17191876The report, `The Pan-STARRS Image Processing Pipeline Computational
     
    17351892calibration frames & 1.8 TB \\
    17361893metadata db        & 0.2 TB \\
    1737 AP db          &  24 TB \\
     1894AP db              &  24 TB \\
    17381895\hline
    17391896total              & 461 TB \\
     
    17511908types of data. 
    17521909
    1753 The IPP must store all raw images from the first year from the AP and
    1754 IVP surveys.  This corresponds to 175,000 images, or 175 TB, assuming
    1755 1 GB per image and compression.  The IPP will require space for 200 TB
    1756 of raw imagery to store the data from these two survey components
    1757 along with raw calibration, test, and other raw images not in the AP
    1758 and IVP surveys.
    1759 
    1760 The IPP must store a single copy of the complete static sky in all
    1761 four filters.  With the assumed image sampling of 0.2 arcsec per
    1762 pixel, this corresponds to 9.7 Tpix per filter, or a total of 235 TB
    1763 for the 6 filters, with 2 bytes for the noise map and 2 bytes for the
    1764 image map. 
    1765 
    1766 The IPP must also store other, smaller collections of data.  The other
    1767 components contribute only a small fraction of the data storage
    1768 requirement.  The metadata is a fraction of a terabyte, while the
    1769 calibration frames (all master detrend frames) represent at most a few
    1770 terabytes.  The AP object and detection data make up a total of 24
    1771 terabytes (see Table~\ref{APrates}). 
    1772 
    1773 The IPP must have storage capacity for a total of 461 TB of data.
     1910\begin{enumerate}
     1911\item The IPP must store all raw images from the first year from the
     1912  AP and IVP surveys.  This corresponds to 175,000 images, or 175 TB,
     1913  assuming 1 GB per image and compression.  The IPP will require space
     1914  for 200 TB of raw imagery to store the data from these two survey
     1915  components along with raw calibration, test, and other raw images
     1916  not in the AP and IVP surveys.
     1917
     1918\item The IPP must store a single copy of the complete static sky in
     1919  all four filters.  With the assumed image sampling of 0.2 arcsec per
     1920  pixel, this corresponds to 9.7 Tpix per filter, or a total of 235 TB
     1921  for the 6 filters, with 2 bytes for the noise map and 2 bytes for
     1922  the image map.
     1923
     1924\item The IPP must also store other, smaller collections of data.  The
     1925  other components contribute only a small fraction of the data
     1926  storage requirement.  The metadata is a fraction of a terabyte,
     1927  while the calibration frames (all master detrend frames) represent
     1928  at most a few terabytes.  The AP object and detection data make up a
     1929  total of 24 terabytes (see Table~\ref{APrates}).
     1930
     1931\item The IPP must have storage capacity for a total of 461 TB of data.
     1932\end{enumerate}
    17741933
    17751934\subsubsection{CPU Requirements}
     
    17881947be addressed independently. 
    17891948
    1790 The IPP must perform the Phase 2 analysis within an average time of 45
     1949\begin{enumerate}
     1950\item The IPP must perform the Phase 2 analysis within an average time of 45
    17911951seconds per single Gigapixel camera image.  The Phase 2 analysis has
    17921952been measured to require 3200 GHz-sec on a x86/32 bit machine,
     
    17941954are devoted to I/O.
    17951955
    1796 The IPP must perform the Phase 4 analysis on a set of 4 input frames
     1956\item The IPP must perform the Phase 4 analysis on a set of 4 input frames
    17971957within an average time of 180 seconds.  The Phase 4 analysis has been
    17981958measured to require a total of 7800 GHz-sec on an x86/32 bit machine
    17991959for a major frame of 4 input Gigapixel camera images. 
     1960\end{enumerate}
    18001961
    18011962\subsubsection{Network I/O Requirements}
     
    18081969and updated static sky images, the total I/O for a 180 second
    18091970major-frame period is:
    1810 \begin{itemize}
     1971\begin{enumerate}
    18111972\item 8 GB from summit to Phase 2 (4 images @ 2 GB each)
    18121973\item 18 GB from Phase 2 to Phase 4 (3 bytes per pixel for image +
     
    18151976  input image pixel, 4 bytes per pixel).
    18161977\item 9 GB from Phase 4 to Static Sky
    1817 \end{itemize}
     1978\end{enumerate}
    18181979for a grand total of 44 GB over 180 seconds, or 244 MB/second, of
    18191980which 26 GB are processed by the Phase 2 nodes and 36 are processed by
     
    18261987read from and written to disk. For each major frame processed, the
    18271988total I/O to and from disk for Phase 2 is:
    1828 \begin{itemize}
     1989\begin{enumerate}
    18291990\item 8 GB raw image from summit to Phase 2 nodes (4 images @ 2 GB each)
    18301991\item 8 GB raw image from Phase 2 disk to memory
     
    18331994  + 1 byte mask).
    18341995\item 18 GB processed image from Phase 2 disk to Phase 4
    1835 \end{itemize}
     1996\end{enumerate}
    18361997for a grand total of 86 GB I/O for Phase 2.  Equivalently, for each
    18371998major frame processed, the total I/O to and from disk for Phase 4 is:
    1838 \begin{itemize}
     1999\begin{enumerate}
    18392000\item 18 GB processed image from Phase 2 disk to Phase 4
    18402001\item  9 GB static image from Phase 4 disk to memory
    18412002\item  9 GB static image from memory to Phase 4 disk
    1842 \end{itemize}
     2003\end{enumerate}
    18432004for a total of 36 GB I/O for Phase 4. 
    18442005
     
    18762037
    18772038\subsection{Software Integrity Tests}
    1878 
    1879 MHPCC must test that the code does not produce memory leaks.
    1880 
    1881 MHPCC must test that the code does not produce segmentation faults.
     2039\begin{enumerate}
     2040
     2041\item MHPCC must test that the code does not produce memory leaks.
     2042
     2043\item MHPCC must test that the code does not produce segmentation faults.
     2044\end{enumerate}
    18822045
    18832046\subsection{Basic Unit Tests}
Note: See TracChangeset for help on using the changeset viewer.