Changeset 320

Timestamp:

Mar 31, 2004, 11:27:48 AM (22 years ago)

Author:

eugene

Message:

major revision of the text for sections on:
memory, logging, tracing

File:

• trunk/doc/pslib/psLibSDRS.tex (modified) (107 diffs)

trunk/doc/pslib/psLibSDRS.tex

@@ -1 @@
-%%% $Id: psLibSDRS.tex,v 1.14 2004-03-30 05:40:01 price Exp $
+%%% $Id: psLibSDRS.tex,v 1.15 2004-03-31 21:27:48 eugene Exp $
 \documentclass[panstarrs]{panstarrs}
 %\documentclass[panstarrs]{panstarrs}
 
 % basic document variables
-\title{Pan-STARRS IPP Library Design}
+\title{Pan-STARRS IPP Library\\
+Supplementary Design Requirements}
 \author{Paul Price, Eugene Magnier, Robert Lupton}
 \shorttitle{PSLib Design}
@@ -24 +25 @@
 % use \hline between each table row
 \RevisionsStart
-% version     Date         Description
-\theversion & 2003 Mar 11-- & Draft \\
+% version  Date            Description
+DR-1       & 2003 Mar 11 & Draft \\
+\hline
+DR-2       & 2003 Mar 29 & Reorganized, spell-checked \\
 \RevisionsEnd
 
-\pagebreak
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\DocumentsInternal
+PSCD-430-xxx  &   PS-1 Design Reference Mission \\
+\hline
+PSCD-430-004  &   Pan-STARRS IPP C Code Conventions \\
+\hline
+PSCD-430-005  &   Pan-STARRS IPP SRS \\
+\hline
+PSCD-430-006  &   Pan-STARRS IPP ADD \\
+\hline
+PSCD-430-007  &   Pan-STARRS IPP PSLib SDR \\
+\hline
+PSCD-430-008  &   Pan-STARRS IPP Architecture SDR \\
+
+\DocumentsExternal
+Posix Standard & Open Group Based Specifications Issue 6, IEEE Std 1003.1, 2003 \\
+\DocumentsEnd
+
 \tableofcontents
-
 \pagebreak 
 \pagenumbering{arabic}
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\section{\PS{} Library (PSLib)}
-
-PSLib is a library of basic functions required for IPP (and MOPS)
-operations.  We expect that it will be, to a large extent, the major
-workhorse of the IPP.
-
-PSLib will consist of a variety of data structures and associated APIs
-to perform common functions.  It will provide low-level capabilities
-in each of the following areas:
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Introduction}
+
+This document describes the Pan-STARRS Image Processing Pipeline (IPP)
+Toolkit Library, PSLib.  Any large software project such as the IPP
+benefits from the existence of a library of basic software functions
+which can be used throughout the project to simplify programming
+tasks.  Among the benefits are the ability to reuse code,
+simplification of the testing process, streamlining of the code, and
+the isolation and encapsulation of concepts which may be subject to
+change.  The component functions of such a library should represent
+well-defined, concise operations which can be coded with only a modest
+number of lines.  PSLib is a library of basic functions required by
+the IPP, but should include many programming concepts which may be useful
+for other software projects, especially those which deal with
+astronomical data handling tasks.
+
+PSLib consists of a collection of library function calls -- the
+Application Programming Interfaces (APIs) -- and the associated data
+structures.  The capabilities provided by PSLib are grouped into the
+following areas:
+%
 \begin{itemize}
-\item System utilities;
-\item Data containers;
-\item Data manipulation; and
-\item Astronomy-specific tasks.
+\item System Utilities
+\item Basic Data Collections
+\item Data Manipulation
+\item Astronomy-Specific Functions.
 \end{itemize}
-
-Below we sketch out approximately 200~APIs which specify the required
-capabilities.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+This list is sorted in a hierarchical order: the later entries depend
+on the functions and data types of the earlier entries.  
+
+The installed base of code for PSLib consists of header files, the
+binary library code, \code{libpslib.a} and the shared-library
+equivalent, \code{libpslib.so}.  Assuming these components have been
+installed into the library and search path, PSLib may be used within a
+program by including the line \code{#include <pslib.h>} into the C
+code and linking with \code{-lpslib}.
+
+This document describes the data structures and details the functions
+calls. The specified data structures and functions follow the naming
+conventions detailed in the IPP Code Conventions (PSDC-430-004).  In
+particular, these coding conventions restrict the namespace used by
+the library functions by requiring that all globally visible symbols
+start with the two letters \code{ps}.  Further namespace organization
+is achieved by encouraging functions to be named in the form
+psNounVerbPhrase, where Noun is the data type of principle relevance
+and VerbPhase describes the operation applied to that data type.  For
+example, the function which copies an image (of type \code{psImage})
+is called \code{psImageCopy()}.
+
+\tbd{discuss the use of external libraries (list them)}
+
+\tbd{discussion of thread safe?}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{System Utilities}
@@ -70 +120 @@
 The \PS{} software system will need a level of memory management
 placed between the operating system (\code{malloc}/\code{free}) and
-the high level routines (e.g.\ \code{psMetaDataAlloc}).
-
-This layer is in addition to the possibility that specific heavily
-used data types may need their own special-purpose memory managers;
-but as we have specified that all user-level objects be allocated via
-\code{typeAlloc/typeFree} functions, we will easily be able to
-implement such functionality without impacting the facilities
-described here.
+the high level routines (e.g.\ \code{psMetaDataAlloc}).  This layer is
+in addition to the possibility that specific heavily used data types
+may need their own special-purpose memory managers.  However, since we
+require that all user-level objects be allocated via associated
+\code{Alloc/Free} functions, we will easily be able to implement such
+functionality without impacting the facilities described here.
 
 \subsubsection{Rationale}
@@ -107 +155 @@
 \item
   While debugging complex scientific code, it is very useful to be
-  able to trace a given data structure as it passed through the
+  able to trace a given data structure as it passes through the
   processing pipeline.
 
@@ -115 +163 @@
 \end{itemize}
 
-\subsubsection{A Minimal Specification}
-
-The previous section laid out a number of desiderata for a memory
-management system.  Rather than implement our own heap-manager at this
-stage of the project, we propose to put in place a sufficient set of
-data structures and APIs, but initially implement only a simple subset
-(e.g.\ that we not implement a multi-bucket memory manager that takes
-in 100Mb chunks from the system and performs sophisticated
-defragmentation and garbage collection).
-
-Subject to agreement with the IfA, some of the functions proposed
-here may be omitted from the initial implementation; in particular
-the routines \code{psMemCheckCorruption} and \code{psMemCheckLeaks}
-may be hard to build efficiently in a simple implementation based
-upon adding extra fields to allocation requests.
-
-\paragraph{Extra information to be saved by Memory Allocation Functions}
-
-It is required that a table of all allocated memory blocks be
-maintained by the \PS{} memory system.
-
-Each allocated block should preserve at least the information present
-in the following struct definition:
+\subsubsection{Memory Management}
+
+In the following sections, we specify the API set, and define the
+appropriate data structures, needed by the PSLib memory management
+system in order to meet the requirements specified by the desiderata
+listed above.  
+
+Within the PSLib memory management system, every allocated memory
+block which is provided to the user is bounded by two additions memory
+segments.  The segment preceeding the user-memory contains data
+describing the allocated block, using the \code{psMemBlock} structure.
+The final element of this structure is a \code{void} pointer called
+\code{magic} and is assigned a special value, \code{PS_MEM_MAGIC}.
+The segment following the user-memory block consists of a single
+\code{void} pointer, and is also assigned the special value of
+\code{PS_MEM_MAGIC}.
+
+In practice, these bounding memory blocks mean that when a user is
+requests $N$ bytes of memory, the memory management system in fact
+allocates \code{N + sizeof(psMemBlock) + sizeof(void)} bytes, starting
+at a paricular address, \code{ADDR}.  It then fills in the first
+\code{sizeof(psMemBlock)} bytes with the data of the \code{psMemBlock}
+structure, and the last \code{sizeof(void)} bytes with the
+\code{PS_MEM_MAGIC}.  It returns to the user the pointer corresponding
+to the address \code{ADDR + sizeof(psMemBlock)}.  If the memory
+management system reallocates a block of memory, it must also allocate
+the additional space and fill in the boundary values.  If the memory
+management system is give a specific pointer for some operation, it is
+able to find the corresponding \code{psMemBlock} by simply subtracting
+\code{sizeof(psMemBlock)} from the pointer address.
+
+The purpose of the two boundary markers is to catch corruption and to
+act as an aid in low-level debugging.  In the first case, memory over-
+and under-run errors are likely to overwrite the special values in
+either the leading or trailing boundaries.  The typical situation is
+one where the coder mis-counts the range and either fills the data
+just before the start of the valid memory or just after the end of the
+valid memory.  These actions will alter the boundary-post values and
+can be detected by the memory management system.  In the second case,
+hexidecimal dumps of large blocks of memory are easier to examine if
+the value of \code{PS_MEM_MAGIC} is chosen to catch the eye.  A
+traditional value for \code{PS_MEM_MAGIC} is \code{0xdeadbeef} which
+is also easily recognized in a dump of the memory table.  
+
+The PSLib memory management system must maintain a private table of
+the allocated memory blocks.  The table includes a list of pointers to
+structures of type \code{psMemBlock}, defined as follows:
+%
 \begin{verbatim}
 typedef struct {
-    const void *magic0;                 //!< initialised to p_psMEMMAGIC
-    const unsigned long id;             //!< a unique ID for this allocation
-    const char *file;                   //!< set from __FILE__ in e.g. p_psAlloc
-    const int lineno;                   //!< set from __LINE__ in e.g. p_psAlloc
-    int refCounter;                     //!< how many times pointer is referenced
-    const void *magic;                  //!< initialised to p_psMEMMAGIC
+    const unsigned long id;             ///< a unique ID for this allocation
+    const char *file;                   ///< set from __FILE__ in e.g. p_psAlloc
+    const int lineno;                   ///< set from __LINE__ in e.g. p_psAlloc
+    int refCounter;                     ///< how many times pointer is referenced
+    int state;                          ///< current state of memory block
+    const void *magic;                  ///< initialised to PS_MEM_MAGIC
 } psMemBlock;
 \end{verbatim}
-
-The value of \code{P_PS_MEMMAGIC} shall be \code{(void *)0xdeadbeef}%
-\footnote{Why this choice? Tradition, and because it's easy to notice
-  in a hex dump.}
-
-\paragraph{APIs for using Memory Allocation Functions}
-
-The types and function prototypes for the part of the memory API
-concerned with allocating and freeing memory are shown below.
-
-\begin{verbatim}
-/// Memory allocation. Underlying private function called by macro psAlloc. 
-void *p_psAlloc(size_t size,            //!< Size required
-                const char *file,       //!< File of call
-                int lineno              //!< Line number of call
-                );
-
-/// Memory re-allocation.  Underlying private function called by macro psRealloc. 
-void *p_psRealloc(void *ptr,            //!< Pointer to re-allocate
-                  size_t size,          //!< Size required
-                  const char *file,     //!< File of call
-                  int lineno            //!< Line number of call
-                  );
-
-/// Free memory.  Underlying private function called by macro psFree. 
-void p_psFree(void *ptr,                //!< Pointer to free
-              const char *file,         //!< File of call
-              int lineno                //!< Line number of call
-              );
-
-/// Memory allocation. psAlloc sends file and line number to p_psAlloc. 
-#define psAlloc(size) p_psAlloc(size, __FILE__, __LINE__)
-
-/// Memory re-allocation.  psRealloc sends file and line number to p_psRealloc. 
-#define psRealloc(ptr, size) p_psRealloc(ptr, size, __FILE__, __LINE__)
-
-/// Free memory.  psFree sends file and line number to p_psFree. 
-#define psFree(size) p_psFree(size, __FILE__, __LINE__)
-\end{verbatim}
-
-N.b.
+%
+The first element in the structure is a sequential memory block ID.
+The memory management system must maintain an internal memory block ID
+counter from which a new ID may be supplied to each newly allocated
+block of memory and saved in the element \code{id}.  This ID should
+also be the key to the memory block in the memory block table.
+
+The two entries \code{file} and \code{lineno} are set to the line
+number and file at which the memory was originally allocated.  This is
+most easily implemented by the use of the C preprocessor macros
+\code{__LINE__} and \code{__FILE__}.  For this reason, we specify
+below that the basic memory managment functions be implemented as
+preprocessor macros which wrap the intrinsic C level functions.
+
+The \code{psMemBlock} structure element \code{refCounter} is provided
+so APIs may cleanly manage multiple references to the same block of
+memory.  As discussed below, the basic free function, \code{psFree},
+is specified to free the memory block only if the reference counter is
+set to 1.  See the discussion of the \code{psDlist} and \code{psHash}
+data containers for an example of the usage.  Usage of this feature is
+strongly encouraged, but not enforced by the memory management system.
+
+The element \code{state} defines the state of the memory block.  In
+order to trace double frees and other memory errors, the memory block
+reference is not automatically deleted when the assocated memory is
+deleted.  Valid states are \code{PS_MEM_FREE} and \code{PS_MEM_ALLOC}.
+
+\subsubsection{APIs for Allocation and Freeing}
+
+PSLib provides the following APIs to create and destroy memory blocks:
+%
+\begin{verbatim}
+void *psAlloc(size_t size);
+void *psRealloc(void *ptr, size_t size);
+void *psFree(void *ptr);
+\end{verbatim}
+%
+From the user's perspective, the functions \code{psAlloc},
+\code{psRealloc}, and \code{psFree} have identical semantics to the
+standard C library functions \code{malloc}, \code{realloc}, and
+\code{free}.  In fact, these functions shall be implemented as C
+preprocessor macros which call the following private functions:
+%
+\begin{verbatim}
+void *p_psAlloc(size_t size, char *file, int line);
+void *p_psRealloc(void *ptr, size_t size, char *file, int line);
+void p_psFree(void *ptr, char *file, int line);
+\end{verbatim}
+%
+In these function calls, \code{size} is the number of bytes required
+to be allocated, \code{ptr} is the pointer to the allocated memory
+block, \code{file} is the file containing the calling function (set by
+\code{__FILE__}), and \code{line} is the calling function line number
+in the source-code file (set by \code{__LINE__}).  Because the user
+should only see the preprocessor versions (ie, \code{psAlloc}), we do
+not distinguish between these two in the following discussion.
+
+In order to enforce the use of the PSLib versions, the header file
+shall take steps to ensure that code calling the functions
+\code{malloc}, \code{calloc}, \code{realloc}, or \code{free} will not
+compile.  This may be achieved by defining preprocessor macros which
+mask these functions with invalid statements (\eg{} \code{#define
+malloc(S) for}).  In exceptional cases, such as the memory management
+system itself, programmers may choose to override this prohibition by
+defining the symbol \code{PS_ALLOW_MALLOC}.  Application code will
+call \code{p_psAlloc,p_psRealloc,p_psFree} via the macros defined
+above.
+  
+The functions \code{psAlloc} and \code{psRealloc} shall never return a
+\code{NULL} pointer. If they are unable to provide the requested
+memory they should attempt to obtain the desired memory by calling the
+routine registered by \code{psMemExhaustedSetCB} (see subsubsection
+\ref{secMemAdvanced}), and if still unsuccessful, call \code{psAbort()}.
+
+Note that we have not specified an equivalent of the \code{calloc}
+function.  The \code{calloc} function provides two aspects which
+\code{malloc} originally did not: aligned memory and inialization.
+Neither of these are required: under POSIX, \code{malloc} is required
+to be aligned.  Also, for all structures it is necessary to explicity
+define the initialization independently since a byte value of 0 is
+usually insufficient.
+
+\subsubsection{Callback Routines}
+
+The PSLib memory management system uses callback functions to handle
+certain errors and trace conditions.  The callbacks are registered by
+the programmer and called by the basic memory management functions if
+needed.  The four callbacks currently defined are called in the
+following situations:
+%
 \begin{itemize}
-\item
-  The functions \code{psAlloc}, \code{psRealloc}, and \code{psFree}
-  are defined, and are required to be equivalent to \code{p_psAlloc},
-  \code{p_psRealloc}, and \code{p_psFree} with the final two arguments
-  \code{__FILE__} and \code{__LINE__}.
-
-  In the descriptions that follow, I shall not distinguish between the
-  functions with and without the \code{p_} prefix.
-
-\item
-  Except as specified below, the functions \code{psAlloc},
-  \code{psRealloc}, and \code{psFree} have identical semantics
-  to the standard C library functins \code{malloc}, \code{realloc},
-  and \code{free}.
-  
-\item
-  The header file (\file{psMemory.h}) shall take steps to ensure that
-  code calling the functions \code{malloc}, \code{calloc},
-  \code{realloc}, or \code{free} shall not compile (\eg{}
-  \code{#define malloc(S) for}) unless the symbol
-  \code{PS_ALLOC_MALLOC} is defined.
-
-\item
-  In all cases, application code will call
-  \CODE|p_{psAlloc,psRealloc,psFree}| via the macros defined above.
-  
-\item
-  The functions \code{psAlloc} and \code{psRealloc} shall never
-  return a \code{NULL} pointer. If they are unable to provide
-  the requested memory they should attempt to obtain the desired
-  memory by calling the routine registered by \code{psMemExhaustedSetCB} (see
-  subsubsection \ref{secMemAdvanced}), and if still unsuccessful,
-  call \code{abort()}.
-
-\item
-  The memory management routines shall maintain the field
-  \code{psMemBlock.refcntr}. It should be set to \code{1}
-  when a block is returned to the user. It is an error to
-  attempt to free a block with \code{psMemBlock.refcntr != 1}
-  (see subsubsection \ref{secMemRefcounter}).
-  
-\item
-  Where practical and efficient, the memory manager shall call the
-  routine registered using the \code{psMemProblemSetCB} (see section
-  \ref{secMemAdvanced}) whenever a corrupted block of memory is
-  discovered. For example, doubly-freed blocks can be detected by
-  checking \code{psMemBlock.refcntr}.
-
-\item
-  There is no \code{psCalloc} function. Initialisation of data is
-  almost always more complex than setting all bytes to 0.
-  Correspondingly, the system call \code{calloc} should be disabled
-  using the pre-processor.
+\item when insufficient memory is available (\code{psMemExhaustedCB})
+\item when a memory block is found to be corrupted (\code{psMemProblemCB})
+\item when a specified memory ID is allocated (\code{psMemAllocateCB})
+\item when a specified memory ID is freed (\code{psMemFreeCB})
 \end{itemize}
-
-\paragraph{APIs for Tracing and Debugging Memory}
+%
+The callback functions are defined in terms of specific callback
+types, specified below.  The callbacks are set using functions with
+names of the form \code{psCallbackSet}.  In all cases, the
+`\code{Set}' routine takes a pointer to the desired callback
+function and returns a pointer to the one that was previously
+installed. If the function pointer is \code{NULL}, the default
+callback function is reinstalled.  We discuss the use of each of the
+four callbacks below.
+
+\subsubsubsection{\tt psMemExhaustedCB}
+
+If not enough memory is available to satisfy a request by
+\code{psAlloc} or \code{psRealloc}, these functions attempt to find an
+alternative solution by calling the \code{psMemExhaustedCB}, a
+function which may be set by the programmer in appropriate
+circumstances, rather than immediately fail.  The typical use of such
+a feature may be when a program needs a large chunk of memory to do an
+operation, but the exact size is not critical.  This feature gives the
+programmer the opportunity to make a smaller request and try again,
+limiting the size of the operating buffer.  This callback has the
+following form:
+%
+\begin{verbatim}
+typedef void *(*psMemExhaustedCB)(size_t size);
+psMemExhaustedCB psMemExhaustedCBSet(psMemExhaustedCB func);
+\end{verbatim}
+%
+The callback function is called with the attempted size and is
+expected to return a pointer to the allocated memory or \code{NULL}.
+Until the callback function is set with \code{psMemExhaustedCBSet},
+the default callback function immediately returns \code{NULL}, in
+which case \code{psAlloc} will call \code{psAbort}.
+
+\subsubsubsection{\tt psMemProblemCB}
+
+At various occasions, the memory manager can check the state of the
+memory stack.  If any of these checks discover that the memory stack
+is corrupted, the \code{psMemProblemCB} is called.  The callback
+has the following form:
+%
+\begin{verbatim}
+typedef void (*psMemProblemCB)(psMemBlock *ptr, char *file, int lineno);
+psMemProblemCB psMemProblemCBSet(psMemProblemCB func);
+\end{verbatim}
+%
+This callback may be used to report the error and other status
+information.  No return value is accepted, and no specific operations
+are expected.  The callback is for informational purposes only.  Where
+practical and efficient, the memory manager shall call the routine
+registered using the \code{psMemProblemCBSet} whenever a corrupted
+block of memory is discovered.  For example, doubly-freed blocks can
+be detected by checking \code{psMemBlock.refcntr}.
+
+\subsubsubsection{\tt psMemAllocateCB \& psMemFreeCB}
+
+Two private variables, \code{p_psMemAllocateID} and
+\code{p_psMemFreeID}, can be used to trace the allocation and freeing
+of specific memory blocks.  If the first (\code{p_psMemAllocateID}) is
+set and a memory block with that ID is allocated, the corresponding
+callback is called just before memory is returned to the calling
+function.  If the second (\code{p_psMemFreeID}) is set and the memory
+block with the ID is about to be freed, the corresponding callback is
+called just before the memory block is freed.  These variables are
+internal and private to the memory manager and should be set using the
+following two functions:
+%
+\begin{verbatim}
+long psMemAllocateCBSetID(long id);
+long psMemFreeCBSetID(long id);
+\end{verbatim}
+%
+The corresponding callback have the following form:
+%
+\begin{verbatim}
+typedef long (*psMemAllocateCB)(const psMemBlock *ptr);
+psMemAllocateCB psMemAllocateCBSet(psMemAllocateCB func);
+typedef long (*psMemFreeCB)(const psMemBlock *ptr);
+psMemFreeCB psMemFreeCBSet(psMemFreeCB func);
+\end{verbatim}
+%
+The callback functions are called with a pointer to the corresponding
+memory block.  The routines \code{psMemFreeIDSet} and
+\code{psMemAllocateIDSet} accept the desired ID value and return the
+old value to the user.  The return values of the handlers installed by
+\code{psMemAllocateCBSet} and \code{psMemFreeCBSet} are used to
+increment the values of \code{p_psMemAllocateID} and
+\code{p_psMemFreeID} respectively.  For example, the return value
+\code{0} implies that the value is unchanged; if the value is \code{2}
+the callback will be called again when the memory ID counter has
+increased by two.  This functionality may be useful to check, for
+example, every 100th block allocated.  The function, \code{long
+psMemGetId(void);} returns the next identification number to be
+assigned to a memory block.  This function can be used to guide the
+choice of ID set with the functions above.
+
+\subsubsection{Memory Tracing and Corruption Checks}
 \hlabel{secMemAdvanced}
 
-The types and function prototypes for this part of the memory API
-are shown below.
-
-\begin{verbatim}
-/// prototype of a basic callback used by memory functions 
-typedef int (*psMemCallback)(const psMemBlock *ptr);
-
-/// prototype of a callback used in error conditions 
-typedef void (*psMemProblemCallback)(const psMemBlock *ptr, const char *file, int lineno);
-
-/// prototype of a callback used when memory runs out 
-typedef void *(*psMemExhaustedCallback)(size_t size);
-
-/// Set callback for problems 
-psMemProblemCallback psMemProblemSetCB(psMemProblemCallback func //!< Function to run
-                                       );
-
-/// Set callback for out-of-memory 
-psMemExhaustedCallback psMemExhaustedSetCB(psMemExhaustedCallback func //!< Function to run
-                                           );
-
-/// Set call back for when a particular memory block is allocated 
-psMemCallback psMemAllocateSetCB(psMemCallback func //!< Function to run
-                                 );
-
-/// Set call back for when a particular memory block is freed 
-psMemCallback psMemFreeSetCB(psMemCallback func
-                             );
-
-/// get next memory ID 
-int psMemGetId(void);
-
-/// set p_psMemAllocateID to id 
-long psMemSetAllocateID(long id         //!< ID to set
-                        );
-
-/// set p_psMemFreeID to id 
-long psMemSetFreeID(long id             //!< ID to set
-                    );
-
-/// Check for memory leaks 
-int psMemCheckLeaks(int id0,            //!< don't list blocks with id < id0
-                    psMemBlock ***arr,  //!< pointer to array of pointers to leaked blocks, or NULL
-                    FILE *fd            //!< print list of leaks to fd (or NULL)
-                    );
-/// Check for memory corruption 
-int psMemCheckCorruption(int abort_on_error //!< Abort on detecting corruption?
-                         );
-\end{verbatim}
-
-\paragraph{Callback Routines}
-
-The four `\code{SetCB}' routines are:
-
-\begin{tabular}{ll|l}
-\textbf{type} &  \textbf{Name} & \textbf{Function of callback} \\
-\code{psMemProblemCallback} & \code{psMemProblemSetCB} &
-Called when a problem is detected with data being managed on the heap \\
-\code{psMemExhaustedCallback} & \code{psMemExhaustedSetCB} &
-Called when \code{psAlloc} is unable to satisfy a memory request. \\
-\code{psMemCallback} & \code{psMemAllocateSetCB} &
-Callback is called when the \code{psMemBlock} with a specified ID is
-allocated. \\
-\code{psMemCallback} & \code{psMemFreeSetCB} &
-Callback is called when the \code{psMemBlock} with a specified ID is
-freed. \\
-\end{tabular}
-
-N.b.
-\begin{itemize}
-\item
-  In all cases, the `\code{SetCB}' routine takes a pointer to the
-  desired callback function, and returns a pointer to the one that was
-  previously installed. If the function pointer is \code{NULL}, the
-  default callback function is reinstalled.
-
-\item
-  The routine installed by \code{psMemExhaustedSetCB} should return
-  a pointer to the desired memory or \code{NULL}; in the latter case
-  \code{psAlloc} will call \code{abort()}
-
-\item
-  \code{psMemGetId} returns the next identification number to be
-  assigned to a \code{psMemBlock.id}.
-
-\item
-  The handler specified by \code{psMemAllocateSetCB} is called just
-  before the pointer with \code{psMemBlock.id} equal to
-  \code{p_psMemAllocateID} is returned to the user. The variable
-  \code{p_psMemAllocateID} should not be set directly in any
-  delivered code, \code{psMemSetAllocateID} should be used instead.
-
-\item
-  The handler specified by \code{psMemFreeSetCB} is called just
-  before the pointer with \code{psMemBlock.id} equal to
-  \code{p_psMemFreeID} is freed. The variable
-  \code{p_psMemFreeID} should not be set directly in any delivered
-  code, \code{psMemSetFreeID} should be used instead.
-
-\item
-  The routines \code{psMemSetFreeID} and \code{psMemSetAllocateID}
-  accept the desired ID value (see previous two items) and return
-  the old value to the user.
-
-\item
-  The return values of the handlers installed by \code{psMemAllocateSetCB}
-  and \code{psMemFreeSetCB} are used to increment the values of
-  \code{p_psMemAllocateID} and \code{p_psMemFreeID} respectively.
-
-  For example, the return value \code{0} implies that the value is unchanged;
-  if the value is \code{2} the callback will be called again when the
-  memory ID counter has increased by two.
-\end{itemize}
-
-\paragraph{Memory Tracing and Corruption Routines}
-
+The PSLib memory management system includes features to facilitate
+tracing the memory allocation and freeing process and to debug memory
+errors in the calling code.  The types and function prototypes for
+this part of the memory API are shown below.
+%
+\begin{verbatim}
+int psMemCheckLeaks(long id0, psMemBlock ***array, FILE *fd);
+int psMemCheckCorruption(int abort_on_error);
+\end{verbatim}
+%
 The routine \code{psMemCheckLeaks} may be used to check for memory
 leaks. The return value is the number of blocks that have been
-allocated but not freed.
-
-Only blocks with \code{psMemBlock.id} greater than \code{id0}
-are checked; this allows the user to ignore blocks allocated
-by initialisation routines.
-
-If the argument \code{arr} is non-\code{NULL}, then upon entering
-the call \code{**arr} should be \code{NULL}. Upon return it is set
-to an array of \code{psMemBlock *} pointers, one for each block
+allocated but not freed.  Only blocks with \code{psMemBlock.id}
+greater than \code{id0} are checked; this allows the user to ignore
+blocks allocated by initialization routines.
+
+If the argument \code{array} is non-\code{NULL}, then \code{**array}
+is set to an array of \code{psMemBlock *} pointers when the function
+returns.  These pointers represent the blocks which have been
 allocated but not freed.  It is the caller's responsibility to free
-this array with \code{psFree}.
+this array with \code{psFree}. Also note that \code{**array} should be
+\code{NULL} (or not point to allocated memory) upon entering the call
+or the corresponding memory reference will be lost.
 
 If the argument \code{fd} is non\code{NULL}, a one-line summary
@@ -384 +446 @@
 The routine \code{psMemCheckCorruption} checks the entire heap for
 corruption, calling the routine registered with
-\code{psMemProblemSetCB} for each block detected as being corrupted.
+\code{psMemProblemCBSet} for each block detected as being corrupted.
 The return value is the number of corrupted blocks detected. If the
-argument \code{abort_on_error} is true,
-\code{psMemCheckCorruption} shall call \code{abort()} as soon as
-memory corruption is detected.
-
-\paragraph{Reference Counting}
+argument \code{abort_on_error} is true, \code{psMemCheckCorruption}
+shall call \code{psAbort} as soon as memory corruption is detected.
+
+\subsubsection{Reference Counting}
 \hlabel{secMemRefcounter}
 
-The memory management routines include a field
-\code{psMemBlock.refcntr} which must equal \code{1} whenever
-a pointer is presented to \code{psFree}.
-
-The API for this field is:
-\begin{verbatim}
-/// Return reference counter 
-int psMemGetRefCounter(void *vptr       //!< Pointer to get refCounter for
-                       );
-
-/// Increment reference counter and return the pointer 
-void *psMemIncrRefCounter(void *vptr    //!< Pointer to increment refCounter, and return
-                          );
-
-/// Decrement reference counter and return the pointer 
-void *psMemDecrRefCounter(void *vptr    //!< Pointer to decrement refCounter, and return
-                          );
-\end{verbatim}
-
-The functions \code{psMemIncrRefCounter} and \code{psMemDecrRefCounter} shall
-return \code{NULL} if passed a \code{NULL} pointer.
-
-\subsubsubsection{Rationale}
-
-The \code{psMemBlock.refcounter} is clearly useful for detecting
-attempts to free memory that is already free.  A more complex
-application is for allowing pointers to complex data-objects (e.g.\
-images) to appear in more than one data structure simultaneously:
-
-\begin{verbatim}
-typedef struct {
-    char *name;
-    int value;
-} psSimple;
-
-psSimple *psSimpleAlloc(const char *name, int val)
-{
-    psSimple *simp = psAlloc(sizeof(psSimple));
-    simp->name = strcpy(psAlloc(strlen(name) + 1), name);
-    simp->value = val;
-
-    return simp;
-}
-
-void psSimpleFree(psSimple *simp)
-{
-    if (simp == NULL) { return; }
-
-    if (psMemGetRefCounter(simp) > 1) {
-        (void)psMemDecrRefCounter(simp);
-        return;
-    }
-}
-\end{verbatim}
-
-
-Because of the use of the \code{refcounter} field, we can safely put items of
-this type onto many lists:
-\goodbreak
-\begin{verbatim}
-simp = psSimpleAlloc("RHL", 0);
-psDlistAppend(list1, psMemIncrRefCounter(simp));
-psDlistAppend(list2, psMemIncrRefCounter(simp));
-psSimpleFree(simp);
-\end{verbatim}
-
-(Note: in fact there is no need to explicitly increment the counter
-in this case, as the \code{psDlistAppend} (section \ref{psDlist})
-API specifies that it
-does it for you.)
+As mentioned above, the memory management system provides the
+\code{refCounter} element in \code{psMemBlock} to allow for the
+management of multiple references to the same block of memory.
+External APIs which make use of this structure must increment the
+counter for every additional reference to an allocated memory block,
+and decrement it when those references are removed.  The memory
+management routines respect the use of the \code{refCounter} field:
+\code{psFree} will not free a block for which \code{refCounter != 1},
+and \code{psAlloc} will initialize the field to 1.  \tbd{should psFree
+give an error if refCounter $>$ 1 or actually do the decrement?}
+However, they do not (and cannot practically) enforce the use of the
+counters; this is a requirement of external APIs which intend to use
+the feature.
+
+Several APIs are provided to manage the reference counters.  These
+APIs are:
+%
+\begin{verbatim}
+int psMemGetRefCounter(void *vptr);
+void *psMemIncrRefCounter(void *vptr);
+void *psMemDecrRefCounter(void *vptr);
+\end{verbatim}
+%
+The functions all take a pointer to the start of a user block of
+memory.  The first simply returns the value of the reference counter.
+The next two functions increment or decrement the reference counter,
+returning the pointer which was passed in. These functions must
+validate the memory pointer by determining the corresponding
+\code{psMemBlock.id} and checking for consistency in the internal
+memory block table (the table pointer for \code{psMemBlock.id} should
+be in the valid range and should correspond to the address of the
+\code{psMemBlock}).  For an example implementation of the
+\code{refCounter} facilities, see the discussion of \code{psDlist}
+
+\tbd{REF}.
 
 \subsection{Tracing and Logging}
@@ -468 +494 @@
 This section defines the \PS{} Tracing and Logging APIs; the former
 refers to debug information that we wish to be able to turn on and off
-without recompiling (although it will \emph{not} be available in
-production code); the latter means information about the processing
-that must be collected and saved, even in the production system.
-
-We envision that we will make extensive use of \code{psTrace} throughout
-the \PS{} code.
+without recompiling (it will \emph{not} be available in production
+code); the latter means information about the processing that must be
+collected and saved, even in the production system.  We envision that
+we will make extensive use of \code{psTrace} throughout the \PS{}
+code.
 
 \subsubsection{Tracing APIs}
 \hlabel{psTrace}
 
-\begin{verbatim}
-#if defined(PS_NTRACE)
-#  define psTrace(facil, level, ...) /* do nothing */
-#else
-#  define psTrace(facil, level, ...) \
-          p_psTrace(facil, level, __VA_ARGS__)
-#endif
-
-/// Send a trace message
-void p_psTrace(const char *facil,       ///< facilty of interest
-               int level,               ///< desired trace level
-               ...                      ///< trace message arguments 
-    );
-
-/// Set trace level 
-int psSetTraceLevel(const char *facil,  ///< facilty of interest
-                    int level           ///< desired trace level
-    );
-
-/// Get the trace level 
-int psGetTraceLevel(const char *name    ///< facilty of interest
-    );
-
-/// turn off all tracing, and free trace's allocated memory 
-void psTraceReset(void);
-
-/// print trace levels 
-void psPrintTraceLevels(void);
-\end{verbatim}
-
-\begin{itemize}
-  \item
-    Logging is provided by the function \code{psTrace},
-    which is actually a macro.  When the macro \code{PS_NTRACE}
-    is defined, all occurrences of \code{psTrace} shall
-    be replaced by whitespace.
-
-  \item
-    The first argument to \code{psTrace} is a name of the form
-    \code{aaa.bbb.ccc}. The second is an integer,
-    \code{level}. The remaining arguments are a printf-style format
-    and a (possibly empty) set of values for that formatting
-    string. When this trace is active, \code{psTrace} shall generate
-    the requested output on \code{stdout}, preceded by \code{level}
-    spaces.
-
-  \item
-    The call \code{psSetTraceLevel(name, level)} shall set the trace
-    level for \code{name} to \code{level}.
-
-  \item
-    The call \code{psGetTraceLevel(name)} shall return the level
-    associated with \code{name}.  If \code{name} has not been
-    associated with a level, the routine shall remove the \textit{last}
-    element of \code{name} and attempt to look up its level; this
-    procedure shall be applied recursively until the name is reduced
-    to \code{""}; if this name has no level associated with
-    it, then the value \code{0} shall be returned (see examples
-    below).
-
-    \item
-      The call \code{psTrace(name, level, ...)}
-      shall print the message if and only if
-      \code{psGetTraceLevel(name)} returns a value greater than
-      or equal to \code{level}.
-
-    \item
-      \code{psPrintTraceLevels} shall print a listing of all
-      declared levels, displayed as a hierarchy with sub-components
-      sorted within each component (see examples below).
-
-      Note in particular that the root of the tree, the name \code{""}
-      should print as \code{(root)}, and nodes which have not
-      been assigned a value should list their level as \code{.}.
-
-    \item
-      All of the tracing facilities shall be SWIGed, with the exception
-      of \code{p_psTrace}.
-
-    \item
-      There is no requirement that \code{psTrace} be usable from
-      within the functions that implement the tracing or memory
-      management systems.
-
-    \item
-      \code{psTraceReset} shall reset all tracing to the state that it
-      had at program initiation, including freeing any memory that the
-      tracing subsystem may have allocated.
-\end{itemize}
-
-\paragraph{Examples of the use of Tracing Facilities}
-
-For example, after the commands:
-\begin{verbatim}
-    psSetTraceLevel("utils.dlist.add.head", 9);
-    psSetTraceLevel("utils.dlist.add", 3);
-    psSetTraceLevel("utils.dlist.remove", 4);
-    psSetTraceLevel("coadd", 2);
-    psSetTraceLevel("coadd.CR.remove.morphology", 5);
-    psSetTraceLevel("utils.hash", 2);
-    psSetTraceLevel("utils.dlist.add", 9);
-    psSetTraceLevel("utils", 1);
-\end{verbatim}
-the command \code{psPrintTraceLevels()} should print:
-\begin{verbatim}
-(root)               0
- utils               1
-  hash               2
-  dlist              .
-   remove            4
-   add               9
-    head             9
- coadd               2
-  CR                 .
-   remove            .
-    morphology       5
-\end{verbatim}
-where \code{.} means that the trace level should be inherited from its parent.
-
-After this set of \code{psSetTraceLevel} commands, and if
-\code{PS_NTRACE} is not defined, the following commands
-\begin{verbatim}
-    psTrace("utils.dlist.remove", 2, "Removing psDList key \"%s\"\n", "my_key");
-    psTrace("utils", 2, "Initialising utilities library\n");
-    psTrace("", 2, "This is turned on by trace component \"\"");
-    psTrace("utils.dlist", 2, "Initialising psDList\n");
-    psTrace("utils.dlist.remove", 4, "Removing psDList key \"%s\" (value: \"%d\")\n", "my_key", 12345);
-    psTrace("utils.hash.remove", 4, "Removing hash key \"%s\" (value: \"%d\")\n", "my_key", 12345);
-    psTrace("utils.dlist.add", 1, "Adding psDList key \"%s\"\n", "your_key");
-    psTrace("utils.dlist.find", 2, "Looking up psDList key \"%s\"\n", "some_key");
-    psTrace("coadd.CR.remove", 4, "Removing CRs\n");
-    psTrace("coadd.CR.remove.morphology", 4, "CRs are not fuzzy\n");
-\end{verbatim}
-should produce this output:
-\begin{verbatim}
-  Removing psDList key "my_key"
-    Removing psDList key "my_key" (value: "12345")
- Adding psDList key "your_key"
-    CRs are not fuzzy
-\end{verbatim}
-
-Note that
-\begin{description}
-\item
-  \code{utils.dlist} messages are at level 1, inherited from \code{utils}, so the
-  \code{Initialising utilities library}, \code{Initialising psDList}, and
-  \code{Looking up psDList key} messages are \emph{not} printed (the traces are at level 2).
-
-\item
-  \code{utils.dlist.remove} messages are at level 4, and are printed.
-
-\item
-  \code{utils.hash} messages are at level 2, and are not printed (the traces are at level 4)
-
-\item
-  \code{coadd.CR.remove} is at level 2 (inherited from \code{coadd}) so \code{Removing CRs}
-  isn't printed.  \code{coadd.CR.remove.morphology} is at level 4, so \code{CRs are not fuzzy} is printed.
-\end{description}
+A code-tracing facility should allow the programmer to place messages
+in the code which, when called, will print some useful information
+about the containing block of code.  Ideally, different messages may
+be specified to have different levels (of severity or interest).  For
+a given run of the program, the level of interest should be set to
+provide more or less feedback, depending on the needs of the
+programmer.  In a typical situation, low-level messages would be
+placed generously throughout the code, indicating the flow of the
+program.  Higher-level messages would be placed in a limited number of
+special locations, such as the start of major code segments or where a
+particularly unusual condition is met.  Top-level messages would be
+placed in code triggered under serious error conditions.  A normal run
+of the program would have the trace messages printed only for the
+top-level.  If the user needs to dig deeper into the code, the trace
+level should be set lower, and the more detailed messages could be
+examined.  In a case of a real, poorly-understood problem with the
+code, the trace threshold would be placed to the bottom and the
+lowest-level step-by-step messages would be printed.
+
+The PSLib tracing facility provides the above functionality along with
+the ability to assign different trace levels to different types of
+messages.  Each trace message when placed in the code is assigned to
+be part of a specific tracing 'facility', defined in more detail
+below.  The trace level for that specific message is also set when the
+message is placed.  Each facility may have its trace level set
+independently.  Thus, it is possible to request detailed trace output
+for one facility while minimizing the verbosity of the trace output
+from the rest of the program.  
+
+The trace facilities consist of a hierarchy of names.  A trace
+facility is defined by a string consisting of words separated by dots,
+with a hierarchy equivalent to that of UNIX directory names.  The
+top-level facility is simply \code{'.'} (one dot).  The next level
+would be \code{'.A'}, followed by \code{.A.B}, and so on.  The
+relationship is seen in two ways.  First, a facility inherits the
+trace level of its parent unless specified.  Second, the hierarchy is
+used to format the listing of the trace facilities.  The first of
+these rules provides a mechanism to define the default trace levels
+for any facility even if it has not been registered explicitly since
+all named facilities are implicitly children of the top level facility
+(\code{.}).  The second rule is simply an organizational technique to
+make the listing of facility information clear.
+
+The API to place a trace message in the code, and simultaneously set
+its trace level and facility, is:
+%
+\begin{verbatim}
+void psTrace(char *facil, int myLevel,...);
+\end{verbatim}
+% 
+where the last argument is a printf-style formatting code and possible
+arguments to that formatting statement, to be implemented using the
+\code{vprintf} functions.  This command specifies the name of the
+facility to which the message belongs (\code{facil}), the trace level
+for this message in that facility (\code{myLevel}) and the message
+itself.
+
+The trace level for any facility may be set at any time with the
+following function:
+%
+\begin{verbatim}
+int psSetTraceLevel(char *facil, int level);
+\end{verbatim}
+% 
+where \code{level} specifies the current trace level for the facility
+named by \code{facil}.  The currently defined trace level for a given
+facility is determined by the function:
+%
+\begin{verbatim}
+int psGetTraceLevel(char *facil);
+\end{verbatim}
+% 
+which returns the trace level of the named facility following the
+rules specified above.  A specified trace message (identified by
+\code{psTrace}) shall be printed if and only if
+\code{psGetTraceLevel(facil)} returns a value greater than or equal to
+the value of \code{myLevel} for that message.
+
+PSLib includes a utility function for examining the current tracing
+levels of all facilities: \code{void psPrintTraceLevels(void);}.  This
+function will print the hierarchy of trace facilities along with the
+current trace level for each facility.  For example, a particular
+program may have a few facilities defined, along with their trace
+levels.  A call to \code{psPrintTraceLevels} may produce a listing
+which appears as:
+\begin{verbatim}
+.                    0
+.debias              1
+.flatten             1
+.flatten.divide      2
+.object.findpeak     1
+.object.getsky       1
+\end{verbatim}
+
+Considering the same program, the programmer might place a variety of
+trace messages throughout the \code{flatten} portion of the code with
+different types of messages, such as:
+%
+\begin{verbatim}
+psTrace("flatten", 2, "starting flatten function\n");
+psTrace("flatten", 0, "ERROR: flat-field image \%s is invalid\n", filename);
+psTrace("flatten.divide", 2, "doing the divide\n");
+psTrace("flatten.divide", 3, "trying the loop, i = \%d\n", i);
+psTrace("flatten.divide", 1, "got an invalid pixel value (NaN) at \%d,\%d\n");
+psTrace("flatten.divide", 2, "divide is done\n");
+\end{verbatim}
+%
+Under the trace levels set above, if the code actually reached each of
+these trace messages, the following messages would be printed:
+%
+\begin{verbatim}
+ERROR: flat-field image foo.fits is invalid  (printed)
+doing the divide                             (printed)
+got an invalid pixel value (NaN) at 500,20   (printed)
+divide is done                               (printed)
+\end{verbatim}
+%
+while these two would not be printed because their facility level was
+too low:
+%
+\begin{verbatim}
+starting flatten function
+trying the loop, i = 0   
+trying the loop, i = 1   
+trying the loop, i = 2   
+...
+\end{verbatim}
+%
+
+The tracing facility should be made available, or not, on compilation.
+If the C pre-processor macro \code{PS_NO_TRACE} is defined, all trace
+code shall be replaced by empty space so that none of the code is
+compiled.  This can be implemented via macro front-ends to private
+versions of the user APIs.  In addition, a function \code{void
+psTraceReset(void)} will set all trace levels to 0.
+
+\tbd{ability to send trace messages to output locations other than stdout?}
 
 \subsubsection{Message Logging}
 \hlabel{psLogMsg}
 
-\begin{verbatim}
-enum { PS_LOG_ABORT = 0, PS_LOG_ERROR, PS_LOG_WARN, PS_LOG_INFO }; //!< Status codes for log messages
-
-enum { PS_LOG_TO_STDERR, PS_LOG_TO_STDOUT }; //!< Destinations for log messages
-
-/// Logs a message
-void psLogMsg(const char *name, int level, const char *fmt, ...); 
-
-/// Logs a message from varargs
-void p_psVLogMsg(const char *name, int level, const char *fmt, va_list ap); 
-
-/// Sets the log destination 
+Message logging is similar in some respects to tracing.  Like trace
+messages, log messages are placed in the code at various locations to
+provide output describing the current state of the program.  Like
+the PSLib trace facility, a good log facility should have the
+capability of associating each message with an importance or severity
+level, and at any point, the level for which messages are actually
+printed should be set in a flexible manner.   Unlike trace messages,
+however, log messages are always part of the code and are available in
+the production version as well as in test versions.  
+
+The PSLib logging facility does not include the extensive facility
+levels which are provided by the trace facility.  Less control over
+the granularity is needed for the log messages than for the trace
+messages.  
+
+A log message is placed in the code with the command:
+%
+\begin{verbatim}
+void psLogMsg(char *name, int myLevel, char *fmt, ...); 
+void psVLogMsg(char *name, int myLevel, char *fmt, va_list ap); 
+\end{verbatim}
+where \code{name} is a word to describe the source of the message,
+\code{myLevel} is the severity level of this message, and \code{fmt}
+is a printf-style formatting statement defining the actual message,
+and is followed by the arguments to the formatting code.  The second
+form, \code{psVLogMsg} is an equivalent command which takes a
+\code{va_list} argument.
+
+A log message may have any level specified in the range 0-9, though
+the first 4 levels are associated with symbolic names:
+%
+\begin{verbatim}
+enum { PS_LOG_ABORT = 0, PS_LOG_ERROR, PS_LOG_WARN, PS_LOG_INFO };
+\end{verbatim}
+%
+The default level is set to \code{PS_LOG_INFO}.  
+
+At any time, the program may set the current log level, the level
+above which log messages are ignored, using the function:
+%
+\begin{verbatim}
+int psSetLogLevel(int level);           
+\end{verbatim}
+%
+A specific message invoked with \code{psLogMsg} is only printed if its
+value of \code{myLevel} is less than the current value set by
+\code{psSetLogLevel}.  
+
+Log messages are sent to the destination most recently set using:
+%
+\begin{verbatim}
 int psSetLogDestination(int dest);      
-
-/// Sets the log level
-int psSetLogLevel(int level);           
-
-/// sets the log format
+\end{verbatim}
+%
+The only two values that are initially defined are
+\code{PS_LOG_TO_STDERR} and \code{PS_LOG_TO_STDOUT} to write to
+\code{stderr} and \code{stdout} respectively.  \tbd{define other
+destinations? why not use file descriptors?, etc?}.
+
+The output format is controlled with the function:
+%
+\begin{verbatim}
 void psSetLogFormat(const char *fmt);   
 \end{verbatim}
-
-The function \code{psSetLogLevel} may be used to set the current
-level of logging; the previous value is returned.
-The default value is \code{PS_LOG_INFO}. Valid values are 0---9
-inclusive (note that only the first four are required to have
-symbolic names).
-
-A call to \code{psLogMsg(name, level, msg, ...)} shall generate
-a log message if \code{level} is less than or equal to the
-value most recently set using \code{psSetLogLevel}. The function
-\code{p_psLogMsg} is identical, except that it expects a
-final \code{va_list} argument.
-
-The format of the log message shall be of the form:
-\begin{verbatim}
-YYYY:MM:DD hh:mm:ssZ|hostname|l|name            |msg
-\end{verbatim}
-\code{YYYY}, \code{MM}, \code{DD}, \code{hh}, \code{mm}, and \code{ss}
-are the year, month (Jan == 1), day of the month, hours (0--23),
-minutes, and seconds when the log message was received.  Note that the
-timestamp is in ISO order, and that the timezone is GMT (hence the
-\code{Z}).
-
-The \code{hostname} is returned by \code{gethostname}, \code{l} is a
-letter associated with the level (\code{A}, \code{E}, \code{W}, and
-\code{I} for \code{PS_LOG_ABORT}, \code{PS_LOG_ERROR}, \code{PS_LOG_WARN},
-and \code{PS_LOG_INFO} respectively. Other levels are represented
-numerically (\code{5} etc.). The other two field, \code{name} and
-\code{msg} are arguments to \code{psLogMsg}; note that \code{name} has
-a fixed width of 15 characters. If \code{msg} doesn't end in a newline,
-a single newline is emitted to terminate the message.
-
-An example message is:
-\begin{verbatim}
-2004:02:24 20:14:18Z|alibaba.IfA.Hawaii.Edu|I|utils          |Hello World
-\end{verbatim}
-
-Log messages are sent to the destination most recently set using
-\code{psSetLogDestination}.  The only two values that are initially
-defined are \code{PS_LOG_TO_STDERR} and \code{PS_LOG_TO_STDOUT} to
-write to \code{stderr} and \code{stdout} respectively.
-
-The fields included in the log message may be controlled using \code{psSetLogFormat} which
-expects a string consisting of the letters \code{H} (host), \code{L} (level), \code{M} (message),
-\code{N} (name), and \code{T} (time).  The default is \code{THLNM}, as shown above.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+which expects a string consisting of the letters \code{H} (host),
+\code{L} (level), \code{M} (message), \code{N} (name), and \code{T}
+(time).  The default is \code{THLNM}, which produces log messages in
+the form:
+\begin{verbatim}
+YYYY:MM:DD hh:mm:ssZ | hostname | L | name            | msg
+\end{verbatim}
+where \code{YYYY}, \code{MM}, \code{DD}, \code{hh}, \code{mm}, and
+\code{ss} are the year, month (Jan == 1), day of the month, hours
+(0--23), minutes, and seconds when the log message was received.  Note
+that the timestamp is in ISO order, and that the timezone is GMT
+(hence the \code{Z}).  The \code{hostname} is returned by
+\code{gethostname}, \code{L} is a letter associated with the level
+(\code{A}, \code{E}, \code{W}, and \code{I} for \code{PS_LOG_ABORT},
+\code{PS_LOG_ERROR}, \code{PS_LOG_WARN}, and \code{PS_LOG_INFO}
+respectively. Other levels are represented numerically (\code{5}
+etc.). The other two field, \code{name} and \code{msg} are the
+arguments to \code{psLogMsg}; note that \code{name} has a fixed width
+of 15 characters. If \code{msg} doesn't end in a newline, a single
+newline is emitted to terminate the message.  An example message is:
+%
+\begin{verbatim}
+2004:02:24 20:14:18Z | alibaba.IfA.Hawaii.Edu |I | utils          | Hello World
+\end{verbatim}
+%
+The possible order of the format entries is fixed and not determined
+by the order of the letters.  Selecting an output format with fewer
+than the complete set of 5 entries simply removes those entries from
+the output messages.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Miscellaneous Utilities}
 
-\begin{verbatim}
-#define PS_STRING(S) #S                 // converts argument S to string
-    
-/// Prints an error message and aborts
-void psAbort(const char *name,          ///< Category of code that caused the abort
-             const char *fmt,           ///< Format
-             ...                        ///< Extra arguments to use format
-             );
-
-/// Prints an error message and doesn't abort
-void psError(const char *name,          ///< Category of code that caused the abort
-             const char *fmt,           ///< Format
-             ...                        ///< Extra arguments to use format
-    );
-
-/// Allocates and returns a copy of a string
-char *psStringCopy(const char *str      ///< string to copy
-    );
-
-/// Allocates nChar and returns a copy of the string or segment
-char *psStringNCopy(const char *str,    ///< string to copy
-                    int nChar           //!< Number of characters (including \0 )
-    );
-\end{verbatim}
-
-\code{psAbort} shall call \code{psMsgLog} with a level of \code{PS_LOG_ABORT},
-and then call \code{abort}.
-\code{psError} shall call \code{psMsgLog} with a level of \code{PS_LOG_ERROR},
-and then return.
-In cases of doubt, a good choice for
-\code{name} is \code{__func__}.
-
-\code{psStringCopy} shall allocate and return a copy of the input string.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+We require a several very low-functions.  Two functions provide
+conveniences tied to the logging facilities:
+%
+\begin{verbatim}
+void psAbort(char *name, char *fmt,...);
+void psError(char *name, char *fmt,...);
+\end{verbatim}
+%
+The first function, \code{psAbort}, shall call \code{psMsgLog} with a
+level of \code{PS_LOG_ABORT}, and then call \code{abort}.  The second
+function, \code{psError}, shall call \code{psMsgLog} with a level of
+\code{PS_LOG_ERROR}, and then return.  In cases of doubt, a good
+choice for \code{name} is \code{__func__}.  
+
+A few useful string functions are also necessary:
+%
+\begin{verbatim}
+PS_STRING(S);
+char *psStringCopy(char *string);
+char *psStringNCopy(char *string, int nChar);
+\end{verbatim}
+%
+The first function simply converts the argument to a string
+\tbd{explanation of usage and rational?}.  The second function,
+\code{psStringCopy}, shall allocate a sufficient memory block and
+return a copy of the input string.  Similarly, \code{psStringNCopy}
+shall allocate exactly \code{nChar} bytes and copy as much of the
+input string as possible into the resulting block without overrunning
+the available space.  The last byte of the output string is required
+to be \code{NULL}, as well as any additional bytes if \code{string} is
+not long enough to fill the output string.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{Basic Data Collections}
@@ -933 +962 @@
 /** Doubly-linked list element */
 typedef struct psDlistElem {
-   struct psDlistElem *prev;            //!< previous link in list
-   struct psDlistElem *next;            //!< next link in list
-   void *data;                          //!< real data item
+   struct psDlistElem *prev;            ///< previous link in list
+   struct psDlistElem *next;            ///< next link in list
+   void *data;                          ///< real data item
 } psDlistElem;
 
 /** Doubly-linked list */
 typedef struct {
-   int n;                               //!< number of elements on list
-   psDlistElem *head;                   //!< first element on list (may be NULL)
-   psDlistElem *tail;                   //!< last element on list (may be NULL)
-   psDlistElem *iter;                   //!< iteration cursor
+   int n;                               ///< number of elements on list
+   psDlistElem *head;                   ///< first element on list (may be NULL)
+   psDlistElem *tail;                   ///< last element on list (may be NULL)
+   psDlistElem *iter;                   ///< iteration cursor
 } psDlist;
 
 /** Special values of index into list */
 enum {
-   PS_DLIST_HEAD = 0,                   //!< at head
-   PS_DLIST_TAIL = -1,                  //!< at tail
-   PS_DLIST_UNKNOWN = -2,               //!< unknown position
-   PS_DLIST_PREV = -3,                  //!< previous element
-   PS_DLIST_NEXT = -4                   //!< next element
+   PS_DLIST_HEAD = 0,                   ///< at head
+   PS_DLIST_TAIL = -1,                  ///< at tail
+   PS_DLIST_UNKNOWN = -2,               ///< unknown position
+   PS_DLIST_PREV = -3,                  ///< previous element
+   PS_DLIST_NEXT = -4                   ///< next element
 };
 \end{verbatim}
@@ -960 +989 @@
 \begin{verbatim}
 /** Constructor */
-psDlist *psDlistAlloc(void *data        //!< initial data item; may be NULL
+psDlist *psDlistAlloc(void *data        ///< initial data item; may be NULL
                       );
 
 /** Destructor */
-void psDlistFree(psDlist *list,         //!< list to destroy
-                void (*elemFree)(void *) //!< destructor for data on list
+void psDlistFree(psDlist *list,         ///< list to destroy
+                void (*elemFree)(void *) ///< destructor for data on list
                  );
 
@@ -971 +1000 @@
 
 /** Add to list */
-psDlist *psDlistAdd(psDlist *list,      //!< list to add to (may be NULL)
-                    void *data,         //!< data item to add
-                    int where           //!< index, PS_DLIST_HEAD, or PS_DLIST_TAIL
+psDlist *psDlistAdd(psDlist *list,      ///< list to add to (may be NULL)
+                    void *data,         ///< data item to add
+                    int where           ///< index, PS_DLIST_HEAD, or PS_DLIST_TAIL
                     );
 
 /** Append to a list */
-psDlist *psDlistAppend(psDlist *list,   //!< list to append to (may be NULL)
-                       void *data       //!< data item to add
+psDlist *psDlistAppend(psDlist *list,   ///< list to append to (may be NULL)
+                       void *data       ///< data item to add
                        );
 
 /** Remove from a list */
-void *psDlistRemove(psDlist *list,      //!< list to remove element from
-                    void *data,         //!< data item to remove
-                    int which           //!< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
-                                        //!< PS_DLIST_PREV
+void *psDlistRemove(psDlist *list,      ///< list to remove element from
+                    void *data,         ///< data item to remove
+                    int which           ///< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
+                                        ///< PS_DLIST_PREV
                     );
 /** Retrieve from a list */
-void *psDlistGet(const psDlist *list,   //!< list to retrieve element from
-                 int which              //!< index of item, or PS_DLIST_NEXT, or PS_DLIST_PREV
+void *psDlistGet(const psDlist *list,   ///< list to retrieve element from
+                 int which              ///< index of item, or PS_DLIST_NEXT, or PS_DLIST_PREV
                  );
 
 /** Convert doubly-linked list to an array */
-psVoidPtrArray *psDlistToArray(psDlist *dlist //!< List to convert
+psVoidPtrArray *psDlistToArray(psDlist *dlist ///< List to convert
                                );
 
 /** Convert array to a doubly-linked list */
-psDlist *psArrayToDlist(psVoidPtrArray *arr //!< Array to convert
+psDlist *psArrayToDlist(psVoidPtrArray *arr ///< Array to convert
                         );
 \end{verbatim}
@@ -1015 +1044 @@
 \begin{verbatim}
 /** Set the iterator */
-void psDlistSetIterator(psDlist *list,  //!< list to retrieve element from
-                        int where,      //!< index, PS_DLIST_HEAD, or PS_DLIST_TAIL
-                        int which       //!< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
-                                        //!< PS_DLIST_PREV
+void psDlistSetIterator(psDlist *list,  ///< list to retrieve element from
+                        int where,      ///< index, PS_DLIST_HEAD, or PS_DLIST_TAIL
+                        int which       ///< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
+                                        ///< PS_DLIST_PREV
                         );
 
 /** Get next element */
-void *psDlistGetNext(psDlist *list,     //!< list to retrieve element from
-                     int which          //!< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
-                                        //!< PS_DLIST_PREV
+void *psDlistGetNext(psDlist *list,     ///< list to retrieve element from
+                     int which          ///< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
+                                        ///< PS_DLIST_PREV
                      );
 
 /** Get previous element */
-void *psDlistGetPrev(psDlist *list,     //!< list to retrieve element from
-                     int which          //!< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
-                                        //!< PS_DLIST_PREV
+void *psDlistGetPrev(psDlist *list,     ///< list to retrieve element from
+                     int which          ///< index of item, or PS_DLIST_UNKNOWN, or PS_DLIST_NEXT, or
+                                        ///< PS_DLIST_PREV
                      );
 \end{verbatim}
@@ -1048 +1077 @@
 \code{psDlist}).
 
+\subsubsubsection{Rationale}
+
+\tbd{defer this example to the psDlist section?}
+
+The \code{psMemBlock.refcounter} is clearly useful for detecting
+attempts to free memory that is already free.  A more complex
+application is for allowing pointers to complex data-objects (e.g.\
+images) to appear in more than one data structure simultaneously:
+
+\begin{verbatim}
+typedef struct {
+    char *name;
+    int value;
+} psSimple;
+
+psSimple *psSimpleAlloc(const char *name, int val)
+{
+    psSimple *simp = psAlloc(sizeof(psSimple));
+    simp->name = strcpy(psAlloc(strlen(name) + 1), name);
+    simp->value = val;
+
+    return simp;
+}
+
+void psSimpleFree(psSimple *simp)
+{
+    if (simp == NULL) { return; }
+
+    if (psMemGetRefCounter(simp) > 1) {
+        (void)psMemDecrRefCounter(simp);
+        return;
+    }
+}
+\end{verbatim}
+
+Because of the use of the \code{refcounter} field, we can safely put items of
+this type onto many lists:
+\goodbreak
+\begin{verbatim}
+simp = psSimpleAlloc("RHL", 0);
+psDlistAppend(list1, psMemIncrRefCounter(simp));
+psDlistAppend(list2, psMemIncrRefCounter(simp));
+psSimpleFree(simp);
+\end{verbatim}
+
+(Note: in fact there is no need to explicitly increment the counter
+in this case, as the \code{psDlistAppend} (section \ref{psDlist})
+API specifies that it
+does it for you.)
+
 \subsection{Hash Tables}
 \hlabel{psHash}
@@ -1087 +1166 @@
 \end{verbatim}
 
-
 A hash table is an abstract type \code{psHash}.  The argument
 \code{nbucket} to \code{psHashAlloc} is a non-binding suggestion
@@ -1108 +1186 @@
 \code{psHashRemove} removes the entry associated with the
 key from the table, and returns the \code{data}; if the key's invalid it returns NULL.
+
+\code{psHashKeylist} returns the complete list of defined keys
+associated with the psHash table. \tbd{API is not yet specified.}
 
 \section{Data manipulation}
@@ -1123 +1204 @@
 \item (Fast) Fourier Transforms;
 \item General functions; and
-\item Minimisation and fitting routines.
+\item Minimization and fitting routines.
 \end{itemize}
 
@@ -1137 +1218 @@
 /** A bitmask of arbitrary length. */
 typedef struct {
-    int n;                              //!< Number of chars that form the mask
-    char *bits;                         //!< The bits
+    int n;                              ///< Number of chars that form the mask
+    char *bits;                         ///< The bits
 } psBitMask;
 \end{verbatim}
@@ -1146 +1227 @@
 \begin{verbatim}
 /** Constructor */
-psBitMask *psBitMaskAlloc(int n         //!< Number of bits required
+psBitMask *psBitMaskAlloc(int n         ///< Number of bits required
     );
  
 /** Destructor */
-void psBitMaskFree(psBitMask *restrict myMask //!< Bit mask to destroy
+void psBitMaskFree(psBitMask *restrict myMask ///< Bit mask to destroy
     );
 \end{verbatim}
@@ -1165 +1246 @@
 /** Set a bit mask */
 psBitMask *
-psBitMaskSet(psBitMask *outMask,        //!< Output bit mask or NULL
-             const psBitMask *myMask,   //!< Input bit mask
-             int bit                    //!< Bit to set
+psBitMaskSet(psBitMask *outMask,        ///< Output bit mask or NULL
+             const psBitMask *myMask,   ///< Input bit mask
+             int bit                    ///< Bit to set
     );
 \end{verbatim}
@@ -1174 +1255 @@
 /** Check a bit mask.  Returns true or false */
 int
-psBitMaskTest(const psBitMask *checkMask, //!< Bit mask to check
-              int bit                   //!< Bit to check
+psBitMaskTest(const psBitMask *checkMask, ///< Bit mask to check
+              int bit                   ///< Bit to check
     );
 \end{verbatim}
@@ -1182 +1263 @@
 /** apply the given operator to two bit masks */
 psBitMask *
-psBitMaskOp(psBitMask *outMask,         //!< Output bit mask or NULL
-            const psBitMask *restrict inMask1, //!< Input bit mask 1
-            char *operator,             //!< bit mask operator (AND, OR, XOR)
-            const psBitMask *restrict inMask2 //!< Input bit mask 2
+psBitMaskOp(psBitMask *outMask,         ///< Output bit mask or NULL
+            const psBitMask *restrict inMask1, ///< Input bit mask 1
+            char *operator,             ///< bit mask operator (AND, OR, XOR)
+            const psBitMask *restrict inMask2 ///< Input bit mask 2
     );
 \end{verbatim}
@@ -1198 +1279 @@
 /** Sort an array. Inputs not restrict-ed to allow sort in place */
 psFloatArray *
-psSort(psFloatArray *out,               //!< Sorted array to return. May be NULL
-       const psFloatArray *myArray      //!< Array to sort
+psSort(psFloatArray *out,               ///< Sorted array to return. May be NULL
+       const psFloatArray *myArray      ///< Array to sort
     );
 \end{verbatim}
@@ -1205 +1286 @@
 We also require the ability to sort one array based on another.  In
 order to facilitate this, we will have a sort function return an array
-containing the indices for the unsorted list in the order appropriate
+containing the induces for the unsorted list in the order appropriate
 for the sorted array:
 
@@ -1211 +1292 @@
 /** Sort an array, along with some other stuff.  Returns an index array */
 psIntArray *
-psSortIndex(psIntArray *restrict out;   //!< Output index array (may be NULL)
-            const psFloatArray *restrict myArray //!< Array to sort
+psSortIndex(psIntArray *restrict out;   ///< Output index array (may be NULL)
+            const psFloatArray *restrict myArray ///< Array to sort
     );
 \end{verbatim}
@@ -1251 +1332 @@
 /** Do Statistics on an array.  Returns a status value. */
 psStats *
-psArrayStats(const psFloatArray *restrict myArray, //!< Array to be analysed
-             const psIntArray *restrict maskArray, //!< Ignore elements where (maskArray & maskVal) != 0
-                                                   //!< May be NULL
-             unsigned int maskVal,      //!< Only mask elements with one of these bits set in maskArray
-             psStats *stats             //!< stats structure defines stats to be calculated and how
+psArrayStats(const psFloatArray *restrict myArray, ///< Array to be analysed
+             const psIntArray *restrict maskArray, ///< Ignore elements where (maskArray & maskVal) != 0
+                                                   ///< May be NULL
+             unsigned int maskVal,      ///< Only mask elements with one of these bits set in maskArray
+             psStats *stats             ///< stats structure defines stats to be calculated and how
              );
 \end{verbatim}
@@ -1265 +1346 @@
 /** generic statistics structure */
 typedef struct {
-    double sampleMean;                  //<! formal mean of sample
-    double sampleMedian;                //<! formal median of sample
-    double sampleMode;                  //!< Formal mode of sample
-    double sampleStdev;                 //<! standard deviation of sample
-    double sampleUQ;                    //<! upper quartile of sample
-    double sampleLQ;                    //<! lower quartile of sample
-    double robustMean;                  //<! robust mean of array
-    double robustMeanError;             //<! error on robust mean
-    int    robustMeanNvalues;           //<! number of measurements used for robust mean
-    double robustMedian;                //<! robust median of array
-    double robustMedianError;           //<! error on robust median
-    int    robustMedianNvalues;         //<! number of measurements used for robust median
-    double robustMode;                  //!< Robust mode of array
-    double robustModeErr;               //!< Error in robust mode
-    int    robustModeNvalues;           //!< Number of measurements used for robust mode
-    double robustStdev;                 //<! robust standard deviation of array
-    double robustUQ;                    //<! robust upper quartile
-    double robustLQ;                    //<! robust lower quartile
-    double clippedMean;                 //<! Nsigma clipped mean
-    double clippedMeanError;            //<! error on clipped mean
-    int    clippedMeanNvalues;          //<! number of data points used for clipped mean
-    double clippedStdev;                //!< standard deviation after clipping
-    double clipSigma;                   //<! Nsigma used for clipping; user input
-    int    clipIter;                    //!< Number of clipping iterations; user input
-    double min;                         //<! minimum data value in array
-    double max;                         //<! maximum data value in array
-    int    nValues;                     //<! number of data values in array
-    psStatsOptions options;             //<! bitmask of calculated values
+    double sampleMean;                  ///< formal mean of sample
+    double sampleMedian;                ///< formal median of sample
+    double sampleMode;                  ///< Formal mode of sample
+    double sampleStdev;                 ///< standard deviation of sample
+    double sampleUQ;                    ///< upper quartile of sample
+    double sampleLQ;                    ///< lower quartile of sample
+    double robustMean;                  ///< robust mean of array
+    double robustMeanError;             ///< error on robust mean
+    int    robustMeanNvalues;           ///< number of measurements used for robust mean
+    double robustMedian;                ///< robust median of array
+    double robustMedianError;           ///< error on robust median
+    int    robustMedianNvalues;         ///< number of measurements used for robust median
+    double robustMode;                  ///< Robust mode of array
+    double robustModeErr;               ///< Error in robust mode
+    int    robustModeNvalues;           ///< Number of measurements used for robust mode
+    double robustStdev;                 ///< robust standard deviation of array
+    double robustUQ;                    ///< robust upper quartile
+    double robustLQ;                    ///< robust lower quartile
+    double clippedMean;                 ///< Nsigma clipped mean
+    double clippedMeanError;            ///< error on clipped mean
+    int    clippedMeanNvalues;          ///< number of data points used for clipped mean
+    double clippedStdev;                ///< standard deviation after clipping
+    double clipSigma;                   ///< Nsigma used for clipping; user input
+    int    clipIter;                    ///< Number of clipping iterations; user input
+    double min;                         ///< minimum data value in array
+    double max;                         ///< maximum data value in array
+    int    nValues;                     ///< number of data values in array
+    psStatsOptions options;             ///< bitmask of calculated values
 } psStats;
 \end{verbatim}
@@ -1305 +1386 @@
 /** Histograms */
 typedef struct {
-    const psFloatArray *restrict lower; //!< Lower bounds for the bins
-    const psFloatArray *restrict upper; //!< Upper bounds for the bins
-    psIntArray *nums;                   //!< Number in each of the bins
-    const float minVal, maxVal;         //!< Minimum and maximum values
-    int minNum, maxNum;                 //!< Number below the minimum and above the maximum
+    const psFloatArray *restrict lower; ///< Lower bounds for the bins
+    const psFloatArray *restrict upper; ///< Upper bounds for the bins
+    psIntArray *nums;                   ///< Number in each of the bins
+    const float minVal, maxVal;         ///< Minimum and maximum values
+    int minNum, maxNum;                 ///< Number below the minimum and above the maximum
 } psHistogram;
 \end{verbatim}
@@ -1321 +1402 @@
 /** Constructor */
 psHistogram *
-psHistogramAlloc(float lower,           //!< Lower limit for the bins
-                 float upper,           //!< Upper limit for the bins
-                 float size             //!< Size of the bins
+psHistogramAlloc(float lower,           ///< Lower limit for the bins
+                 float upper,           ///< Upper limit for the bins
+                 float size             ///< Size of the bins
     );
 \end{verbatim}
@@ -1330 +1411 @@
 /** Generic constructor */
 psHistogram *
-psHistogramAllocGeneric(const psFloatArray *restrict lower, //!< Lower bounds for the bins
-                        const psFloatArray *restrict upper, //!< Upper bounds for the bins
-                        float minVal,   //!< Minimum value
-                        float maxVal    //!< Maximum value
+psHistogramAllocGeneric(const psFloatArray *restrict lower, ///< Lower bounds for the bins
+                        const psFloatArray *restrict upper, ///< Upper bounds for the bins
+                        float minVal,   ///< Minimum value
+                        float maxVal    ///< Maximum value
     );
 \end{verbatim}
@@ -1340 +1421 @@
 /** Destructor */
 void
-psHistogramFree(psHistogram *restrict myHist //!< Histogram to destroy
+psHistogramFree(psHistogram *restrict myHist ///< Histogram to destroy
     );
 \end{verbatim}
@@ -1354 +1435 @@
 /** A matrix */
 typedef struct {
-    int xSize, ySize;                   //!< Dimensions in x and y
-    float *restrict *restrict value;    //!< Values in matrix
+    int xSize, ySize;                   ///< Dimensions in x and y
+    float *restrict *restrict value;    ///< Values in matrix
 } psMatrix;
 \end{verbatim}
@@ -1364 +1445 @@
 /** Constructor */
 psMatrix *
-psMatrixAlloc(int Xdimen,                       //!< x dimension of new matrix
-              int Ydimen                        //!< y dimension of new matrix
+psMatrixAlloc(int Xdimen,                       ///< x dimension of new matrix
+              int Ydimen                        ///< y dimension of new matrix
               );
 \end{verbatim}
@@ -1372 +1453 @@
 /** Destructor */
 void
-psMatrixFree(psMatrix *restrict myMatrix        //!< Matrix to destroy
+psMatrixFree(psMatrix *restrict myMatrix        ///< Matrix to destroy
              );
 \end{verbatim}
@@ -1389 +1470 @@
 /** Invert matrix.  Not using restrict, to allow inversion to be done in-place */
 psMatrix *
-psMatrixInvert(psMatrix *out,           //!< Matrix to return, or NULL
-               const psMatrix *myMatrix, //!< Matrix to be inverted
-               float *restrict determinant //!< Determinant to return, or NULL
+psMatrixInvert(psMatrix *out,           ///< Matrix to return, or NULL
+               const psMatrix *myMatrix, ///< Matrix to be inverted
+               float *restrict determinant ///< Determinant to return, or NULL
     );
 \end{verbatim}
@@ -1405 +1486 @@
 /** Matrix operations */
 psMatrix *
-psMatrixOp(psMatrix *out,               //!< Matrix to return, or NULL
-           const psMatrix *matrix1,     //!< Matrix 1
-           const char *op,              //!< Operation to perform
-           const psMatrix *matrix2      //!< Matrix 2
+psMatrixOp(psMatrix *out,               ///< Matrix to return, or NULL
+           const psMatrix *matrix1,     ///< Matrix 1
+           const char *op,              ///< Operation to perform
+           const psMatrix *matrix2      ///< Matrix 2
            );
 \end{verbatim}
@@ -1415 +1496 @@
 /** Transpose Matrix */
 psMatrix *
-psMatrixTranspose(psMatrix *out,        //!< Matrix to return, or NULL
-                  const psMatrix *myMatrix //!< Matrix to transpose
+psMatrixTranspose(psMatrix *out,        ///< Matrix to return, or NULL
+                  const psMatrix *myMatrix ///< Matrix to transpose
                   );
 \end{verbatim}
@@ -1423 +1504 @@
 /** Convert matrix to vector.  Intended for a 1-d matrix. */
 psVector *
-psMatrixToVector(psMatrix *myMatrix     //!< Matrix to convert
+psMatrixToVector(psMatrix *myMatrix     ///< Matrix to convert
     );
 \end{verbatim}
@@ -1573 +1654 @@
     is not a Gaussian deviate.  The evaluated Gaussian is: \f[ exp(-\frac{(x-mean)^2}{2\sigma^2}) \f] */
 float
-psGaussian(float x,                     //!< Value at which to evaluate
-           float mean,                  //!< Mean for the Gaussian
-           float stddev                 //!< Standard deviation for the Gaussian
+psGaussian(float x,                     ///< Value at which to evaluate
+           float mean,                  ///< Mean for the Gaussian
+           float stddev                 ///< Standard deviation for the Gaussian
            );
 \end{verbatim}
@@ -1583 +1664 @@
 floating-point (for speed) and double-precision (for milli-arcsec
 precision) versions.  This comes from the need to fit over $x$,$y$,
-colour and magnitude simultaneously.  We must also be able to
+color and magnitude simultaneously.  We must also be able to
 calculate the errors in the fit coefficients, as well as be able to
 turn on and off each coefficient.  This leads us to define polynomial
@@ -1591 +1672 @@
 /** One-dimensional polynomial */
 typedef struct {
-    int n;                              //!< Number of terms
-    float *restrict coeff;              //!< Coefficients
-    float *restrict coeffErr;           //!< Error in coefficients
-    char *restrict mask;                //!< Coefficient mask
+    int n;                              ///< Number of terms
+    float *restrict coeff;              ///< Coefficients
+    float *restrict coeffErr;           ///< Error in coefficients
+    char *restrict mask;                ///< Coefficient mask
 } psPolynomial1D;
 \end{verbatim}
@@ -1601 +1682 @@
 /** Two-dimensional polynomial */
 typedef struct {
-    int nX, nY;                         //!< Number of terms in x and y
-    float *restrict *restrict coeff;    //!< Coefficients
-    float *restrict *restrict coeffErr; //!< Error in coefficients
-    char *restrict *restrict mask;      //!< Coefficients mask
+    int nX, nY;                         ///< Number of terms in x and y
+    float *restrict *restrict coeff;    ///< Coefficients
+    float *restrict *restrict coeffErr; ///< Error in coefficients
+    char *restrict *restrict mask;      ///< Coefficients mask
 } psPolynomial2D;
 \end{verbatim}
@@ -1613 +1694 @@
 /** Double-precision one-dimensional polynomial */
 typedef struct {
-    int n;                              //!< Number of terms
-    double *restrict coeff;             //!< Coefficients
-    double *restrict coeffErr;          //!< Error in coefficients
-    char *restrict mask;                //!< Coefficient mask
+    int n;                              ///< Number of terms
+    double *restrict coeff;             ///< Coefficients
+    double *restrict coeffErr;          ///< Error in coefficients
+    char *restrict mask;                ///< Coefficient mask
 } psDPolynomial1D;
 \end{verbatim}
@@ -1623 +1704 @@
 /** Double-precision two-dimensional polynomial */
 typedef struct {
-    int nX, nY;                         //!< Number of terms in x and y
-    double *restrict *restrict coeff;   //!< Coefficients
-    double *restrict *restrict coeffErr; //!< Error in coefficients
-    char *restrict *restrict mask;      //!< Coefficients mask
+    int nX, nY;                         ///< Number of terms in x and y
+    double *restrict *restrict coeff;   ///< Coefficients
+    double *restrict *restrict coeffErr; ///< Error in coefficients
+    char *restrict *restrict mask;      ///< Coefficients mask
 } psDPolynomial2D;
 \end{verbatim}
@@ -1636 +1717 @@
 The constructor and destructor are:
 \begin{verbatim}
-psDPolynomial2D *psDPolynomial2DAlloc(int nX, int nY //!< Number of terms in x and y
-    );
-void psDPolynomial2DFree(psDPolynomial2D *restrict myPoly //!< Polynomial to destroy
+psDPolynomial2D *psDPolynomial2DAlloc(int nX, int nY ///< Number of terms in x and y
+    );
+void psDPolynomial2DFree(psDPolynomial2D *restrict myPoly ///< Polynomial to destroy
     );
 \end{verbatim}
@@ -1647 +1728 @@
 /** Evaluate 2D polynomial (double precision) */
 double
-psEvalDPolynomial2D(double x,           //!< Value x at which to evaluate
-                    double y,           //!< Value y at which to evaluate
-                    const psDPolynomial2D *restrict myPoly //!< Coefficients for the polynomial
+psEvalDPolynomial2D(double x,           ///< Value x at which to evaluate
+                    double y,           ///< Value y at which to evaluate
+                    const psDPolynomial2D *restrict myPoly ///< Coefficients for the polynomial
                     );
 \end{verbatim}
 
+\tbd{generate a Gaussian deviate vector}
+
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Minimization and fitting routines}
 
-We require a general minimisation routine, a routine that will
-specifically minimise $\chi^2$ given a list of data with associated
+We require a general minimization routine, a routine that will
+specifically minimize $\chi^2$ given a list of data with associated
 errors, and a function that will analytically determine the polynomial
 fit by least-squares.  The APIs are:
@@ -1665 +1748 @@
 /** Minimize a particular function */
 psFloatArray *
-psMinimize(float (*myFunction)(const psFloatArray *restrict),   //!< Function to minimize
-           psFloatArray *restrict initialGuess //!< Initial guess
+psMinimize(float (*myFunction)(const psFloatArray *restrict),   ///< Function to minimize
+           psFloatArray *restrict initialGuess ///< Initial guess
            );
 \end{verbatim}
@@ -1674 +1757 @@
 psFloatArray *
 psMinimizeChi2(float (*evalModel)(const psFloatArray *restrict,
-                                  const psFloatArray *restrict), //!< Model to fit; receives domain and
-                                                                 //!< parameters
-               const psFloatArray *restrict domain, //!< The domain values for the corresponding measurements
-               const psFloatArray *restrict data, //!< Data to fit
-               const psFloatArray *restrict errors, //!< Errors in the data
-               psFloatArray *restrict initialGuess, //!< Initial guess
-               const psIntArray *restrict guessMask //!< 1 = fit for parameter, 0 = hold parameter constant
+                                  const psFloatArray *restrict), ///< Model to fit; receives domain and
+                                                                 ///< parameters
+               const psFloatArray *restrict domain, ///< The domain values for the corresponding measurements
+               const psFloatArray *restrict data, ///< Data to fit
+               const psFloatArray *restrict errors, ///< Errors in the data
+               psFloatArray *restrict initialGuess, ///< Initial guess
+               const psIntArray *restrict guessMask ///< 1 = fit for parameter, 0 = hold parameter constant
                );
 \end{verbatim}
@@ -1687 +1770 @@
 /** Derive a polynomial that goes through the points --- can be done analytically */
 psFloatArray *
-psGetArrayPolynomial(const psFloatArray *restrict ord, //!< Ordinates (or NULL to just use the indices)
-                     const psFloatArray *restrict coord //!< Coordinates
+psGetArrayPolynomial(const psFloatArray *restrict ord, ///< Ordinates (or NULL to just use the indices)
+                     const psFloatArray *restrict coord ///< Coordinates
     );
 \end{verbatim}
@@ -1718 +1801 @@
         psF32 **rows_f32;               ///< pointers to psF32 data 
         psF64 **rows_f64;               ///< pointers to psF64 data 
-        psComplex **rows_complex;       //!< pointers to psComplex data
+        psComplex **rows_complex;       ///< pointers to psComplex data
     } rows;
     psImage *parent;                    ///< parent, if a subimage 
@@ -1732 +1815 @@
 restricted to the datatypes equivalent to the valid BITPIX values 8,
 16, 32, -32, -64).  The image represented in the data structure may
-represent a subset of the pixels in a complete array.  The offset of
-the \code{(0,0)} pixel in this array relative to the parent array is
-given by the elements \code{(x0,y0)}.  The structure may include
-references to subrasters (\code{children, Nchildren}) and/or to a
-containing array (\code{parent}).
+represent a subset of the pixels in a complete array, in which case
+the image is considered to be the child of that parent array.  The
+offset of the \code{(0,0)} pixel in this array relative to the parent
+array is given by the elements \code{(x0,y0)}.  The structure may
+include references to subrasters (\code{children, Nchildren}) and/or
+to a containing array (\code{parent}).
 
 We require a variety of functions to manipulate these image
@@ -1759 +1843 @@
 /// Create a subimage of the specified area.
 psImage *
-psImageSubset(psImage *out,             //!< Subimage to return, or NULL
+psImageSubset(psImage *out,             ///< Subimage to return, or NULL
               const psImage *image,     ///< parent image 
               int nx,                   ///< subimage width (<= image.nx - x0)  
@@ -1779 +1863 @@
 \begin{verbatim}
 psImage *
-psImageFreePixels(psImage *restrict image //!< Image whose pixels are to be freed
+psImageFreePixels(psImage *restrict image ///< Image whose pixels are to be freed
     );
 \end{verbatim}
@@ -1810 +1894 @@
 \begin{verbatim}
 psFloatArray *
-psImageSlice(psFloatArray *out,         //!< Vector to output, or NULL
+psImageSlice(psFloatArray *out,         ///< Vector to output, or NULL
              const psImage *input,      ///< extract slice from this image
              int x,                     ///< starting x coord of region to slice
@@ -1832 +1916 @@
 \begin{verbatim}
 psFloatArray *
-psImageCut(psFloatArray *out,           //!< Vector to output, or NULL
+psImageCut(psFloatArray *out,           ///< Vector to output, or NULL
            const psImage *input,        ///< extract cut from this image
            float xs,                    ///< starting x coord of cut
@@ -1844 +1928 @@
 
 
-Extract radial annulii data to a vector.  A vector is constructed
+Extract radial annuli data to a vector.  A vector is constructed
 where each vector elements is derived from the statistics of the
-pixels which land in one of a sequence of annulii.  The annulii are
+pixels which land in one of a sequence of annuli.  The annuli are
 centered on the image pixel coordinate \code{x,y}, and have width
-\code{dr}.  The number of annulii is $radius / dr$.  The statistic
+\code{dr}.  The number of annuli is $radius / dr$.  The statistic
 used to derive the output vector value is specified by \code{psStats
 stats}
 \begin{verbatim}
 psFloatArray *
-psImageRadialCut(psFloatArray *out,     //!< Vector to output, or NULL
+psImageRadialCut(psFloatArray *out,     ///< Vector to output, or NULL
                  const psImage *input,  ///< extract profile from this image
                  float x,               ///< center x coord of annulii
@@ -1872 +1956 @@
 \begin{verbatim}
 psImage *
-psImageRebin(psImage *out,              //!< Image to output, or NULL
+psImageRebin(psImage *out,              ///< Image to output, or NULL
              const psImage *input,      ///< rebin this image
              float scale,               ///< rebinning scale: doutput = scale*dinput
@@ -1887 +1971 @@
 \begin{verbatim}
 psImage *
-psImageRotate(psImage *out,             //!< Image to output, or NULL
+psImageRotate(psImage *out,             ///< Image to output, or NULL
               const psImage *input,     ///< rotate this image
               float angle               ///< rotate by this amount anti-clockwise (degrees)
@@ -1897 +1981 @@
 values should interpolate between the input pixel values.  The output
 image has the same dimensions as the input image.  Pixels which fall
-off the edge of the output image are loast.  Newly exposed pixels are
+off the edge of the output image are lost.  Newly exposed pixels are
 set to the value given by \code{exposed}.  
 \begin{verbatim}
 psImage *
-psImageShift(psImage *out,              //!< Image to output, or NULL
+psImageShift(psImage *out,              ///< Image to output, or NULL
              const psImage *input,      ///< shift this image
              float dx,                  ///< shift by this amount in x
@@ -1914 +1998 @@
 \begin{verbatim}
 psImage *
-psImageRoll(psImage *out,               //!< Image to output, or NULL
+psImageRoll(psImage *out,               ///< Image to output, or NULL
             const psImage *input,       ///< roll this image
             int dx,                     ///< roll this amount in x
@@ -1941 +2025 @@
 Fit a 2-D polynomial surface to an image.  The input structure
 \code{coeffs} contains the desired order and terms of interest.
-\tbd{how do we specify the renomalization?}
+\tbd{how do we specify the renormalization?}
 \begin{verbatim}
 psPolynomial2D *
@@ -2008 +2092 @@
 
 Write an image section to named file, which may exist.  This
-operatation may write a portion of an image over the existing bytes of
+operation may write a portion of an image over the existing bytes of
 an existing image.  If the file does not exist, it should be created.
 If the specified extension does not exist, it should be created.  If
@@ -2067 +2151 @@
 \begin{verbatim}
 psImage *
-psImageFFT(psImage *output,             //!< Output image
+psImageFFT(psImage *output,             ///< Output image
            const psImage *input,        ///< image to FFT
            int direction                ///< FFT direction 
@@ -2198 +2282 @@
 \section{Astronomy-Specific Functions}
 
-Some basic, relatively simple astronomy-specific functions are
-required which will serve as the foundation for building the Phase $N$
-modules.  These functions are not expected to cover every forseeable
-function, but will serve as the building blocks of more complicated
-processing functions.
+The previous sections of this document defined basic functionality
+needed by a wide range of scientific programming.  The rest of this
+document concentrates on aspects of the library which are specific to
+astronomy.  Some basic, relatively simple astronomy-specific functions
+are required which will serve as the foundation for building the
+higher level modules.  These functions are not expected to cover every
+foreseeable function, but will serve as the building blocks of more
+complicated processing functions.  
 
 We require functions covering each of the following areas:
 \begin{itemize}
-\item Astrometry;
-\item Dates and times;
-\item Image handling;
-\item Metadata;
-\item Detector and sky positions;
-\item Astronomical objects; and
-\item Photometry.
+\item Dates and times
+\item Metadata
+\item Detector and sky positions
+\item Astronomy Image
+\item Astrometry
+\item Photometry
+\item Astronomical objects
 \end{itemize}
-These are each dealt with below.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
@@ -2224 +2310 @@
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
-\subsection{Detector and sky positions}
-
-Both detector and sky positions will be used extensively in the IPP.
-Since these both contain two coordinates with their associated errors,
-we bundle these into a single generic structure, \code{psCoord},
-containing \code{union}s to handle the semantic differences.
-
-\begin{verbatim}
-/** A point in 2-D space, with errors.
- */
-typedef union {
-    struct {
-        double x;                       //!< x position
-        double y;                       //!< y position
-        double xErr;                    //!< Error in x position
-        double yErr;                    //!< Error in y position
-    } xy;
-    struct {
-        double r;                       //!< RA
-        double d;                       //!< Dec
-        double rErr;                    //!< Error in RA
-        double dErr;                    //!< Error in Dec
-    } rd;
-} psCoord;
-\end{verbatim}
-
-\subsubsection{Transformations}
-
-We specify two types of transforms between coordinate systems.  The
-first consists simply of two 2D polynomials to transform both
-components; this will be used to apply the coordinate transformations
-between Cells, Chips and the Focal Plane.  The second consists of two
-4D polynomials; this will be used to apply position-, colour- and
-magnitude-dependent distortions between the Focal Plane and the
-Tangent Plane.
-
-\begin{verbatim}
-/** A polynomial transformation between coordinate frames.  This may be a linear relationship, or may
- *  represent a higher-order transformation.
- */
-typedef struct {
-    psDPolynomial2D *x;
-    psDPolynomial2D *y;
-} psCoordXform;
-\end{verbatim}
-
-\begin{verbatim}
-/** The optical distortion terms.  The lowest two terms are the x and y axis of the target system.  The higher
- *  two terms represent magnitude and color terms.
- */
-typedef struct {
-    psDPolynomial4D *x;
-    psDPolynomial4D *y;
-} psDistortion;
-\end{verbatim}
-
-We require corresponding functions to apply the transformations:
-
-\begin{verbatim}
-/** apply the coordinate transformation to the given coordinate */
-psCoord *psCoordXformApply (psCoord *out, //!< Output coordinates, or NULL
-                            const psCoordXform *frame, ///< coordinate transformation
-                            const psCoord *coords ///< input coordiate
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** apply the optical distortion to the given coordinate, magnitude, color */
-psCoord *psDistortionApply (psCoord *out, //!< Output coordinates, or NULL
-                            const psdistortion *pattern, ///< optical distortion pattern
-                            const psCoord *coords, ///< input coordinate
-                            float mag,  ///< magnitude of object
-                            float color ///< color of object
-    );
-\end{verbatim}
-
-
-\subsubsection{Offsets}
-
-We require a function to calculate the offset between two positions on
-the sky, as well as a function to apply an offset to a position.
-
-\begin{verbatim}
-/** Get offset (RA,Dec) on the sky between two positions position1 and position2 may not be identical */
-psCoord *
-psGetOffset(const psCoord *restrict position1, //!< Position 1
-            const psCoord *restrict position2, //!< Position 2
-            const char *type            //!< Type of offset: Linear, Spherical/Arcsec, Spherical/Degreees etc
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Apply an offset to a position */
-psCoord *
-psApplyOffset(const psCoord *restrict position, //!< Position
-              const psCoord *restrict offset, //!< Offset
-              const char *type          //!< Type of offset: Linear, Spherical/Arcsec, Spherical/Degreees etc
-    );
-\end{verbatim}
-
-
-\subsubsection{Positions of Major SS Objects}
-
-We require the ability to calculate the position of major Solar System
-objects, as well as Lunar phase.
-
-\begin{verbatim}
-/** Get Sun Position */
-psCoord *
-psGetSunPos(float mjd                   //!< MJD to get position for
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Get Moon position */
-psCoord *
-psGetMoonPos(float mjd,                 //!< MJD to get position for
-             double latitude,           //!< Latitude for apparent position
-             double longitude           //!< Longitude for apparent position
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Get Moon phase */
-float
-psGetMoonPhase(float mjd                //!< MJD to get phase for
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Get Planet positions */
-psCoord *
-psGetSolarSystemPos(const char *solarSystemObject, //!< Named S.S. object
-                    float mjd           //!< MJD to get position for
-    );
-\end{verbatim}
-
-\subsubsection{Celestial Coordinate Conversions}
-
-We need to be able to convert between ICRS, Galactic and Ecliptic
-coordinates.
-
-\begin{verbatim}
-/** Convert ICRS to Ecliptic */
-psCoord *
-psCoordinatesItoE(const psCoord *restrict coordinates //!< ICRS coordinates to convert
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Convert Ecliptic to ICRS */
-psCoord *
-psCoordinatesEtoI(const psCoord *restrict coordinates //!< Ecliptic coordinates to convert
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Convert ICRS to Galactic */
-psCoord *
-psCoordinatesItoG(const psCoord *restrict coordinates //!< ICRS coordinates to convert
-    );
-\end{verbatim}
-
-\begin{verbatim}
-/** Convert Galactic to ICRS */
-psCoord *
-psCoordinatesGtoI(const psCoord *restrict coordinates //!< Galactic coordinates to convert
-    );
-\end{verbatim}
-
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
 \subsection{Metadata}
+
+\subsubsection{Conceptual Overview}
+
+Within PSLib, we provide a data structure to carry metadata and
+mechanisms to manipulate the metadata.  Metadata is a general concept
+that requires some discussion.  In any data analysis task, the
+ensemble of all possible data may be divided into two or three
+classes: there is the specific data of interest, there is data which
+is related or critical but not the primary data of interest, and there
+is all of the other data which may or may not be interesting.  For
+example, consider a simple 2D image obtained of a galaxy from a CCD
+camera on a telescope.  If you want to study the galaxy, the specific
+data of interest is the collection of pixels.  There are a variety of
+other pieces of data which are closely related and crucial to
+understanding the data in those pixels, such as the dimensions of the
+image, the coordinate system, the time of the image, the exposure
+time, and so forth.  Other data may be known which may be less
+critical to understanding the image, but which may be interesting or
+desired at a later date.  For example, the observer who took the
+image, the filter manufacturer, the humidity at the telescope, etc.
+
+Formally, all of the related data which describe the principal data of
+interest are metadata.  Note that which piece is the metadata and
+which is the data may depend on the context.  If you are examining the
+pixels in an image, the coordinate and flux of an object may be part
+of the metadata.  However, if you are analyzing a collection of
+objects extracted from an image, you may consider then pixel data
+simply part of the metadata associated with the list of objects.  
+
+There are various ways to handle metadata vs data within a programming
+environment.  In C, it is convenient to use structures to group
+associated data together.  One possibility is to define the metadata
+as part of the associated data structure.  For example, the image data
+structure would have elements for all possible associated measurement.
+This approach is both cumbersome (because of the large number metadata
+types), impractical (because the full range of necessary metadata is
+difficult to know in advance) and inflexible (because any change in
+the collection of metadata requires addition of new structure elements
+and recompilation).  
+
+An alternative is to place the metadata in a generic container and use
+lookup mechanisms to extract the appropriate metadata when needed.  An
+example of this is would be a text-based FITS header for an image read
+into a flat text buffer.  In this implementation, metadata lookup
+functions could return the current value of, for example, NAXIS1 (the
+number of columns of the image) by scanning through the header buffer.
+This method has the benefits of flexibility and simplicity of
+programming interface, but it has the disadvantage that all metadata
+is accessed though this lookup mechanism.  This may make the code less
+readable and it may slow down the access.  
+
+We propose an intermediate solution to this problem.  We specify a
+flexible, generic metadata container and access methods.  Data types
+which require access to a general collection of metadata should
+include an entry of this metadata type.  However, a subset of metadata
+concepts which are basic and frequently required may be placed in the
+coded structure elements.  This approach allows the code to refer to
+the basic metadata concepts as part of the data structure (ie,
+image.nx), but also allows us to provide access to any arbitrary
+metadata which may be generated.  As a practical matter, the choice of
+which entries are only in the metadata and which are part of the
+explicit structure elements is rather subjective.  Any data elements
+which are frequently used should be put in the structure; those which
+are only infrequently needed should be left in the generic metadata.
+
+There are some points of caution which must be noted.  Any
+manipulation of the data should be reflected in the metadata where
+appropriate.  This is always an issue of concern.  For example,
+consider an image of dimensions \code{Nx, Ny}.  If a function extracts
+a subraster, it must change the values of \code{Nx, Ny} to match the
+new dimensions.  What should it do to the corresponding metadata?
+Clearly, it should change the corresponding value which defines
+\code{Nx, Ny}.  However, it is not quite so simple: there may be other
+metadata values which depend on those values.  These must also be
+changed appropriately.  What if the metadata element points to a
+copy of the metadata which may be shared by other representations of
+the image?  These must be treated differently because the change would
+invalidate those other references.  
+
+A related issue is the definition of metadata names.  Entries in a
+structure have the advantage of being hardwired: every instance of
+that structure will have the same name for the same entry.  This is
+not necessarily the case with a more flexible metadata container.  The
+image exposure time is a notorious example in astronomy.  Different
+observatories use different header keywords (ie, metadata names) for
+the same concept of the exposure time (EXPTIME, EXPOSURE, OPENTIME,
+INTTIME, etc).  Any system which operates on these metadata needs to
+address the issue of identifying these names.  This issue seems like an
+argument for hardwiring metadata in the structure, but in fact it does
+not present such a strong case.  If the metadata are hardwired, some
+function will still have to know how to interpret the various names to
+populate the structure.  The concept can still be localized with
+generic metadata containers by including abstract metadata names
+within the code which are tied to the various implementations-specific
+metadata names.  
+
+\subsubsection{Metadata Representation}
 
 This section addresses the question of how \PS{} metadata should be
@@ -2403 +2413 @@
 represented on disk.
 
-\subsubsection{Metadata Representation}
-
 We propose that an item of metadata be represented as a C structure with at least the following
 fields:
@@ -2410 +2418 @@
 /** A struct to define a single item of metadata */
 typedef struct {
-    const int id;                       //!< unique ID for this item
-    char *restrict name;                //!< Name of item
-    psMetaDataType type;                //!< type of this item
-    psMetaDataFlags flags;              //!< flags associated with this item
+    const int id;                       ///< unique ID for this item
+    char *restrict name;                ///< Name of item
+    psMetaDataType type;                ///< type of this item
+    psMetaDataFlags flags;              ///< flags associated with this item
     const union {
-        float f;                        //!< floating value
-        int i;                          //!< integer value
-        void *v;                        //!< other type
-    } val;                              //!< value of metadata
-    char *comment;                      //!< optional comment ("", not NULL)
-    psDlist *restrict items;            //!< list of psMetaDataItems with the same name
+        float f;                        ///< floating value
+        int i;                          ///< integer value
+        void *v;                        ///< other type
+    } val;                              ///< value of metadata
+    char *comment;                      ///< optional comment ("", not NULL)
+    psDlist *restrict items;            ///< list of psMetaDataItems with the same name
 } psMetaDataItem;
 \end{verbatim}
@@ -2437 +2445 @@
 \begin{verbatim}
 /** Possible types of metadata. */
-typedef enum {                          //!< type of val is:
-    PS_META_ITEM_SET = 0,               //!< NULL; metadata is in psMetaDataType.items
-    PS_META_FLOAT,                      //!< float (.f)
-    PS_META_INT,                        //!< int (.i)
-    PS_META_STR,                        //!< string (.v)
-    PS_META_IMG,                        //!< image (.v)
-    PS_META_JPEG,                       //!< JPEG (.v)
-    PS_META_PNG,                        //!< PNG (.v)
-    PS_META_ASTROM,                     //!< astrometric coefficients (.v)
-    PS_META_UNKNOWN,                    //!< other (.v)
-    PS_META_NTYPE                       //!< Number of types; must be last
+typedef enum {                          ///< type of val is:
+    PS_META_ITEM_SET = 0,               ///< NULL; metadata is in psMetaDataType.items
+    PS_META_FLOAT,                      ///< float (.f)
+    PS_META_INT,                        ///< int (.i)
+    PS_META_STR,                        ///< string (.v)
+    PS_META_IMG,                        ///< image (.v)
+    PS_META_JPEG,                       ///< JPEG (.v)
+    PS_META_PNG,                        ///< PNG (.v)
+    PS_META_ASTROM,                     ///< astrometric coefficients (.v)
+    PS_META_UNKNOWN,                    ///< other (.v)
+    PS_META_NTYPE                       ///< Number of types; must be last
 } psMetaDataType;
 \end{verbatim}
@@ -2455 +2463 @@
 \textbf{Value} & \textbf{Type} & \textbf{member of union} & \textbf{Comments}\\
 \hline
-\code{PS_META_FLOAT} & float & f & value, not pointer, is stored \\
-\code{PS_META_INT} & int & i &  value, not pointer, is stored \\
-\code{PS_META_STR} & string & v &  value, not pointer to original, is stored \\
-\code{PS_META_IMG} & psImage & v &  \\
-\code{PS_META_JPEG} & JPEG & v &  \\
-\code{PS_META_PNG} & PNG & v &  \\
-\code{PS_META_ASTROM} & psAstrom & v &  \\
-\code{\code{PS_META_UNKNOWN} & other & v &  \\
-PS_META_NTYPE} & (none) & & The number of types defined
+\code{PS_META_FLOAT}   & float    & f & value, not pointer, is stored \\
+\code{PS_META_INT}     & int      & i & value, not pointer, is stored \\
+\code{PS_META_STR}     & string   & v & value, not pointer to original, is stored \\
+\code{PS_META_IMG}     & psImage  & v & \\
+\code{PS_META_JPEG}    & JPEG     & v & \\
+\code{PS_META_PNG}     & PNG      & v & \\
+\code{PS_META_ASTROM}  & psAstrom & v & \\
+\code{PS_META_UNKNOWN} & other    & v & \\
+\code{PS_META_NTYPE}   & (none)   &   & The number of types defined
 \end{tabular}
 \begin{caption}{Supported Metadata Types}
@@ -2476 +2484 @@
 /** A set of metadata */
 typedef struct {
-    psDlist *restrict list;             //!< list of psMetaDataItem
-    psHash *restrict table;             //!< hash table of the same metadata
+    psDlist *restrict list;             ///< list of psMetaDataItem
+    psHash *restrict table;             ///< hash table of the same metadata
 } psMetaDataSet;
 \end{verbatim}
@@ -2559 +2567 @@
 \begin{verbatim}
 /** Constructor */
-psMetaDataItem *psMetaDataItemAlloc(int typeFlags, //!< type of this piece of metadata + flags
-                                    const void *val, //!< value of new item N.b. a pointer even if the item
-                                                     //!< is of type e.g. int
-                                    const char *comment, //!< comment associated with item
-                                    const char *name, //!< name of new item of metadata (may be in sprintf
-                                                      //!< format)
-                                    ... //!< possible arguments for name format
+psMetaDataItem *psMetaDataItemAlloc(int typeFlags, ///< type of this piece of metadata + flags
+                                    const void *val, ///< value of new item N.b. a pointer even if the item
+                                                     ///< is of type e.g. int
+                                    const char *comment, ///< comment associated with item
+                                    const char *name, ///< name of new item of metadata (may be in sprintf
+                                                      ///< format)
+                                    ... ///< possible arguments for name format
     );
 
 /** Destructor */
-void psMetaDataItemFree(psMetaDataItem *ms //!< piece of metadata to destroy
+void psMetaDataItemFree(psMetaDataItem *ms ///< piece of metadata to destroy
     );
 /** Constructor */
-psMetaDataSet *psMetaDataSetAlloc(void);   //!< make a new set of metadata
+psMetaDataSet *psMetaDataSetAlloc(void);   ///< make a new set of metadata
 
 /** Destructor */
-void psMetaDataSetFree(psMetaDataSet *ms //!< destroy a set of metadata
+void psMetaDataSetFree(psMetaDataSet *ms ///< destroy a set of metadata
     );
 
@@ -2581 +2589 @@
 
 /// Add entry to the end of the metadata set
-psMetaDataItem *psMetaDataAppend(psMetaDataSet *restrict ms, //!< Metadata set to add to
-                                 psMetaDataItem *restrict item //!< Metatdata to add
+psMetaDataItem *psMetaDataAppend(psMetaDataSet *restrict ms, ///< Metadata set to add to
+                                 psMetaDataItem *restrict item ///< Metatdata to add
     );
 
 /// delete entry from the metadata set
-psMetaDataItem *psMetaDataRemove(psMetaDataSet *restrict ms, //!< Metadata set to delete from
-                                 const char *restrict key //!< Key to delete
+psMetaDataItem *psMetaDataRemove(psMetaDataSet *restrict ms, ///< Metadata set to delete from
+                                 const char *restrict key ///< Key to delete
     );
 
 /// reset the iterator to the start of the list
-void psMetaDataSetIterator(psMetaDataSet *ms //!< Metadata set to set iterator for
+void psMetaDataSetIterator(psMetaDataSet *ms ///< Metadata set to set iterator for
     );
 
 /// get the next entry in the sequence
-psMetaDataItem *psMetaDataGetNext(psMetaDataSet *restrict ms, //!< Metadata set to get from
-                                  const char *restrict match //!< Match this
+psMetaDataItem *psMetaDataGetNext(psMetaDataSet *restrict ms, ///< Metadata set to get from
+                                  const char *restrict match ///< Match this
     );
 
 /// find the metadata with the specified key
-psMetaDataItem *psMetaDataLookup(const psMetaDataSet *restrict ms, //!< Metadata set to look up
-                                 const char *restrict key //!< Key to find
+psMetaDataItem *psMetaDataLookup(const psMetaDataSet *restrict ms, ///< Metadata set to look up
+                                 const char *restrict key ///< Key to find
     );
 
 /// print metadata item to the specified stream
-void psMetaDataItemPrint(FILE *fd,              //!< file descriptor to write to
-                         const psMetaDataItem *restrict ms, //!< item of metadata to print
-                         const char *prefix        //!< print this at the beginning of each line
-    );
-\end{verbatim}
+void psMetaDataItemPrint(FILE *fd,              ///< file descriptor to write to
+                         const psMetaDataItem *restrict ms, ///< item of metadata to print
+                         const char *prefix        ///< print this at the beginning of each line
+    );
+\end{verbatim}
+
+\subsection{Detector and sky positions}
+
+Both detector and sky positions will be used extensively in the IPP.
+The first are linear coordinates which conform to Euclidean geometry
+while the second are angular coordinates for which additional care
+must often be taken.  Since these both consist of two coordinates with
+their associated errors, we bundle these into a single generic
+structure, \code{psCoord}, containing \code{union}s to handle the
+semantic differences.  A variety of functions should operate
+equivalently on both types of structures (\tbd{list}), while other
+functions must behave differently when the operand is an angular vs a
+linear coordinate system.
+
+\begin{verbatim}
+/** A point in 2-D space, with errors.
+ */
+typedef union {
+    struct {
+        double x;                       ///< x position
+        double y;                       ///< y position
+        double xErr;                    ///< Error in x position
+        double yErr;                    ///< Error in y position
+    } xy;
+    struct {
+        double r;                       ///< RA
+        double d;                       ///< Dec
+        double rErr;                    ///< Error in RA
+        double dErr;                    ///< Error in Dec
+    } rd;
+} psCoord;
+\end{verbatim}
+
+Three major classes of coordinate transformations are necessary.
+First, linear coordinates from one frame must be converted to linear
+coordinates in a different frame of references.  Simple transformations
+of this type are independent of other quantities of the positions --
+they are simply mapping between two linear spaces.  In practice, these
+transformations may often be a function of the magnitude or color of
+the imaged object.  The second type of conversion is the
+transformation of linear coordinates to angular coordinates and
+vice-versa.  This conversion depends on the desired projection, and
+may represent the real mapping performed by the telescope or may
+simply represent a convenient mechanism to display 3D coordinates in
+useful forms.  The third conversion of interest is the transformation
+of one set of spherical coordinates to another set.  Frequently in
+astronomy, these conversions consist only of rotations between the two
+spherical coordinates systems, where the coordinates of the pole and
+equatorial rotation between the two systems define the
+transformation.  Conversions between standard coordinate systems such
+as Galactic, Ecliptic, and various epochs of the Celestial coordinates
+are represented by these spherical transformations.  
+
+\subsubsection{Linear Coordinate Transformations}
+
+We specify two types of transforms between coordinate systems.  The
+first consists simply of two 2D polynomials to transform both
+components -- the output coordinates depend only on the input
+coordinates and no other quantities of objects at those coordinates.
+The second consists of two 4D polynomials in which the output
+coordinates are also specified to be a function of the magnitude and
+color of the object with the given coordinates.  This type of
+coordinate transformation is necessary to represent the
+(color-dependent) optical distortions caused by the atmosphere and
+camera optics, and the possibly effects of charge transfer
+inefficiency.  We specify two structures to represent the coefficients
+of these transformations:
+
+\begin{verbatim}
+/** A polynomial transformation between coordinate frames.  This may be a linear relationship, or may
+ *  represent a higher-order transformation.
+ */
+typedef struct {
+    psDPolynomial2D *x;
+    psDPolynomial2D *y;
+} psCoordXform;
+\end{verbatim}
+
+The \code{psDPolynomial2D} structures represent polynomials of
+arbitrary order as a function of two dimensions.  There is one of
+these structures for each of the two output dimensions.  As an
+example, consider the simple transformation from one linear coordinate
+frame \code{x,y} (say a single CCD) to a second frame \code{p,q} (say,
+a second CCD image). If we have only first order terms in the
+transformation \code{psCoordXform T}, the new coordinates would be
+represented by the terms:
+%
+\begin{verbatim}
+p = T.x->coeff[0][0] + x*T.x->coeff[1][0] + y*T.x->coeff[0][1];
+q = T.y->coeff[0][0] + x*T.y->coeff[1][0] + y*T.y->coeff[0][1];
+\end{verbatim}
+%
+where we have excluded the basic cross-term (\code{x*y}) by using the
+mask: \code{T.x->mask[1][1] = 0; T.y->mask[1][1] = 0;}
+
+\begin{verbatim}
+/** The optical distortion terms.  The lowest two terms are the x and y axis of the target system.  The higher
+ *  two terms represent magnitude and color terms.
+ */
+typedef struct {
+    psDPolynomial4D *x;
+    psDPolynomial4D *y;
+} psDistortion;
+\end{verbatim}
+
+Like \code{psCoordXform}, \code{psDistortion} contains two
+\code{psDPolynomial4D} structures representing polynomials of
+arbitrary order as a function of four, rather than two dimensions.
+There is one of these structures for each of the two output
+dimensions.  In this structure, the highest two dimensions could
+represent a magnitude and a color.  As an example, consider the simple
+transformation from one linear coordinate frame \code{x,y} (say a
+single CCD) of an object with magnitude and color \code{m,c} to a
+second frame \code{p,q} (say, a second CCD image). If we have only
+first order terms in the transformation \code{psCoordXform T}, the new
+coordinates would be represented by the terms:
+%
+\begin{verbatim}
+p = T.x->coeff[0][0][0][0] + x*T.x->coeff[1][0][0][0] + y*T.x->coeff[0][1][0][0] + m*T.x->coeff[0][0][1][0] + c*T.x->coeff[0][0][0][1]
+q = T.y->coeff[0][0][0][0] + x*T.y->coeff[1][0][0][0] + y*T.y->coeff[0][1][0][0] + m*T.y->coeff[0][0][1][0] + c*T.y->coeff[0][0][0][1]
+\end{verbatim}
+%
+where we have again excluded the cross-terms (\code{x*y}) by using the
+mask.
+
+We require corresponding functions to apply the transformations:
+%
+\begin{verbatim}
+/** apply the coordinate transformation to the given coordinate */
+psCoord *psCoordXformApply (psCoord *out, ///< Output coordinates, or NULL
+                            const psCoordXform *frame, ///< coordinate transformation
+                            const psCoord *coords ///< input coordiate
+    );
+\end{verbatim}
+%
+\begin{verbatim}
+/** apply the optical distortion to the given coordinate, magnitude, color */
+psCoord *psDistortionApply (psCoord *out, ///< Output coordinates, or NULL
+                            const psdistortion *pattern, ///< optical distortion pattern
+                            const psCoord *coords, ///< input coordinate
+                            float mag,  ///< magnitude of object
+                            float color ///< color of object
+    );
+\end{verbatim}
+%
+
+\subsubsection{Celestial Coordinate Conversions}
+
+We need to be able to convert between ICRS, Galactic and Ecliptic
+coordinates, and potentially between arbitrary spherical coordinate
+systems.  All of these basic spherical transformations represent
+rotations of the spherical coordinate reference.  We specify a general
+transformation function which takes a structure \code{psCoordSphere}
+defining the relationship between two spherical coordinate systems.
+We also define a function to generate \code{psCoordSphere}, which
+contains the appropriate sines and cosines of the angles, based on the
+three angles describing the location of the pole and the relative
+equatorial rotations of the two systems.  We also specify special
+coordinate transformations between standard coordinate systems which
+define the value of psCoordSphere internally and automatically.
+
+\begin{verbatim}
+/** Convert to new spherical coordinate system */
+psCoord *
+psCoordTransform (psCoord *coord, 
+                  psCoordSphere *sphere) ///< ICRS coordinates to convert
+;
+\end{verbatim}
+
+\begin{verbatim}
+/** Construct  to new spherical coordinate system */
+psCoordSphere *
+psCoordSphereDefine (double pole1, double pole2, double rotation);
+\end{verbatim}
+
+\begin{verbatim}
+/** Convert ICRS to Ecliptic */
+psCoord *
+psCoordinatesItoE(const psCoord *restrict coordinates ///< ICRS coordinates to convert
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Convert Ecliptic to ICRS */
+psCoord *
+psCoordinatesEtoI(const psCoord *restrict coordinates ///< Ecliptic coordinates to convert
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Convert ICRS to Galactic */
+psCoord *
+psCoordinatesItoG(const psCoord *restrict coordinates ///< ICRS coordinates to convert
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Convert Galactic to ICRS */
+psCoord *
+psCoordinatesGtoI(const psCoord *restrict coordinates ///< Galactic coordinates to convert
+    );
+\end{verbatim}
+
+
+\subsubsection{Projections}
+
+We require functions to convert between spherical and linear
+coordinate systems based on a variety of projections.  The required
+projections include:
+\begin{itemize}
+\item TAN
+\item SIN
+\item AIT
+\item PAR
+\item GLS
+\end{itemize}
+
+\begin{verbatim}
+psCoord *psCoordProject (psCoord *coord, char *projection);
+\end{verbatim}
+
+\subsubsection{Offsets}
+
+We require a function to calculate the offset between two positions on
+the sky, as well as a function to apply an offset to a position.
+
+\begin{verbatim}
+/** Get offset (RA,Dec) on the sky between two positions position1 and position2 may not be identical */
+psCoord *
+psGetOffset(const psCoord *restrict position1, ///< Position 1
+            const psCoord *restrict position2, ///< Position 2
+            const char *type            ///< Type of offset: Linear, Spherical/Arcsec, Spherical/Degreees etc
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Apply an offset to a position */
+psCoord *
+psApplyOffset(const psCoord *restrict position, ///< Position
+              const psCoord *restrict offset, ///< Offset
+              const char *type          ///< Type of offset: Linear, Spherical/Arcsec, Spherical/Degreees etc
+    );
+\end{verbatim}
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Astronomy Images}
 
-\subsubsection{Terminology}
-
-Some brief review of terminology would be useful so that previous
-definitions do not influence the understanding of this document.
-
-A ``readout'' is a read of the detector.
-
-A ``cell'' is defined as the smallest element of the detector readout;
-usually associated with an amplifier.  Correspondingly, each cell has
-its own overscan region.  There may be multiple readouts in a cell if
-the cell was used to provide fast guiding.
-
-A ``chip'' is defined as a contiguous piece of silicon, and consists
-of a group of cells.
-
-A ``focal plane'' is defined as a mosaic of chips, and consists of a
-group of chips.
+\subsubsection{Overview}
+
+Above, we have defined a basic container for a single 2D collection of
+pixels (\code{psImage}), along with basic operations to manipulate the
+image pixels.  For astronomical applications, this data structure is
+insufficient for two reasons.  First, it does provide sufficient
+additional metadata to describe the data in detail.  Second, astronomy
+applications frequent involve multiple, related images.  For
+PanSTARRS, and for general astronomical applications, we require a
+richer collection of data structures which describe a very general
+image concept.  We have defined several layers in the hierarchy which
+are necessary to describe the image data which will be produced by the
+PanSTARRS Gigapixel cameras as well as other standard astronomical
+images.  
+
+A simple 2D image is a basic data unit for much of astronomical
+imaging.  If we consider various optical and IR array cameras, a
+single readout of the detector produces a collection of pixels
+measurements.  We define our lowest-level astronomical image
+structure, \code{psReadout}, to contain the pixels produced by a
+single readout of the detector, along with metadata needed to define
+that readout: the origin and binning of the image relative to the
+original detector pixels explicitly in the structure, a pointer to the
+appropriate overscan region, and pointers to the general metadata and
+derived objects, if any.
+
+A single detector may produce more than one read which is associated.
+For example, infrared detectors frequently produce an image
+immediately after the detector is reset followed by an image after the
+basic exposure is complete.  Both readouts correspond to the same
+pixels, though the binning or rastering may be different between the
+two readouts.  Another example is the video sequence produced by the
+PanSTARRS Gigapix camera guide cells, each of which represents a
+series of many images from a subraster of pixels in the detector
+readout portion.  The second level of our image container hierarchy,
+\code{psCell}, consists of a collection of readouts from a single
+detector.
+
+In the PanSTARRS Gigapix camera, the basic readout region is a
+fraction of the full imaging area of a single CCD chip.  The chip is
+divided into 64 cells, any fraction of which may have been readout
+for a given exposure.  In other cameras, such as Megacam at CFHT, the
+individual CCDs have multiple amplifiers addressing contiguous
+portions of the detector.  In such cameras, each amplifier produces a
+separate collection of pixels.  In the third level of our image
+container hierarchy, the data structure \code{psChip} represents a
+collection of different cells.   
+
+The top level of our image container hierarchy is a complete focal
+plane array (\code{psFPA}).  This structure represents the collection
+of chips in the camera, all of which are read out in a given
+exposure.  
 
 For example, take a mosaic camera consisting of eight $2k\times 4k$
@@ -2641 +2930 @@
 a few cells may be dead).  Some cells (those containing guide stars
 for the orthogonal transfer) will contain multiple readouts.
+
+These data structures represent containers with which to carry around
+the collection of related image data.  There is no requirement on the
+functions or the structures that each instance of one of these data
+structures represent the physical hardware.  For example, it is not
+necessary that an instance of \code{psFPA} always carry the data for
+all 60 (or 64) Gigapixel camera OTAs.  The usage of these structures
+is such that all astronomical operations which apply to a CCD image
+should be performed on an instance of \code{psFPA}.  If a particular
+circumstance only requires a single 2D image, then that is represented
+by an instance of \code{psFPA} with one \code{psChip}, which in turn
+has one \code{psCell}, which in turn has one \code{psReadout}.  
+
+These container levels also include in their definition the information
+needed to transform the coordinates in one of the levels to the
+coordinate system relevant at the higher levels.  
 
 \subsubsection{A Readout}
@@ -2654 +2959 @@
 /** a Readout: a collection of pixels */
 typedef struct {
-    int x0, y0;                         //!< Offset from the lower-left corner
+    int x0, y0;                         ///< Offset from the lower-left corner
     psImage *image;                     ///< imaging area of cell 
     psDlist *objects;                   ///< objects derived from cell
     psImage *overscan;                  ///< bias region (subimage) of cell
-    psMetaDataSet *md;                  //!< Readout-level metadata
+    psMetaDataSet *md;                  ///< Readout-level metadata
 } psReadout;
 \end{verbatim}    
-
 
 \subsubsection{A Cell}
@@ -2686 +2990 @@
     int nReadouts;                      ///< number of readouts in this cell realization; each may have its own
                                         ///< image, objects and overscan.
-    struct psReadout *readouts;         //!< Readouts from the cell
+    struct psReadout *readouts;         ///< Readouts from the cell
     psMetaDataSet *md;                  ///< Cell-level metadata
 
@@ -2719 +3023 @@
     psMetaDataSet *md;                  ///< Chip-level metadata
     psCoordXform *chipToFPA;            ///< Transformations from chip coordinates to FPA coordinates
-    psCoordXform *FPAtoChip;            //!< Transformations from FPA coordinates to chip
+    psCoordXform *FPAtoChip;            ///< Transformations from FPA coordinates to chip
 
     struct psFPA *parentFPA;            ///< FPA which contains this chip
@@ -2734 +3038 @@
 transformation will consist of two 4D polynomials (i.e.\ a function of
 two coordinates in position, the magnitude of the object, and the
-colour of the object) in order to correct for optical distortions and
+color of the object) in order to correct for optical distortions and
 the effects of the atmosphere; hence we expect that it is prudent to
 include a reverse transformation which will be derived from
-numerically inverting the forward transformation.  Since colours are
-involved in the transformation, it is necessary to specify the colour
+numerically inverting the forward transformation.  Since colors are
+involved in the transformation, it is necessary to specify the color
 the transformation is defined for.  We also include some values to
-characterise the quality of the transformation: the root mean square
+characterize the quality of the transformation: the root mean square
 deviation for the x and y transformation fits, and the $\chi^2$ for
 the transformation fit.
@@ -2756 +3060 @@
     psDistortion *TPtoFP;               ///< Transformation term from 
     psDistortion *FPtoTP;               ///< Transformation term from 
-    psFixedPattern *pattern;            //!< Fixed pattern residual offsets
+    psFixedPattern *pattern;            ///< Fixed pattern residual offsets
     psExposure *exp;                    ///< information about this exposure
     psPhotSystem colorPlus, colorMinus; ///< Colour reference
-    float rmsX, rmsY;                   //!< Dispersion in astrometric solution
-    float chi2;                         //!< chi^2 of astrometric solution
+    float rmsX, rmsY;                   ///< Dispersion in astrometric solution
+    float chi2;                         ///< chi^2 of astrometric solution
 } psFPA;
 \end{verbatim}
-
 
 \subsubsection{Exposure information}
@@ -2775 +3078 @@
 typedef struct {
     // Telescope longitude, latitude and height are stored separately, since they don't change with pointing
-    double ra, dec;                     //!< Telescope boresight
-    double ha;                          //!< Hour angle
-    double zd;                          //!< Zenith distance
-    double az;                          //!< Azimuth
-    double lst;                         //!< Local Sidereal Time
-    float mjd;                          //!< MJD of observation
-    float rotAngle;                     //!< Rotator position angle
-    float temp;                         //!< Air temperature, for estimating refraction
-    float pressure;                     //!< Air pressure, for calculating refraction
-    float humidity;                     //!< Relative humidity, for calculating refraction
-    float exptime;                      //!< Exposure time
+    double ra, dec;                     ///< Telescope boresight
+    double ha;                          ///< Hour angle
+    double zd;                          ///< Zenith distance
+    double az;                          ///< Azimuth
+    double lst;                         ///< Local Sidereal Time
+    float mjd;                          ///< MJD of observation
+    float rotAngle;                     ///< Rotator position angle
+    float temp;                         ///< Air temperature, for estimating refraction
+    float pressure;                     ///< Air pressure, for calculating refraction
+    float humidity;                     ///< Relative humidity, for calculating refraction
+    float exptime;                      ///< Exposure time
     /* Derived quantities */
-    float posAngle;                     //!< Position angle
-    float parallactic;                  //!< Parallactic angle
-    float airmass;                      //!< Airmass, calculated from zenith distance
-    float pf;                           //!< Parallactic factor
+    float posAngle;                     ///< Position angle
+    float parallactic;                  ///< Parallactic angle
+    float airmass;                      ///< Airmass, calculated from zenith distance
+    float pf;                           ///< Parallactic factor
     char *cameraName;                   ///< name of camera which provided exposure
     char *telescopeName;                ///< name of telescope which provided exposure
-    psGrommit *grommit;                 //!< Data needed to convert from the sky to the tangent plane
+    psGrommit *grommit;                 ///< Data needed to convert from the sky to the tangent plane
 } psExposure;
 \end{verbatim}
@@ -2809 +3112 @@
 /** Constructor */
 psExposure *
-psExposureAlloc(double ra, double dec,  //!< Telescope boresight
-                double ha,              //!< Hour angle
-                double zd,              //!< Zenith distance
-                double az,              //!< Azimuth
-                double lst,             //!< Local Sidereal Time
-                float mjd,              //!< MJD
-                float rotAngle,         //!< Rotator position angle
-                float temp,             //!< Temperature
-                float pressure,         //!< Pressure
-                float humidity,         //!< Relative humidity
-                float exptime           //!< Exposure time
+psExposureAlloc(double ra, double dec,  ///< Telescope boresight
+                double ha,              ///< Hour angle
+                double zd,              ///< Zenith distance
+                double az,              ///< Azimuth
+                double lst,             ///< Local Sidereal Time
+                float mjd,              ///< MJD
+                float rotAngle,         ///< Rotator position angle
+                float temp,             ///< Temperature
+                float pressure,         ///< Pressure
+                float humidity,         ///< Relative humidity
+                float exptime           ///< Exposure time
                 );
 \end{verbatim}
@@ -2827 +3130 @@
 
 Astrometry is a basic functionality required for the IPP that will be
-used repeatedly, both for low-precision (roughly where is my favourite
+used repeatedly, both for low-precision (roughly where is my favorite
 object?) and high-precision (what is the proper motion of this star?).
 As such, it must be flexible, yet robust.  Accordingly, we will wrap
@@ -2847 +3150 @@
 
 The following steps are required to convert from the cell coordinates to
-the sky:
+the sky \tbd{convert Cell->Chip and Chip->FP to psDistortion}:
 \begin{itemize}
 \item Cell $\longleftrightarrow$ Chip: two 2D polynomials, $(X,Y) = f(x,y)$;
 \item Chip $\longleftrightarrow$ FP: two 2D polynomials, $(p,q) = g(X,Y)$;
 \item FP $\longleftrightarrow$ TP: two 4D polynomials, $(l,m) =
-h(p,q,m,c)$, where $m$ and $c$ are the magnitude and colour of the
+h(p,q,m,c)$, where $m$ and $c$ are the magnitude and color of the
 object, respectively; and
 \item TP $\longleftrightarrow$ Sky: SLALib transformation using a
@@ -2861 +3164 @@
 Plane is a four-dimensional polynomial, in order to account for any
 possible dependencies in the astrometry on the stellar magnitude and
-colour; the former serves as a check for charge transfer
+color; the former serves as a check for charge transfer
 inefficiencies, while the latter will correct chromatic refraction,
 both through the atmosphere and the corrector lenses.
-
-\textbf{[If the magnitude terms serve to check CTI, then shouldn't we
-put them in the cell $\leftrightarrow$ chip section?]}
 
 We require structures to contain each of the above transformations as
@@ -2897 +3197 @@
 \end{verbatim}
 
-
 \subsubsection{Fixed Pattern}
 
@@ -2911 +3210 @@
 /** The fixed pattern residual offsets.  These are specified via a coarse grid of x and y offsets. */
 typedef struct {
-    int nX, nY;                         //!< Number of elements in x and y
-    double x0, y0;                      //!< Position of the lower-left corner of the grid on the focal plane
-    double xScale, yScale;              //!< Scale of the grid
-    double **x, **y;                    //!< The grid of offsets in x and y
+    int nX, nY;                         ///< Number of elements in x and y
+    double x0, y0;                      ///< Position of the lower-left corner of the grid on the focal plane
+    double xScale, yScale;              ///< Scale of the grid
+    double **x, **y;                    ///< The grid of offsets in x and y
 } psFixedPattern;
 \end{verbatim}
 
 
-\subsubsection{Finding}
+\subsubsection{Position Finding}
 
 We require functions to return the structure containing given
@@ -2932 +3231 @@
 /** returns Chip in FPA which contains the given FPA coordinate */
 psChip *
-psChipInFPA (psChip *out,               //!< Chip to return, or NULL
+psChipInFPA (psChip *out,               ///< Chip to return, or NULL
              const psFPA *fpa,          ///< FPA description
              const psCoord *coord       ///< coordinate in FPA
@@ -2941 +3240 @@
 /** returns Cell in Chip which contains the given chip coordinate */
 psCell *
-psCellInChip(psCell *out,               //!< Cell to return, or NULL
+psCellInChip(psCell *out,               ///< Cell to return, or NULL
              const psChip *chip,        ///< chip description
              const psCoord *coord       ///< coordinate in chip
@@ -2954 +3253 @@
 /** Return the cell in FPA which contains the given FPA coordinates */
 psCell *
-psCellInFPA(psCell *out,                //!< Cell to return, or NULL
-            const psFPA *fpa,           //!< FPA description
-            const psCoord *coord        //!< Coordinate in FPA
+psCellInFPA(psCell *out,                ///< Cell to return, or NULL
+            const psFPA *fpa,           ///< FPA description
+            const psCoord *coord        ///< Coordinate in FPA
             );
 \end{verbatim}
@@ -2965 +3264 @@
 
 We require functions to convert between the various coordinate frames
-(Section~\ref{sec:coordinateFrames}).  The heirarchy of the coordinate
+(Section~\ref{sec:coordinateFrames}).  The hierarchy of the coordinate
 frames and the transformations between each are shown in
 Figure~\ref{fig:coco}.  The functions that employ the transformations
 are shown in Figure~\ref{fig:cocoFunc}.  In addition to
 transformations between each adjoining coordinate frame in the
-heirarchy, we also require higher-level functions to convert between
+hierarchy, we also require higher-level functions to convert between
 the Cell and Sky coordinate frames; these will simply perform the
 intermediate steps.
@@ -2993 +3292 @@
 /** Convert (RA,Dec) to cell and cell coordinates */
 psCoord *
-psCoordSkyToCell(psCoord *out,          //!< Coordinates to return, or NULL
-                 psCell *cell,          //!< Cell to return
-                 const psFPA *fpa       //!< FPA description
+psCoordSkyToCell(psCoord *out,          ///< Coordinates to return, or NULL
+                 psCell *cell,          ///< Cell to return
+                 const psFPA *fpa       ///< FPA description
                  );
 \end{verbatim}
@@ -3002 +3301 @@
 /** Convert cell and cell coordinate to (RA,Dec) */
 psCoord *
-psCoordCellToSky(psCoord *out,          //!< Coordinates to return, or NULL
-                 const psCell *cell,    //!< Cell to get coordinates for
-                 const psCoord *coord   //!< cell coordinates to transform
+psCoordCellToSky(psCoord *out,          ///< Coordinates to return, or NULL
+                 const psCell *cell,    ///< Cell to get coordinates for
+                 const psCoord *coord   ///< cell coordinates to transform
                  );
 \end{verbatim}
@@ -3011 +3310 @@
 /** Quick and dirty cell to (RA,Dec) --- employs cellToSky transformation */
 psCoord *
-psCoordCellToSkyQuick(psCoord *out,     //!< Coordinates to return, or NULL
-                      const psCell *cell, //!< Cell description
-                      const psCoord *coord //!< cell coordinates to transform
+psCoordCellToSkyQuick(psCoord *out,     ///< Coordinates to return, or NULL
+                      const psCell *cell, ///< Cell description
+                      const psCoord *coord ///< cell coordinates to transform
                       );
 \end{verbatim}
@@ -3020 +3319 @@
 /** Convert (RA,Dec) to tangent plane coords */
 psCoord *
-psCoordSkyToTP(psCoord *out,            //!< Coordinates to return, or NULL
-               const psExposure *exp,   //!< Exposure description
-               const psCoord *coord     //!< input Sky coordinate
+psCoordSkyToTP(psCoord *out,            ///< Coordinates to return, or NULL
+               const psExposure *exp,   ///< Exposure description
+               const psCoord *coord     ///< input Sky coordinate
                );
 \end{verbatim}
@@ -3029 +3328 @@
 /** Convert tangent plane coords to focal plane coordinates */
 psCoord *
-psCoordTPtoFPA(psCoord *out,            //!< Coordinates to return, or NULL
-               const psFPA *fpa,        //!< FPA description
-               const psCoord *coord     //!< input TP coordinate
+psCoordTPtoFPA(psCoord *out,            ///< Coordinates to return, or NULL
+               const psFPA *fpa,        ///< FPA description
+               const psCoord *coord     ///< input TP coordinate
                );
 \end{verbatim}
@@ -3038 +3337 @@
 /** converts the specified FPA coord to the coord on the given Chip */
 psCoord *
-psCoordFPAtoChip (psCoord *out,         //!< Coordinates to return, or NULL
+psCoordFPAtoChip (psCoord *out,         ///< Coordinates to return, or NULL
                   const psChip *chip,   ///< Chip of interest
                   const psCoord *coord  ///< input FPA coordinate
@@ -3047 +3346 @@
 /** converts the specified Chip coord to the coord on the given Cell */
 psCoord *
-psCoordChiptoCell (psCoord *out,        //!< Coordinates to return, or NULL
+psCoordChiptoCell (psCoord *out,        ///< Coordinates to return, or NULL
                    const psCell *cell,  ///< Cell of interest
                    const psCoord *coord ///< input Chip coordinate
@@ -3056 +3355 @@
 /** converts the specified Cell coord to the coord on the parent Chip */
 psCoord *
-psCoordCelltoChip (psCoord *out,        //!< Coordinates to return, or NULL
+psCoordCelltoChip (psCoord *out,        ///< Coordinates to return, or NULL
                    const psCell *cell,  ///< Cell description
                    const psCoord *coord ///< input Cell coordinate
@@ -3065 +3364 @@
 /** converts the specified Chip coord to the coord on the parent FPA */
 psCoord *
-psCoordChiptoFPA (psCoord *out,         //!< Coordinates to return, or NULL
+psCoordChiptoFPA (psCoord *out,         ///< Coordinates to return, or NULL
                   const psChip *chip,   ///< Chip description
                   const psCoord *coord  ///< input Chip coordinate
@@ -3074 +3373 @@
 /** Convert focal plane coords to tangent plane coordinates */
 psCoord *
-psCoordFPAToTP(psCoord *out,            //!< Coordinates to return, or NULL
-               const psFPA *fpa,        //!< FPA description
-               const psCoord *coord     //!< input FPA coordinate
+psCoordFPAToTP(psCoord *out,            ///< Coordinates to return, or NULL
+               const psFPA *fpa,        ///< FPA description
+               const psCoord *coord     ///< input FPA coordinate
                );
 \end{verbatim}
@@ -3083 +3382 @@
 /** Convert tangent plane coords to (RA,Dec) */
 psCoord *
-psCoordTPtoSky(psCoord *out,            //!< Coordinates to return, or NULL
-               const psExposure *exp,   //!< Exposure description
-               const psCoord *coord     //!< input TP coordinate
+psCoordTPtoSky(psCoord *out,            ///< Coordinates to return, or NULL
+               const psExposure *exp,   ///< Exposure description
+               const psCoord *coord     ///< input TP coordinate
                );
 \end{verbatim}
@@ -3092 +3391 @@
 /** Convert Cell coords to FPA coordinates */
 psCoord *
-psCoordCellToFPA(psCoord *out,          //!< Coordinates to return, or NULL
-                 const psCell *cell,    //!< Cell description
-                 const psCoord *coord   //!< Input cell coordinates
+psCoordCellToFPA(psCoord *out,          ///< Coordinates to return, or NULL
+                 const psCell *cell,    ///< Cell description
+                 const psCoord *coord   ///< Input cell coordinates
                  );
 \end{verbatim}
@@ -3108 +3407 @@
 /** Get the airmass for a given position and sidereal time */
 float
-psGetAirmass(const psCoord *coord,      //!< Position on the sky
-             double siderealTime,       //!< Sidereal time
-             float height               //!< Height above sea level
+psGetAirmass(const psCoord *coord,      ///< Position on the sky
+             double siderealTime,       ///< Sidereal time
+             float height               ///< Height above sea level
              );
 \end{verbatim}
@@ -3117 +3416 @@
 /** Get the parallactic angle for a given position and sidereal time */
 float
-psGetParallactic(const psCoord *coord,  //!< Position on the sky
-                 double siderealTime    //!< Sidereal time
+psGetParallactic(const psCoord *coord,  ///< Position on the sky
+                 double siderealTime    ///< Sidereal time
                  );
 \end{verbatim}
@@ -3125 +3424 @@
 /** Estimate atmospheric refraction, along the parallactic */
 float
-psGetRefraction(float colour,           //!< Colour of object
+psGetRefraction(float colour,           ///< Colour of object
                 psPhotSystem colorPlus, ///< Colour reference
                 psPhotSystem colorMinus, ///< Colour reference
-                const psExposure *exp   //!< Telescope pointing information, for airmass, temp and pressure
+                const psExposure *exp   ///< Telescope pointing information, for airmass, temp and pressure
                 );
 \end{verbatim}
@@ -3135 +3434 @@
 /** Calculate the parallax factor */
 double
-psGetParallaxFactor(const psExposure *exp //!< Exposure details
+psGetParallaxFactor(const psExposure *exp ///< Exposure details
     );
 \end{verbatim}
@@ -3161 +3460 @@
 \begin{verbatim}
 typedef struct {
-    int ID;                             //!< ID number for this photometric system
-    char *name;                         //!< Name of photometric system
-    char *camera;                       //!< Camera for photometric system
-    char *filter;                       //!< Filter used for photometric system
-    char *detector;                     //!< Detector used for photometric system
+    int ID;                             ///< ID number for this photometric system
+    char *name;                         ///< Name of photometric system
+    char *camera;                       ///< Camera for photometric system
+    char *filter;                       ///< Filter used for photometric system
+    char *detector;                     ///< Detector used for photometric system
 } psPhotSystem;
 \end{verbatim}
@@ -3173 +3472 @@
 \begin{verbatim}
 typedef struct {
-    psPhotSystem src;                   //!< Source photometric system
-    psPhotSystem dst;                   //!< Destination photometric system
+    psPhotSystem src;                   ///< Source photometric system
+    psPhotSystem dst;                   ///< Destination photometric system
     psPhotSystem pP, pM;                ///< Primary colour reference
     psPhotSystem sP, sM;                ///< Secondary colour reference
     float pA, sA;                       ///< Colour offset for primary and secondary references
-    psPolynomial3D transform;           //!< Transformation from source to destination
+    psPolynomial3D transform;           ///< Transformation from source to destination
 } psPhotTransform;
 \end{verbatim}
@@ -3192 +3491 @@
 are highly reddened, for example.  Derived photometric quantities may
 have been corrected for airmass variations, in which case only color
-terms may be measureable.  The structure defines the transformation
+terms may be measurable.  The structure defines the transformation
 between a source photometric system (\code{src}) and a target
 photometric system (\code{dst}).  The photometric system of a primary
@@ -3213 +3512 @@
 \subsection{Astronomical objects}
 
-\textbf{[Deferred.]}
+\subsubsection{Positions of Major SS Objects}
+
+We require the ability to calculate the position of major Solar System
+objects, as well as Lunar phase.
+
+\begin{verbatim}
+/** Get Sun Position */
+psCoord *
+psGetSunPos(float mjd                   ///< MJD to get position for
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Get Moon position */
+psCoord *
+psGetMoonPos(float mjd,                 ///< MJD to get position for
+             double latitude,           ///< Latitude for apparent position
+             double longitude           ///< Longitude for apparent position
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Get Moon phase */
+float
+psGetMoonPhase(float mjd                ///< MJD to get phase for
+    );
+\end{verbatim}
+
+\begin{verbatim}
+/** Get Planet positions */
+psCoord *
+psGetSolarSystemPos(const char *solarSystemObject, ///< Named S.S. object
+                    float mjd           ///< MJD to get position for
+    );
+\end{verbatim}
+
+\tbd{The rest of this section is tentative}
+
+We may need a variety of other SLALib-type functions:
+
+\begin{itemize}
+\item Lunation calculations to/from psTime
+\item rise/set for specified object
+\item 12 \& 18 deg twilight for closest sunrise / sunset
+\item length of current night
+\end{itemize}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%