Index: /trunk/doc/modules/pmFPAfile.tex
===================================================================
--- /trunk/doc/modules/pmFPAfile.tex	(revision 8844)
+++ /trunk/doc/modules/pmFPAfile.tex	(revision 8844)
@@ -0,0 +1,120 @@
+
+\subsection{FPA-related file I/O, file naming, and supporting data}
+
+The Pan-STARRS IPP image analysis modules support a very wide range of
+image file formats.  The analysis programs perform a wide range of
+operations and have potentially many different input and output files
+which need to be defined.  The \code{pmFPAfile} system allows a
+program to define complex related I/O operations which may be
+easily controlled within the psModules configuration system.  
+
+To motivate the discussion of the \code{pmFPAfile} system
+capabilities, let us consider a few examples of the types of I/O
+operations which must be regularly performed.  One of the work-horse
+programs is \code{ppImage} which performs the basic detrending
+processing as well as optional object detection, astrometry, cell/chip
+mosaicing, and image jpeg creation.  Even for the simplest type of
+image file, a single array FITS file, this program needs to be able to
+determine filenames for up to 8 different types of output data
+products:
+\begin{itemize}
+\item detrended image
+\item background model
+\item background-subtracted
+\item object-subtracted
+\item object-table
+\item astrometrically calibrated table
+\item jpeg output image
+\item psf-model output
+\end{itemize}
+In addition, there are at least 6 different potential input files in
+addition to the image of interest which may need to be determined:
+\begin{itemize}
+\item input image
+\item mask image
+\item bias image
+\item dark image
+\item flat image
+\item fringe image
+\item shutter correction image
+\item cross-talk matrix
+\item supplied psf-model
+\end{itemize}
+The situation becomes substantially more complex when the image file
+consists of more complex data structures.  The input image files may
+represent a focal plane divided into multiple chips, each read out
+through multiple amplifiers (ie, multiple cells).  A single camera
+exposure may be saved as a single FITS file with many extensions,
+multiple basic FITS files, or multiple FITS files with multiple
+extensions.  The program must be able to determine the filename for
+each of these elements as appropriate to the camera of interest.  The
+\code{pmFPAfile} system provides a flexible and convenient mechanism
+to address this complex situation.  
+
+There are two possible approaches to the filenaming problem which
+represent opposite extremes.  In the one extreme, the program could
+require the user to define each of the input and output filenames on
+the command-line with flags to distinguish the different types of
+data.  Even for the case of a simple camera image format, the required
+command lines become rather complex.  For the case of the multiple
+files or multiple extension formats, the situation quickly becomes
+prohibitively cumbersome.  In the opposite extreme, the program could
+use rigid rules which define the name of an output file, perhaps based
+on the input file names.  For example, extensions or extra letters
+could be added to the filenames to denote the different types of
+output data products.  In such a scenario, the output object table for
+a given input image would always have the same resulting name.  This
+option makes it easy to write the commands, but is likely to be too
+rigid to support analysis of varied data in a range of environments.  
+
+The \code{pmFPAfile} system uses a compromise between the above two
+systems to allow the user to define {\it rules} which determine the
+input and output names (and in the case of input data, their sources).
+The rules may thus be different for different cameras, or even
+different for the same camera in different contexts, while at the same
+time freeing the user from the need to specify every output file for
+every run of the program.
+
+The \code{pmFPAfile} system is also designed to simplify the process
+of writing a program in which there are many possible, optional I/O
+operations, especially in the context of different library functions
+which may need to interact with the same set of data products.  
+
+One of the programming issues which the \code{pmFPAfile} system
+addresses is how to supply the many possible I/O concepts to
+individual APIs within a program.  Consider a high-level function
+which performs a rich set of operations, for example an object
+detection process.  The function may require a certain set of input
+data (eg, the input image) and may always produce an additional set of
+output data (eg, the set of detected objects).  However, there may be
+other data concepts which may be optionally supplied to or generated
+by the function depending on the context.  For example, the function
+may allow the calling function to supply an optional model of the
+image PSF.  Or, the function may generate an object-subtracted image,
+but only supply it back to the calling function if requested (eg,
+freeing it otherwise).  \note{these do not necessarily represent I/O
+  operations.  is there a more general concept which the pmFPAfile is
+  addressing, not just I/O?}  How might these be supplied to the
+function, and how are the choices defined?  
+
+One possibility is that the function simply include in its API of all
+of the inputs and outputs as separate function parameters.  If the
+parameter is NULL, the option is not chosen.  This is a perfectly
+workable solution, but makes for potentially very long function APIs.
+If most of the data types are the same (eg, some image structure),
+then there is the danger that the calling program will not get the
+elements in the correct order if the program or the compiler cannot
+distinguish between elements of the same type.  
+
+Another possibility is to define an structure for that function in
+which the required and optional input and output data elements are
+represented by separate structure elements.  This is also a very
+workable solution, with some aesthetic drawbacks.  If each API has a
+separate structure, then each function call must be proceeded by a
+cumbersome sequence of steps to set up the data elements to be used by
+the function call.  If a single structure is supplied to all APIs,
+then it will be necessary to maintain the synchronization of the
+structure elements between all libraries and programs which use it.  
+
+The \code{pmFPAfile} system uses a dynamic structure (a
+\code{psMetadata}) to register the data