
\subsection{FPA-related file I/O, file naming, and supporting data}

The Pan-STARRS IPP image analysis modules support a very wide range of
image file formats.  The analysis programs perform a wide range of
operations and have potentially many different input and output files
which need to be defined.  The \code{pmFPAfile} system allows a
program to define complex related I/O operations which may be
easily controlled within the psModules configuration system.  

To motivate the discussion of the \code{pmFPAfile} system
capabilities, let us consider a few examples of the types of I/O
operations which must be regularly performed.  One of the work-horse
programs is \code{ppImage} which performs the basic detrending
processing as well as optional object detection, astrometry, cell/chip
mosaicing, and image jpeg creation.  Even for the simplest type of
image file, a single array FITS file, this program needs to be able to
determine filenames for up to 8 different types of output data
products:
\begin{itemize}
\item detrended image
\item background model
\item background-subtracted
\item object-subtracted
\item object-table
\item astrometrically calibrated table
\item jpeg output image
\item psf-model output
\end{itemize}
In addition, there are at least 6 different potential input files in
addition to the image of interest which may need to be determined:
\begin{itemize}
\item input image
\item mask image
\item bias image
\item dark image
\item flat image
\item fringe image
\item shutter correction image
\item cross-talk matrix
\item supplied psf-model
\end{itemize}
The situation becomes substantially more complex when the image file
consists of more complex data structures.  The input image files may
represent a focal plane divided into multiple chips, each read out
through multiple amplifiers (ie, multiple cells).  A single camera
exposure may be saved as a single FITS file with many extensions,
multiple basic FITS files, or multiple FITS files with multiple
extensions.  The program must be able to determine the filename for
each of these elements as appropriate to the camera of interest.  The
\code{pmFPAfile} system provides a flexible and convenient mechanism
to address this complex situation.  

There are two possible approaches to the filenaming problem which
represent opposite extremes.  In the one extreme, the program could
require the user to define each of the input and output filenames on
the command-line with flags to distinguish the different types of
data.  Even for the case of a simple camera image format, the required
command lines become rather complex.  For the case of the multiple
files or multiple extension formats, the situation quickly becomes
prohibitively cumbersome.  In the opposite extreme, the program could
use rigid rules which define the name of an output file, perhaps based
on the input file names.  For example, extensions or extra letters
could be added to the filenames to denote the different types of
output data products.  In such a scenario, the output object table for
a given input image would always have the same resulting name.  This
option makes it easy to write the commands, but is likely to be too
rigid to support analysis of varied data in a range of environments.  

The \code{pmFPAfile} system uses a compromise between the above two
systems to allow the user to define {\it rules} which determine the
input and output names (and in the case of input data, their sources).
The rules may thus be different for different cameras, or even
different for the same camera in different contexts, while at the same
time freeing the user from the need to specify every output file for
every run of the program.

The \code{pmFPAfile} system is also designed to simplify the process
of writing a program in which there are many possible, optional I/O
operations, especially in the context of different library functions
which may need to interact with the same set of data products.  

One of the programming issues which the \code{pmFPAfile} system
addresses is how to supply the many possible I/O concepts to
individual APIs within a program.  Consider a high-level function
which performs a rich set of operations, for example an object
detection process.  The function may require a certain set of input
data (eg, the input image) and may always produce an additional set of
output data (eg, the set of detected objects).  However, there may be
other data concepts which may be optionally supplied to or generated
by the function depending on the context.  For example, the function
may allow the calling function to supply an optional model of the
image PSF.  Or, the function may generate an object-subtracted image,
but only supply it back to the calling function if requested (eg,
freeing it otherwise).  \note{these do not necessarily represent I/O
  operations.  is there a more general concept which the pmFPAfile is
  addressing, not just I/O?}  How might these be supplied to the
function, and how are the choices defined?  

One possibility is that the function simply include in its API of all
of the inputs and outputs as separate function parameters.  If the
parameter is NULL, the option is not chosen.  This is a perfectly
workable solution, but makes for potentially very long function APIs.
If most of the data types are the same (eg, some image structure),
then there is the danger that the calling program will not get the
elements in the correct order if the program or the compiler cannot
distinguish between elements of the same type.  

Another possibility is to define an structure for that function in
which the required and optional input and output data elements are
represented by separate structure elements.  This is also a very
workable solution, with some aesthetic drawbacks.  If each API has a
separate structure, then each function call must be proceeded by a
cumbersome sequence of steps to set up the data elements to be used by
the function call.  If a single structure is supplied to all APIs,
then it will be necessary to maintain the synchronization of the
structure elements between all libraries and programs which use it.  

The \code{pmFPAfile} system uses a dynamic structure (a
\code{psMetadata}) to register the data 
