Index: /tags/ipp-docs/SDRS-03/doc/modules/ChangeLogSDRS.tex =================================================================== --- /tags/ipp-docs/SDRS-03/doc/modules/ChangeLogSDRS.tex (revision 22183) +++ /tags/ipp-docs/SDRS-03/doc/modules/ChangeLogSDRS.tex (revision 22183) @@ -0,0 +1,39 @@ +%%% $Id: ChangeLogSDRS.tex,v 1.11 2005-01-22 01:59:10 eugene Exp $ + +\subsection{Changes from version 00 to version 01} + +\begin{itemize} +\item clarified the image offsets for pmFlatField () +\item changed return value to bool. +\item added \code{pmCameraFromHeader} +\item added \code{pmCameraValidateHeaders} +\item added \code{pmFPAfromHeader} +\item Added \code{pmReadoutCombine} +\end{itemize} + +\subsection{Changes from version 01 to version 02} + +\begin{itemize} +\item \code{nBin} in \code{pmSubtractBias} is also interpreted as the + number of spline pieces if spline fitting is specified. +\item Refined \code{pmReadoutCombine} specification in response to bug 227. +\item added details to the functions \code{pmCameraFromHeader}, \code{pmCameraValidateHeaders}, and \code{pmFPAfromHeader}. + +\item reorganiztion: placed configuration section up front, camera layout next, etc +\item added details about configuration system +\item added utility modules \code{pmConfigLoadSite}, \code{pmConfigLoadCamera}, \code{pmConfigLoadRecipe} +\item added utility modules \code{pmConfigLookupSTR}, \code{pmConfigLookupS32}, \code{pmConfigLookupF64}, \code{pmConfigLookupRegion}, +\item added discussion about Coordinate transforms +\item added discussion about \code{pmReadoutLoad} +\item added module \code{pmSubtractSky} +\end{itemize} + +\subsection{Changes from version 02 to version 03} + +\begin{itemize} +\item Fixed up specification of \code{fitSpec} for \code{pmSubtractSky}. +\item Added a \code{mask} to \code{pmSubtractSky}, and specified that binned pixels which are clipped may be interpolated over, or simply ignored. +\item Added further explanation for \code{pmReadoutCombine}. +\item Added Object Detection section +\item Added PSPhot pseudo-C example +\end{itemize} Index: /tags/ipp-docs/SDRS-03/doc/modules/ModulesSDRS.tex =================================================================== --- /tags/ipp-docs/SDRS-03/doc/modules/ModulesSDRS.tex (revision 22183) +++ /tags/ipp-docs/SDRS-03/doc/modules/ModulesSDRS.tex (revision 22183) @@ -0,0 +1,1664 @@ +%%% $Id: ModulesSDRS.tex,v 1.28 2005-01-22 01:59:10 eugene Exp $ +\documentclass[panstarrs]{panstarrs} + +% basic document variables +\title{Pan-STARRS PS-1 Image Processing Pipeline Modules} +\subtitle{Supplementary Design Requirements} +\shorttitle{Modules SDRS} +\author{Paul Price, Eugene Magnier} +\audience{Pan-STARRS PMO} +\group{Pan-STARRS Algorithm Group} +\project{Pan-STARRS Image Processing Pipeline} +\organization{Institute for Astronomy} +\version{03} +\docnumber{PSDC-430-012} + +\setlength{\topsep}{-2pt} + +\begin{document} +\maketitle +\sloppy + +% -- Revision History -- +% provide explicit values for the old versions +% use '\theversion' for the current version (set above) +% use \hline between each table row +\RevisionsStart +% version Date Description +DR & 2004 Jun 7 & Draft \\ \hline +00 & 2004 Aug 16 & final for cycle 3 \\ \hline +01 & 2004 Oct 12 & draft for cycle 4 \\ \hline +02 & 2004 Nov 30 & final for cycle 4 \\ \hline +03 & 2005 Jan 21 & draft for cycle 5 \\ \hline +\RevisionsEnd + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\DocumentsInternal +PSCD-230-001 & PS-1 Design Reference Mission \\ \hline +PSDC-430-004 & Pan-STARRS PS-1 IPP C Code Conventions \\ \hline +PSDC-430-005 & Pan-STARRS PS-1 IPP Software Requirements Specification \\ \hline +PSDC-430-006 & Pan-STARRS PS-1 IPP Algorithm Design Document \\ \hline +PSDC-430-011 & Pan-STARRS PS-1 IPP System/Subsystem Design Description \\ \hline +\DocumentsExternal +Posix Standard & Open Group Based Specifications Issue 6, IEEE Std 1003.1, 2003 \\ +\DocumentsEnd + +\tableofcontents +\pagebreak +\pagenumbering{arabic} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\section{Introduction} + +This document describes the Pan-STARRS Image Processing Pipeline (IPP) +data analysis Modules. The Modules use the functionality of the +Pan-STARRS Library (PSLib) to perform more complex tasks, especially +tasks which require assumptions of astronomical analysis or the data +organization. Within the IPP, the Modules are tied together into +programs which perform complete data analysis tasks (an ``analysis +stage''). The modules may be tied together within a C framework or +using a high-level scripting language. Bindings of the Modules are +made available to the scripting language using the program SWIG. + +In order to preserve name space, globally-visible structures and +functions shall be prefixed with \code{pm}, for ``Pan-STARRS +Modules''. + +\section{Runtime Configuration Data} + +PSLib defines a \code{psMetadata} structure which can carry labeled +data of arbirtary types. The associated functions implemented by +PSLib consist of tools to manipulate and extract data from +\code{psMetadata} collections. Within PSLib, the \code{psMetadata} +stucture is used to carry the data from a FITS header. In addition, +functions are available to fill a \code{psMetadata} collection from a +text-based configuration file using a particular syntax, and to fill a +\code{psMetadata} collection from a properly formatted XML document. + +In the IPP Modules, we use \code{psMetadata} collections to carry +run-time configuration data used by the data analysis modules. Below, +in the discussion of the various modules, this configuration +information is defined by specifying the name of the data item of +interest, the conceptual meaning of that data item, and the allowed +values for the data item. In this section, we discuss top-level +concepts related to the configuration information, including the +sources of the run-time configuration data and special +operations used to extract information from the configuration system. + +\subsection{Configuration Data Sources} + +All modules need to load some configuration information defining +parameters which may be configured at run-time. Some module require +only a basic amount of information, while others require configuration +information defining the camera or details of an analysis (the {\it +recipe}). We define three utility functions to load these files from +standard locations, with optional command-line override values. + +\begin{verbatim} +bool pmConfigLoadSite (psMetadata **site, int *argc, char **argv); +bool pmConfigLoadCamera (psMetadata **camera, psMetadata *site, psMetadata *header, int *argc, char **argv); +bool pmConfigLoadRecipe (psMetadata **recipe, psMetadata *camera, int *argc, char **argv, char *script); +\end{verbatim} + +\code{pmConfigLoadSite} loads basic configuration information from a +general configuration file. By default, this file has the name +\code{~/.ipprc} in the user's home directory. This file may be +overridden on the command line with the optional flag \code{-site +(file)}. + +% eventually, we should define the complete sequence: +% $FOO, ~/.foorc, /prefix/config/foorc + +\code{pmConfigLoadCamera} loads camera configuration information from +a file which depends on the camera of interest. In some cases, the +camera may be determined from the header of an image currently in +hand. In other cases, the camera must be specified without reference +to a file. This function attempts to determine the appropriate camera +configuration file by examining, in order, the command-line options, +the header data, and the site configuration data. The camera config +file may be specified on the command line with the option +\code{-camera (file)}. If this does not exist, the +\code{pmConfigLoadCamera} examines the provided header to determine +the camera by using the function \code{pmCameraFromHeader} (defined +below). If this fails, the function attempts to determine the camera +from the provided site config data, looking for the keyword +\code{CAMERA} in the site config data. The camera name is used to +select the camera configuration file, defined in the site +configuration file with the keyword \code{CAMERA.name}, where +\code{name} is substituted with the name of the camera. + +\code{pmConfigLoadRecipe} loads the recipe configuration information +from a file which depends on the camera of interest and the particular +analysis task. The camera configuration file must define recipe files +for each of the analysis tasks of interest. The function attempts to +load the recipe file using the keyword \code{RECIPE.name}, where +\code{name} is substituted with the name of the analysis task, for +example \code{PHASE2}. The choice of the recipe file may be +overridden on the command line with the option \code{-recipe (file)}. + +\subsection{Indirect Configuration Data} + +Some configuration data is not provided directly by the configuration +files. Instead, the configuration system provides a mechanism to +define indirect references to configuration data. Three indirect +configuration data sources are currently defined: data from a FITS +header, data from the Metadata Database, and data from the Pan-STARRS +Status Server. The configuration data may specify that a certain +value is provided by one of these mechanisms. + +In order to specify that the configuration data should be derived from +a FITS header, the data value is given in the form: \code{HD:KEYWORD}, +where \code{KEYWORD} specifies the keyword to be used in search the +FITS header. + +If the data is to be extracted from the Metadata Database, the data +value is given in the form: \code{MD:TABLE,FIELD,KEY}. In this case, +\code{TABLE} specifies the Metadata Database table to be used to find +the value, \code{FIELD} specifies the field in that table to be used, +and \code{KEY} specifies the value of the primary key which gives the +value of interest. + +If the data is to be extracted from the Status Server, the data value +is given in the form: \code{SS:PATH}. In this case, \code{PATH} +specifies the Status Server path to the data value of interest. + +{\bf \it note that the Metadata Database and Status Server + interactions require us to define the APIs for these interactions. + These two methods are not available until the MD and SS APIs are + defined.} + +As an example, consider the definition of the data region of an image. +This value is frequenty represented in the FITS header with the +keyword \code{DATASEC}. However, in some cameras, this value may not +be defined, or another value may be used. We must be able to flexibly +define both the appropriate keyword, or the actual value, if known. +These two cases may be written as follows, using the configuration +file format parsed by \code{psMetadataParseConfig}: +\begin{verbatim} +DATA.REGION STR [1:100,2:400] +DATA.REGION STR HD:DATASEC +\end{verbatim} +In the first case, the data provided in the configuration file is the +data of interest. (\code{[1:100,2:400]}). In the second case, the +three-character code \code{HD:} specifies that the value of interest +may be determined by searching for the value associated with the +header keyword \code{DATASEC}. + +We provide utilities functions to extract this type of potentially +indirect configuration information. We provide four functions +returning four data types. The first three return the value from the +appropriate location assuming the data type to be \code{char*}, +\code{psS32}, and \code{psF64}. The fourth version searches for a +string which is passed to the function \code{psRegionFromString} +before being returned as a \code{psRegion} pointer. + +\begin{verbatim} +psSTR pmConfigLookupSTR (psMetadata *config, psMetadata *header, char *name); +psS32 pmConfigLookupS32 (psMetadata *config, psMetadata *header, char *name); +psF64 pmConfigLookupF64 (psMetadata *config, psMetadata *header, char *name); +psRegion *pmConfigLookupRegion (psMetadata *config, psMetadata *header, char *name); +\end{verbatim} + +\section{Camera Data Organization \& Camera Geometry} + +We require several utility functions to define the geometry of the +detectors in a camera and to specify the organization of the camera +data in real FITS images. The camera data organization is defined in +a set of \code{psMetadata} structures, and may be stored on disk in +the file format parsed by \code{psMetadataParseConfig}. + +PSLib defines a hierarchy of data structures related to the +organization of the pixels in a camera. These structures, starting +from the top level, follow the sequence \code{psFPA}, \code{psChip}, +\code{psCell}, \code{psReadout}, \code{psImage}. Each lower level +structure is carried as an array in the higher level. The containers +as defined by PSLib include functions which specify the astrometric +relationships between these levels, and provide a single +\code{psMetadata} container pointer for each level. In this section, +we define the type of metadata is stored in these metadata containers +and how the image headers are parse to define the data hierarchy in +memory. + +\subsection{Camera Config Data \& the {\tt psFPA}} + +A relevant collection of metadata is the externally supplied camera +definition metadata loaded with the function +\code{pmConfigLoadCamera}. This metadata defines the expected layout +of a specific camera along with information detailing how to interpret +the headers for an image from that camera. The camera metadata +consists of keyword / value pairs which are relevant to the entire +camera, and a set of metadata collections specified for each cell. +This top-level metadata is added to the \code{psFPA} with the name +'CAMERA.LAYOUT'. For example: +\begin{verbatim} +fpa->metadata = psMetadataAdd (fpa->metadata, PS_LIST_TAIL, "CAMERA", PS_META_META, "camera", camera); +\end{verbatim} +The specific cell-level entries are also placed on the correpsonding +\code{psCell} elements. These entries are added to the metadata with +the name 'CELL': +\begin{verbatim} +cell->metadata = psMetadataAdd (cell->metadata, PS_LIST_TAIL, "CELL", PS_META_META, "cell", cellMD); +\end{verbatim} + +\subsection{FITS File Data Representations} + +Within the FITS data representation, there are various choices which +can and have been made for the placement of the pixels in a data file. +In the simplest case, the camera consists of a single chip consisting +of a single cell always read with a single readout. In this case, the +image data could be written as part of the primary data unit. In a +more complex case with multiple chips and multiple cells, the data may +be organized in various ways. The data may be distributed into +multiple files or in multiple FITS data extensions. A single camera +image may be written as a collection of files for individual chips +with separate extensions for each cell (CFH12K.split, GPC). Another +camera may write a single file with multiple extensions for each cell +(Megacam.raw), or multiple extensions per chip, which each cell +representing portions of the chip image (Megacam.splice, CFHT-IR). + +In all of these representations, there are only two principal +distinctions in how the pixel data is stored. Either a single data +block (a single FITS image) represents only a single cell, or it +represents a collection of cells. This affects the way in which an +image is read in, the way the header is associated with a level of the +data hierarchy, and the way in which the header keywords are +interpretted to define the layout of the pixels in the cell. We +distinguish these two cases as 'cell-based' and 'chip-based' layout. + +The FITS headers are attached to the \code{psFPA} data hierarchy by +adding them to the \code{psMetadata} entries for the appropriate data +level. Each of the data levels \code{psFPA}, \code{psChip}, and +\code{psCell} have \code{metadata} elements to store relevant metadata +of any type. A FITS header metadata collection may be added to one of +these metadata collections using the metadata name 'HEADER': +\begin{verbatim} +fpa->metadata = psMetadataAdd (fpa->metadata, PS_LIST_TAIL, "HEADER", PS_META_META, "header", header); +\end{verbatim} + +In 'cell-based' layout, the headers are attached to the corresponding +\code{psCell} element. In 'chip-based' layout, the headers are +attached to the \code{psChip} element. This difference is natural +since a 'cell-based' data file has one FITS extension, and therefore +one FITS header, for each \code{cell}. Conversely, in a 'chip-based' +data file, there is one extension per chip, and therefore multiple +cells for each FITS header, but only one chip for each FITS header. + +\subsection{Determine the Camera from the Primary Header} + +\begin{verbatim} +char *pmCameraFromHeader (psFitsHeader *header, psMetadata *rules); +\end{verbatim} + +This function examines a primary header unit (\code{header}) and +determines the camera which provided the data. The rules which +identify the camera are defined as a \code{psMetadata} structure +containing one element for each known camera. These are generally to +be provided as part of the site configuration information. The +elements of this \code{psMetadata} container are in turn +\code{psMetadata} containers consisting of the expected header +keywords and their required values. The function +\code{pmCameraFromHeader} tests each of the defined cameras in +succession. For each camera, it searches for each of the given +keywords and compares the value with the value expected. If the +keyword does not exist, or if the keyword has the wrong value, the +camera is rejected. The name of first camera which matches the header +is returned as an allocated string. If no match is found, the +returned value must be \code{UNKNOWN}. This condition may be treated +by calling functions as either an error or only a warning. If an +error is encountered in parsing the metadata containers, then +\code{NULL} is returned. An example of the metadata keyword / value +pairs is given below (note that this uses an undefined metadata config +file representation \code{METADATA}): + +\begin{verbatim} +CFH12K.MEF METADATA + TELESCOP STR CFHT + INSTRUME STR CFH12K + EXTEND BOOL T + NEXTEND S32 12 +END + +CFH12K.SPLIT METADATA + TELESCOP STR CFHT + INSTRUME STR CFH12K + EXTEND BOOL F +END +\end{verbatim} + +\begin{figure} +\begin{center} +\psfig{file=CameraHierarchy,width=5.0in} +\caption{Camera Data and Metadata Hierarchy\label{CameraHierarchy}} +\end{center} +\end{figure} + +\subsection{Validate the Header set \& Construct the FPA} + +\begin{verbatim} +bool pmCameraValidateHeaders (psMetadata *headers, psMetadata *camera); +psFPA *pmFPAfromHeader (psMetadata *headers, psMetadata *camera); +\end{verbatim} + +These funtions examine the collection of \code{headers} and compare +them with the camera definitions in the metadata structure +\code{camera}. One function (\code{pmFPAfromHeader}) uses the header +set and camera definition file to construct a \code{psFPA} with all +\code{psChip} and \code{psCell} entries allocated, but without any +\code{psReadout} or pixel data. The other function validates the +header set against the camera definition, but does not actually +construct the \code{psFPA}. + +The camera definition metadata collection defines the relationship +between chips, cells, and the FITS extensions. An example of the data +needed by \code{pmCameraValidateHeaders} and \code{pmFPAfromHeader} is +given below for several types of cameras in the form of a metadata +config file. + +\begin{verbatim} +MEGACAM.RAW METADATA + NCELL S32 72 + CELL.FMT STR CELL.%02d + EXT.TYPE STR CELL + EXT.KEY STR EXTNAME + PHU STR FPA + # + # EXT.KEY CHIP + CELL.00 CELL amp00 CHIP.00 + CELL.01 CELL amp01 CHIP.00 + CELL.02 CELL amp02 CHIP.01 + CELL.03 CELL amp03 CHIP.01 + ... +END + +MEGACAM.SPLICE METADATA + NCELL S32 36 + CELL.FMT STR CELL.%02d + EXT.TYPE STR CHIP + EXT.KEY STR EXTNAME + PHU STR FPA + # + # EXT.KEY CHIP + CELL.00 CELL ccd00 CHIP.00 + CELL.01 CELL ccd00 CHIP.00 + CELL.02 CELL ccd01 CHIP.01 + CELL.03 CELL ccd01 CHIP.01 + ... +END + +CFH12K.SPLIT METADATA + NCELL S32 12 + CELL.FMT STR CELL.%02d + EXT.TYPE STR CELL + EXT.KEY STR EXTNAME + PHU STR NONE + # + # EXT.KEY CHIP + CELL.00 CELL chip00 CHIP.00 + CELL.01 CELL chip01 CHIP.01 + CELL.02 CELL chip02 CHIP.02 + ... +END + +GPC.RAW METADATA + NCELL S32 4096 + CELL.FMT STR CELL.%04d + EXT.TYPE STR CELL + EXT.KEY STR EXTNAME + PHU STR CHIP + # + # EXT.KEY CHIP + CELL.0000 CELL amp00 CHIP.00 + CELL.0001 CELL amp01 CHIP.00 + CELL.0002 CELL amp02 CHIP.00 + CELL.0003 CELL amp03 CHIP.00 + ... + CELL.0065 CELL amp03 CHIP.01 + CELL.0066 CELL amp03 CHIP.01 + ... +END +\end{verbatim} + +Among the important elements of the camera definition information are: +\begin{itemize} +\item \code{NCELL} - this defines the possible number of cells from this + camera. +\item \code{CELL.FMT} - this defines the format of the \code{CELL.nn} + keywords below in the metadata collection, one for each of the + \code{NCELL} values +\item \code{EXT.TYPE} - this defines the lowest data level + corresponding to a single FITS extension: \code{CELL} or + \code{CHIP}. In some cases, a single extension represents a chip, + and is subdivided into cells by header keywords specifying certain + regions. In other cases, a single extension represents only the + data from a single amplifier, ie, a cell. These two cases require + somewhat different handling. +\item \code{EXT.KEY} - this entry defines a header keyword which + allows the unique identification of a given header with one of the + cell or chip entries (depending on the value of \code{EXT.TYPE}). +\item \code{PHU} - this entry defines the meaning of the FITS file + primary header unit. +\end{itemize} +In addition to these generic parameters, the camera definition +metadata includes an entry for each cell giving specific values +required to define that cell. + +Both functions \code{pmCameraValidateHeaders} and +\code{pmFPAfromHeader} must start with the set of headers and attempt +to identify the corresponding cell or chip. All available cells and +chips must be identified, and the cells corresponding to each chip +must be tracked. In the case of \code{pmCameraValidateHeaders}, the +function must only verify the headers contain valid extensions, +without allocating the correpsonding \code{psFPA}, while +\code{pmFPAfromHeader} must also allocate the \code{psFPA}, the +contained \code{psChip} arrays, and the correct number of +\code{psCell} arrays for each \code{psChip}. + +These functions also add to the metadata containers for the chips and +cells, following the rules discussed above. Each \code{psCell} entry +should have the \code{CELL} metadata lines from above attached to the +\code{psCell.metadata} element as an additional metadata collection +with the name \code{CELL.LAYOUT}. The complete camera metadata +collection (including the cells), are attached to the +\code{psFPA.metadata} element with the name \code{CAMERA.LAYOUT}. The +image header data are also attached to metadata entries, at a level +which depends on the value of \code{EXT.TYPE} in the camera layout +data above. If the value is \code{CELL}, the header metadata +collections are attached to the \code{psCell.metadata} elements with +the name \code{HEADER}. If the value of \code{EXT.TYPE} is chip, the +header metadata is attached to the \code{psChip.metadata} element, +again with the name \code{HEADER}. + +An image file may also have a primary header unit which is not +associated with a data block. This metadata block must also be added +to the data heirarchy so successive operations may update the metadata +as needed. The disposition of the primary header unit is defined by +the camera layout key \code{PHU}. If this has a value of \code{NONE}, +there is no primary header unit, or it should be ignored. If the +value is \code{FPR}, the primary header data is attached to the +\code{psFPA.metadata} element with the value PHU. Alternatively, if +it has the value \code{CHIP}, then the primary header metadata is +attached to the \code{psChip.metadata} element. + +\subsection{Coordinate Transforms and Header Data} +\tbd{the algorithms for three functions in this section are not + well-defined. do not code yet}. + +Astrometric and geometric information about an image from a camera may +be represented in a variety of ways. A crude representation of the +pixel geometry is specified in many image headers using the IRAF-style +region keywords \code{DATASEC}, \code{DETSEC}, etc. These keywords +are used to define the location of a single image's pixels in the +frame of the full mosaic of detectors in the assumption that the +mosaic can be represented as a single uniform grid of pixels. An +alternative set of keywords have been used in cases where multiple +cells are saved together in a single FITS image extension. More +sophisticated astrometric representations require elements to define +projections, scaling, distortion, etc. Several versions of header +keywords have been used to represent these astrometric +transformations. In this section, we define three functions to +interpret a collection of image headers and construct the appropriate +offset and/or astrometry parameters. + +\begin{verbatim} +bool pmFPADefineOffsets (psFPA fpa); +bool pmFPADefineWCS (psFPA fpa); +bool pmFPADefineWCSfromOffsets (psFPA fpa); +\end{verbatim} + +The first function takes a \code{psFPA} structure which has been +populated with header and camera configuration metadata in the +appropriate locations as discussed above. Using the information in +the camera config metadata and the headers, the function sets the +values for the elements \code{psChip.col0,row0}, +\code{psCell.col0,row0}, \code{psReadout.col0,row0}, +\code{psReadout.colParity,rowParity}, and +\code{psReadout.colBinning,rowBinning}. This information is +determined by examining the regions defined by the following names. + +% how do we handle the CCDSUM keyword case? +\begin{verbatim} +psCell.metadata:CELL:CCDBIN1 $\rightarrow$ psReadout.colBins +psCell.metadata:CELL:CCDBIN2 $\rightarrow$ psReadout.rowBins + +DETSEC from psCell.metadata:CELL:DETSEC +CCDSEC from psCell.metadata:CELL:CCDSEC +DATASEC from psCell.metadata:CELL:DATASEC + +if (DETSEC.x0 > DETSEC.x1) then psReadout.colParity = -1 +if (DETSEC.y0 > DETSEC.y1) then psReadout.rowParity = -1 +\end{verbatim} + +The second function examines the contents of the headers of the chips +and cells and constructs the collection of astrometric coordinate +transformations.. + +The third function uses the offset information and the basic telescope +pointing information to construct a approximate guess at the +astrometric coefficients based on the detector geometry. + +\begin{figure} +\begin{center} +\psfig{file=CameraLayout,width=5.5in} +\caption{Camera Pixel Layout\label{CameraLayout}} +\end{center} +\end{figure} + +\begin{figure} +\begin{center} +\psfig{file=CameraRegionKeywords.ps,width=5.5in} +\caption{Camera Region Keyword Definitions\label{CameraRegionKeywords}} +\end{center} +\end{figure} + +\subsection{Chip \& Cell from FITS File} +\tbd{the algorithm for the function in this section is not + well-defined. do not code yet}. + +When loading data from disk, it is may be necessary to use the +information in the DATASEC and DETSEC entries to determine which part +of the image should be read. If the data is stored in a chip-based +format, then the data for each cell corresponds to only a fraction of +the pixels stored in a single image extension. We specify the +following function to perform the correct read of data from a FITS +file into the corresponding \code{psCell} entry respecting the +boundaries of the cells within chip-based images. + +\begin{verbatim} +psReadout *pmReadoutLoad (psReadout *input, psFits *f, psCell *cell, int plane); +\end{verbatim} + +\section{Phase 2} + +Phase 2 is the processing stage wherein the instrumental signatures +are removed from the detector images, in preparation for the +combination of multiple images in Phase 4. + +The Phase 2 processing modules are: +\begin{itemize} +\item Subtract bias; +\item Correct for non-linearity; +\item Flat-field; +\item Mask bad pixels; +\item Subtract the background; +\item \tbd{Mask cosmic rays;} +\item \tbd{Mask optical defects;} +\item \tbd{Measure the PSF;} +\item \tbd{Find and measure objects;} +\item \tbd{Correct astrometry; and} +\item \tbd{Get postage stamps.} +\end{itemize} + +Each of these shall be discussed in turn, below. Those modules which +are \tbd{TBD} will be deferred until they may be properly defined, +some of which requires further research to define the best algorithm. + +\subsection{Bias subtraction} +\label{sec:bias} + +The bias subtraction module provides a facility to correct detector +images for the electronic pedestal introduced by the readout +electronics. + +Given an input image and various other parameters, +\code{pmSubtractBias} shall subtract the bias from the image. +The API shall be the following: +\begin{verbatim} +psReadout *pmSubtractBias(psReadout *in, void *fitSpec, const psList *overscans, + pmOverscanAxis overscanAxis, const psStats *stat, + int nBin, pmFit fit, const psReadout *bias); +\end{verbatim} + +Two types of bias correction may optionally be performed on the input +image, \code{in}. The first is the subtraction of an overscan. +Multiple overscan regions may be specified and fit as a function of +row (or column). The second is the subtraction of a full-frame bias +image (or a ``dark'' image). + +The input image, \code{in}, shall have the bias subtracted in-place. + +The type of the overscan fit function, \code{fitSpec}, shall be +dependent upon the value of \code{fit}, which specifies the type of +fit to be employed and is described below. + +The prescan and/or overscan regions to be used are specified in +\code{overscans}, which is a linked list of subimages. In cases where +\code{overscans} is \code{NULL} and \code{overscanAxis} is not +\code{PM_OVERSCAN_NONE}, the function shall generate an error. If +\code{overscans} is non-\code{NULL} and \code{overscanAxis} is +\code{PM_OVERSCAN_NONE}, then the function shall generate a warning, +no overscan subtraction shall be performed, and the function shall +proceed to the full-frame bias subtraction. + +The \code{overscanAxis} specifies how the prescan/overscan subtraction +is to be performed. It is an enumerated type: +\begin{verbatim} +/** Overscan axis */ +typedef enum { + PM_OVERSCAN_NONE, ///< No overscan subtraction + PM_OVERSCAN_ROWS, ///< Subtract rows + PM_OVERSCAN_COLUMNS, ///< Subtract columns + PM_OVERSCAN_ALL ///< Subtract the statistic of all pixels in overscan region +} pmOverscanAxis; +\end{verbatim} + +If the \code{overscanAxis} is \code{PM_OVERSCAN_NONE}, then the +function shall not perform any overscan subtraction, but proceed to +the full-frame bias subtraction. If the \code{overscanAxis} is +\code{PM_OVERSCAN_ALL}, then all the overscan regions shall be used to +generate a single statistic (specified by \code{stat}) which shall be +subtracted from the entire image. A warning shall be generated if the +\code{overscanAxis} is \code{PM_OVERSCAN_NONE} or +\code{PM_OVERSCAN_ALL} and the \code{fit} is not \code{PM_FIT_NONE}. + +If the \code{overscanAxis} is \code{PM_OVERSCAN_ROWS} or +\code{PM_OVERSCAN_COLUMNS}, then the overscan shall be reduced to a +single vector (in the specified dimension) using the specified +statistic (\code{stat}). + +The statistic to use in combining multiple pixels in the +prescan/overscan regions is specified by \code{stat}. \code{stat} is +of type \code{psStats} instead of simply \code{psStatsOptions} so that +clipping levels may be specified, if desired. In the event that +multiple options are specified by \code{stats}, a warning shall be +generated, and the option with the highest priority shall be used, +according to the following priority order: \code{PS_STAT_SAMPLE_MEAN}, +\code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN}, +\code{PS_STAT_ROBUST_MEAN},\ \code{PS_STAT_ROBUST_MEDIAN}, +\code{PS_STAT_ROBUST_MODE}. + +If \code{nBin} is positive and less than the size of the vector, then +the vector shall subsequently be binned into bins that are a relative +size of \code{nBin} compared to the original pixels, again using the +specified statistic (\code{stat}). If \code{fit} is +\code{PM_FIT_SPLINE}, then \code{nBin} also serves as the number of +spline pieces. + +\code{fit} is an enumerated type: +\begin{verbatim} +/** Fit types */ +typedef enum { + PM_FIT_NONE, ///< No fit + PM_FIT_POLYNOMIAL, ///< Fit polynomial + PM_FIT_SPLINE ///< Fit cubic splines +} pmFit; +\end{verbatim} + +If \code{fitSpec} is \code{NULL}, or \code{fit} is \code{PM_FIT_NONE}, +then no fit shall be performed to the overscan. Otherwise, +\code{fitSpec} shall be interpreted to be a structure of the +appropriate type (\code{psPolynomial1D} for \code{PM_FIT_POLYNOMIAL}, +and \code{psSpline1D} for \code{PM_FIT_SPLINE}), and the overscan +shall (after reduction of the vector and binning) be fit using the +specified functional form. Upon return, the \code{fitSpec} shall +contain the coefficients of the overscan fit. If \code{fit} is +\code{PM_FIT_SPLINE}, then the \code{fitSpec} may be \code{NULL}, +in which case a new \code{psSpline} is allocated; in any case, the +number of spline pieces shall be set to \code{nBin}. + +If the overscan is not defined for each row/column, then the function +shall generate a warning and then interpolate using the provided +functional form if \code{fit} is not \code{PM_FIT_NONE}; otherwise, +the function shall generate an error. + +Following any binning, the vector shall be fit by the functional form +specified by \code{fit}. Then the overscan shall be subtracted from +the image, using values from the fit if \code{fit} is not +\code{PM_FIT_NONE}; otherwise using values from the overscan vector if +\code{overscanAxis} is not \code{PM_OVERSCAN_ALL}; otherwise using the +appropriate statistic applied to all the prescan/overscan pixels. + +A bias (or dark) image shall be subtracted pixel-by-pixel from the +input image if \code{bias} is non-NULL. Note that the input image, +\code{in}, and the \code{bias} image need not be the same size, but +the function shall use the offsets in the image (\code{in->x0} and +\code{in->y0}) to determine the appropriate offsets to obtain the +correct pixel on the \code{bias}. In the event that the \code{bias} +image is too small (i.e., pixels on the input image refer to pixels +outside the range of the \code{bias} image), the function shall +generate an error. Any pixels masked in the \code{bias} shall also be +masked in the output. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\subsection{Non-linearity} + +We here specify two functions to perform the non-linearity correction, +since either (or both) might be used to specify the correction. + +The first, \code{pmNonLinearityPolynomial} shall correct the input +image for non-linearity by replacing the flux in each pixel of the +input image, \code{in}, with the result of the specified polynomial, +\code{coeff}, acting on the flux. The API shall be the following: + +\begin{verbatim} +psReadout *pmNonLinearityPolynomial(psReadout *in, const psPolynomial1D *coeff); +\end{verbatim} + +The polynomial coefficients, \code{coeff}, will be supplied by the +caller, likely from the image metadata. + +The second function, \code{pmNonLinearityLookup} shall correct +the input image for non-linearity by using a lookup table. The API +shall be the following: + +\begin{verbatim} +psReadout *pmNonLinearityLookup(psReadout *in, const psVector *inFlux, const psVector *outFlux); +\end{verbatim} + +For each pixel in the input image, the function shall find the flux +value in the \code{inFlux} vector, and replace the flux with the +corresponding value in the \code{outFlux} vector. If the sizes of the +\code{inFlux} and \code{outFlux} vectors differ, then the function +shall generate a warning, and the longer vector shall be truncated to +the length of the shorter. The \code{inFlux} vector may be assumed +to be pre-sorted and not contain duplicate values. + +If the particular value of a pixel is not found in the \code{inFlux} +vector, the corresponding \code{outFlux} shall be calculated through +linear interpolation. If the value of a pixel is beyond the range of +values specified in the \code{inFlux} vector, then the function shall +generate a warning (at most one warning of this type per call, +preferably mentioning the number of pixels out of bounds), and replace +that pixel value with the \code{outFlux} value corresponding to the +minimum or the maximum, depending on whether the pixel value is below +or above the range of \code{inFlux} values. + +In the event that the \code{inFlux} vector does not contain two or +more entries (being the lower limit required for linear +interpolation), then the function shall generate a warning, and make +no correction to any of the pixels. + +Both \code{pmNonLinearityPolynomial} and \code{pmNonLinearityLookup} +shall modify the input image in-place. + +\subsection{Flat-fielding} + +Given an input image and a flat-field image, \code{pmFlatField} shall +divide the input image by the flat-field image and return it in place, +updating the mask as appropriate. The API shall be the following: +\begin{verbatim} +bool pmFlatField(psReadout *in, psReadout *mask, const psReadout *flat); +\end{verbatim} + +Note that the input image, \code{in}, and the flat-field image, +\code{flat}, need not be the same size, since the input image may +already have been trimmed (following overscan subtraction), but the +function shall use the offsets of the readout (\code{in->col0, +in->row0}) and the image subarray (\code{in->image->x0, +in->image->y0}) to determine the appropriate offsets to obtain the +correct detector pixels in the flat-field image. Note that the image +offset is relative to its parent, so this offset must be followed to +the top level image which is not a child of another image and the +offsets summed. The detector pixel coordinates of pixel \code{x,y} in +a top-level image are thus \code{x + in->image->x0 + in->col0, y + +in->image->y0 + in->row0}. In the event that the \code{flat} image is +too small (i.e., pixels on the input image refer to pixels outside the +range of the \code{flat} image), the function shall generate an error. + +Pixels which are negative or zero in the \code{flat} shall be masked +in the input image with the value \code{PM_MASK_FLAT} (see +\S\ref{sec:maskValues}). Negative pixels in the \code{flat} may be +set to zero so that they are treated identically to zeroes. Any +pixels masked in the \code{flat} shall be masked with corresponding +values in the \code{output}. + +The function shall not normalize the \code{flat}; this responsibility +is left to the caller. This function is basically equivalent to a +divide (with \code{psImageOp}), but with care for the region that is +divided, checking for negative pixels, and copying of the mask from +the \code{flat} to the output. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\subsection{Masking} + +\subsubsection{Mask values} +\label{sec:maskValues} + +We define several mask values for use in the phase 2 processing: +\begin{verbatim} +/** Mask values */ +typedef enum { + PM_MASK_TRAP = 0x0001, ///< The pixel is a charge trap + PM_MASK_BADCOL = 0x0002, ///< The pixel is a bad column + PM_MASK_SAT = 0x0004, ///< The pixel is saturated + PM_MASK_FLAT = 0x0008, ///< The pixel is non-positive in the flat-field +} pmMaskValue; +\end{verbatim} + +Of these, masks for the charge traps need to be grown by the extent of +the OT convolution kernel. For other pixel types, orthogonal transfer +of the flux in this pixel will not (necessarily) affect the flux in +neighbouring pixels. + +\subsubsection{Bad pixels} + +Given an input image, \code{in}, a bad pixel \code{mask}, a +corresponding value in the bad pixel mask to mask in the input image, +\code{maskVal}, a saturation level, and a growing radius, +\code{pmMaskBadPixels} shall mask in the input image those +pixels in the bad pixel mask that match the value to mask. The API +shall be the following: +\begin{verbatim} +psReadout *pmMaskBadPixels(psReadout *in, const psImage *mask, unsigned int maskVal, + float sat, unsigned int growVal, int grow); +\end{verbatim} + +Note that the input image, \code{in}, is modified in-place. All +pixels in the \code{mask} which satisfy the \code{maskVal} shall have +their corresponding pixels masked in the input image, \code{in}. All +pixels which satisfy the \code{growVal} shall have their corresponding +pixels, along with all pixels within the \code{grow} radius masked. +Pixels which have flux greater than \code{sat} shall also be masked, +but not grown. + +Note that the input image, \code{in}, and the \code{mask} need not be +the same size, since the input image may already have been trimmed +(following overscan subtraction), but the function shall use the +offsets in the image (\code{in->x0} and \code{in->y0}) to determine +the appropriate offsets to obtain the correct pixel on the mask. In +the event that the \code{mask} image is too small (i.e., pixels on the +input image refer to pixels outside the range of the \code{mask} +image), the function shall generate an error. + +\subsection{Subtract sky} + +Given an input image, a polynomial or spline specifying the order of a +desired fit, a binning factor and statistics to use for the binning, +along with a clipping level, \code{pmSubtractSky} shall fit and +subtract a model for the background of the image. The API shall be +the following: +\begin{verbatim} +psReadout *pmSubtractSky(psReadout *in, psPolynomial2D *poly, psImage *mask, psU8 maskVal, + int binFactor, psStats *stats, float clipSD); +\end{verbatim} + +Note that the input image, \code{in}, shall be subtracted in-place. +The function shall return the subtracted image, and also update the +polynomial, Chebyshev or spline specified by \code{fitSpec}, to hold +the coefficients used in the subtraction. + +The polynomial, \code{poly}, specifies the order of the polynomial, +and on return shall contain the coefficients of the fit. If +\code{poly} is \code{NULL}, then no fit shall be performed, and the +function shall generate a warning and return. + +When fitting the polynomial, the function shall first bin the input +image by \code{binFactor} in order to reduce the required processing +time. In the binning, pixels in the \code{mask} (if non-\code{NULL}) +which satisfy the \code{maskVal} shall be excluded. The statistic to +use in this binning is specified by \code{stat}. \code{stat} is of +type \code{psStats} instead of simply \code{psStatsOptions} so that +clipping levels may be specified, if desired. In the event that +multiple options are specified by \code{stats}, a warning shall be +generated, and the option with the highest priority shall be used, +according to the following priority order: \code{PS_STAT_SAMPLE_MEAN}, +\code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN}, +\code{PS_STAT_ROBUST_MEAN}, \code{PS_STAT_ROBUST_MEDIAN}, +\code{PS_STAT_ROBUST_MODE}. If the \code{binFactor} is non-positive, +or \code{stats} is \code{NULL} or fails to specify an option, a +warning shall be generated, and the fit shall be performed on the +entire image. + +Binned pixels deviating more than \code{clipSD} standard deviations +from the mean of the binned pixels shall be clipped in a single +clipping iteration before polynomial fitting. These pixels may be +interpolated over, or may be simply ignored in the fitting, according +to the choice of algorithm. If the \code{clipSD} is non-positive, +then the function shall generate a warning and not perform any +clipping. + +The \code{mask} shall be of type \code{psU8}, and the input image, +\code{in}, shall be of type \code{psF32}. + +\section{Calibration} + +The calibration module essentially consists of combining multiple +images of a particular type in order to build up signal-to-noise. For +this, we require a general purpose image combination module. We +forsee this module as only acting upon data from the same detector, +and so each input image will have the same noise characteristics. + +\begin{verbatim} +typedef struct { + psStats *stats; // Statistics to use in combining pixels + unsigned int maskVal, // Mask pixels where mask & maskVal == 1 + float fracHigh; // Fraction of high pixels to throw + float fracLow; // Fraction of low pixels to throw + int nKeep; // Number of pixels to be sure to keep +} pmCombineParams; + +psImage * +pmReadoutCombine(psImage *output, // Output image, or NULL + const psList *inputs, // List of input readouts + pmCombineParams *params, // Combination parameters + const psVector *zero, // Offsets to apply for each image + const psVector *scale, // Scales to apply for each image + bool applyZeroScale, // Are zero and scale for application, or only noise properties? + float gain, // Gain in e/ADU + float readnoise // Read noise in e + ); +\end{verbatim} + +\code{pmReadoutCombine} combines input images pixel by pixel --- for +each pixel of the output image, a stack of contributing input pixels +is formed and combined. Several of its input parameters are lists or +vectors, and if these are not all of the same length (or \code{NULL}), +the module shall generate an error and return \code{NULL}. + +If the provided \code{output} is \code{NULL}, then the module shall +allocate a new image of sufficient size for the input images. If the +\code{output} image is non-\code{NULL} and is not of sufficient size +for the combined image, the module shall generate an error and return +\code{NULL}. + +If the \code{inputs} is \code{NULL}, the module shall generate an +error and return \code{NULL}. Otherwise, the \code{inputs} shall be a +list of \code{psReadout}s. The images contained within the +\code{psReadout}s need not all be of the same size, but the module +shall take into account the offsets (\code{col0,row0}) from the corner +of the detector when comparing pixels, so that it is the same +\textit{physical} pixels that are combined. + +The parameters used in the combination, including how the pixels are +to be combined, and how the rejection is performed is contained within +the \code{params}, which may not be \code{NULL} (otherwise the module +shall generate an error and return \code{NULL}). We choose to use +this structure instead of supplying the values separately in order to +keep down the number of parameters to \code{pmReadoutCombine}; the +\code{psCombineParams} may be recycled for subsequent calls to +\code{pmReadoutCombine} since the values are not dependent upon the +choice of inputs, but merely specify how the combination is to be +performed. + +The particular statistic specified by \code{stats} shall be used to +combine each stack of pixels from the input images. Only one of the +statistics choices may be specified, otherwise the module shall +generate an error and return \code{NULL}. + +If the \code{maskVal} is non-zero, then pixels in the \code{mask} of +each \code{psReadout} in the \code{inputs} which satisfy the +\code{maskVal} shall not have the corresponding pixels placed in the +stack for combination. + +After masking, but before performing the combination, the highest +\code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels +in the stack are immediately rejected, unless this would leave less +than \code{nKeep} pixels in the stack, in which case no immediate +rejection is performed. + +If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale} +is \code{true}, then the appropriate values shall be added to the +\code{inputs} before rejection is performed. If \code{zero} is +non-\code{NULL} and \code{applyZeroScale} is false, then the values +shall only be used in calculating the Poisson variances. + +If the \code{scale} vector is non-\code{NULL} and +\code{applyZeroScale} is \code{true}, then the appropriate values +shall multiply the \code{inputs} before rejection is performed. If +\code{scale} is non-\code{NULL} and \code{applyZeroScale} is false, +then the values shall only be used in calculating the Poisson +variances. + +The purpose of \code{applyZeroScale} is to allow combination of fringe +frames, where the frames have been deliberately sky-subtracted and +rescaled (to get the fringes amplitudes running from -1 to 1), which +actions should not be undone when combining, but yet it is desirable +to provide the \code{zero} and \code{scale} values so that the correct +noise properties are used in the combination. + +If the \code{gain} and \code{readnoise} are positive and non-negative +(respectively), then these shall be used to provide weights for the +combination using Poisson statistics ($\sigma_i$ below). + +In summary, pixels corresponding to the same physical pixel are +combined, having values $x_i \pm \sigma_i$. In the case that +\code{applyZeroScale} is \code{true}, then: +\begin{eqnarray} +x_i & = & s_i f_i + z_i \\ +\sigma_i & = & [g x_i + r^2]^{1/2} / g +\end{eqnarray} +Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale +applied to image $i$, $z_i$ is the zero offset applied to image $i$, +$g$ is the gain, and $r$ is the read noise. If scales are not +provided, they are set to unity; if zero offsets are not provided, +they are set to zero. + +If \code{applyZeroScale} is \code{false}, then the values are: +\begin{eqnarray} +x_i & = & f_i \\ +\sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g +\end{eqnarray} +where the same symbols are used as above. + +The \code{inputs, zero} and \code{scale} may be of S16, S32 and F32 +types, and must all be of the same type. The \code{output} shall be +of the same type. + +\section{Object Detection, Measurement, and Classification Routines} + +\subsection{Overview} + +The process of finding, measuring, and classifying astronomical +sources on images is one of the critical tasks of the IPP or any +astronomical software system. In this section, we define structures +and functions related to the task of source detection and measurement. +The elements defined in this section are generally low-level +components which can be connected together to construct a complete +object measurement suite. An example pseudo-C program using these +functions is provided in Appendix~\ref{psphot}. + +We first define the collection of structures needed to carry +information about the detected sources. A major challenge is to +define what we mean by an astronomical object in the context of image +source detection. An astronomical object may be as simple as a +stellar point source, or it may consist of a galaxy which has smooth +extended structure; it may consist of an irregular galaxy or galaxy +group with substantial and complex sub-structure, or it may consist of +complex non-stellar structures such as planetary nebulae, reflection +nebulae, outflows and jets. + +The simplest objects (ie, stars) can be sufficiently modeled by the +point-source function (PSF). More complex objects (such as simple, +smooth galaxies), may have approximate analytical models which +represent their morphology with more-or-less accuracy. In the extreme +cases, the objects are not well modeled at all and must be represented +in other ways. Thus, one aspect of our data structures must be +elements to specify if an object has been represented by a model, what +the model parameters are, and how well it is represented by the model. +Another aspect of the data structures must be a representation of the +pixels associated with the object so complex structures may be +referenced without attempting to supply an analytical model. Finally, +it is often useful to allow a single complex model to be represented +as a collection of simpler contained structures which may be modelled. +Thus, the representation of an object must be capable of identifying +children, or substructures, of that object. + +Two additional aspects must be considered. First, source detection +need not be performed on a single image in isolation: it is necessary +for multiple realizations of the same source in multiple images to be +measured together (whether or not through simultaneous fitting in +multiple bands or via application of the results from one image to +another image). Second, it will be necessary to performed object +measurements on pixels in which no source is actually detected. For +example, this is a convenient way to provide flux upper limits at the +locations of known objects. + +\subsection{Structures to Describe Sources} + +We start by defining a single source detected in a single band: +\begin{verbatim} +typedef struct { + psPeak *peak; // description of peak pixel + psImage *pixels; // rectangular region including object pixels + psImage *mask; // mask to mark pixels associated with object in region + psMoments *moments; // basic moments measure for the object + psModel *models; // model parameters and type + psSourceType type; // best identification of object +} psSource; +\end{verbatim} + +This source has the capacity for several types of measurements. The +simplest measurement of a source is the location and flux of the peak +pixel associated with the source: +\begin{verbatim} +typedef struct { + int x; // x-coordinate of peak pixel + int y; // y-coordinate of peak pixel + float counts; // value of peak pixel (above sky?) + psPeakType class; // description of peak +} psPeak; +\end{verbatim} + +A peak pixel may have several features which may be determined when +the peak is found or measured. These are specified by the +\code{psPeakType} enum. \code{PS_PEAK_LONE} represents a single pixel +which is higher than its 8 immediate neighbors. The +\code{PS_PEAK_EDGE} represents a peak pixel which touching the image +edge. The \code{PS_PEAK_FLAT} represents a peak pixel which has more +than a specific number of neighbors at the same value, within some +tolarence: +\begin{verbatim} +typedef enum { + PS_PEAK_LONE; // isolated peak + PS_PEAK_EDGE; // peak on edge + PS_PEAK_FLAT; // peak has equal-value neighbors +} psPeakType; +\end{verbatim} + +The pixels which contain the source may be specified with the +\code{psImage *pixels} element, and the mask image may be used to +exclude any pixels which are not considered part of the source. Note +that the source image may be simply a subimage of the main image or a +separate copy if the pixels are modified (eg, by subtracting flux from +other sources, etc). + +One of the simplest measurements which can be made quickly for an +object are the object moments. We specify a structure to carry the +moment information for a specific source: + +\begin{verbatim} +typedef struct { + float x; // x-coord of centroid + float y; // y-coord of centroid + float Sx; // x-second moment + float Sy; // y-second moment + float Sxy; // xy cross moment + float Sum; // pixel sum above sky (background) + float Peak; // peak counts above sky + float Sky; // sky level (background) + int nPixels; // number of pixels used +} psMoments; +\end{verbatim} + +An object's flux distribution may be modelled with some analytical +function. The description of the model includes the model parameters +and their errors, along with the fit $\chi^2$. The model type is +specified by the enum \code{psObjectModel}, specified below. We +discuss the details of these models in section~\ref{ObjectModels}. + +\begin{verbatim} +typedef struct { + psObjectModel type; // model to be used + int Nparams; // number of parameters + float *params; // parameter values + float *dparams; // parameter errors + float chisq; // fit chisq +} psModel; +\end{verbatim} + +\begin{verbatim} +typedef enum { + PS_MODEL_GAUSS; + PS_MODEL_PGAUSS; + PS_MODEL_TWIST_GAUSS; + PS_MODEL_WAUSS; + PS_MODEL_SERSIC; + PS_MODEL_SERSIC_CORE; +} psModelType; +\end{verbatim} + +\subsection{Basic Object Detection APIs} + +In this section, we specify a collection of basic functions which +operate on images and sources. We define them roughly in order in +which we expect to use them in a basic object detection process. + +\begin{verbatim} +psVector *pmFindVectorPeaks(const psVector vector, float threshold); +\end{verbatim} + +Find all local peaks in the given vector above the given threshold. A +peak is defined as any element with a value greater than its two +neighbors and with a value above the threshold. Two types of special +cases must be addressed. Equal value elements: If an element has the +same value as the following element, it is not considered a peak. If +an element has the same value as the preceeding element (but not the +following), then it is considered a peak. Note that this rule +(arbitrarily) identifies flat regions by their trailing edge. Edge +cases: At start of the vector, the element must be higher than its +neighbor. At the end of the vector, the element must be higher or +equal to its neighbor. These two rules again places the peak +associated with a flat region which touches the image edge at the +image edge. The result of this function is a vector containing the +coordinates (element number) of the detected peaks (type +\code{psU32}). + +\begin{verbatim} +psList *pmFindImagePeaks(const psImage *image, float threshold); +\end{verbatim} + +Find all local peaks in the given image above the given threshold. +This function should find all row peaks using +\code{pmFindVectorPeaks}, then test each row peak and exclude peaks +which are not local peaks. A peak is a local peak if it has a higher value +than all 8 neighbors. If the peak has the same value as its +y +neighbor or +x neighbor, it is NOT a local peak. If any other +neighbors have an equal value, the peak is considered a valid peak. +Note two points: first, the +x neighbor condition is already enforced +by \code{pmFindVectorPeaks}. Second, these rules have the effect of +making flat-topped regions have single peaks at the (+x,+y) corner. +When selecting the peaks, their type must also be set. The result of +this function is a list of \code{psPeak} entries. + +\tbd{do we need a function psVector *psImageRowVector (psImage *image, int row);} + +\begin{verbatim} +psList *pmCullPeaks(psList *peaks, float maxvalue, const psRegion *valid); +\end{verbatim} + +Remove certain types of peaks from a list of peaks based on the given +criteria. Peaks should be eliminated if they have a peak value above +the given maximum value limit or if the fall outside the valid +region. The result of the function is to reduce the number of peaks +in the input peaks list, and return the resulting list. + +\begin{verbatim} +psSource *pmSourceLocalSky(const psImage *image, const psPeak *peak, float inner_radius, float outer_radius), +\end{verbatim} + +Measure the local sky in the vicinity of the given \code{peak}. The +image pixels in the square annulus with inner and outer radii as +specified are used to measure the median \tbd{(clipped mean?) +(statistic from psStats?)} in the vicinity of the the specified peak +coordinates. The resulting sky is applied to the \code{psMoments} +element of the allocated \code{psSource} structure. The input +\code{peak} is also added to the \code{psSource} element, which is +returned. + +\begin{verbatim} +psSource *pmSourceMoments(psSource *source, const psImage *image, float radius), +\end{verbatim} + +Measure source moments for the given \code{source}, using the value of +\code{source.moments.sky} provided as the local background value. The +resulting moment values are applied to the \code{source.moments} +entry, and the source is returned. The moments are measured within +the given radius of the \code{source.peak} coordinates. + +\begin{verbatim} +psSource *pmSourceRoughClass(psSource *source, float saturate, float SNlim, const psRegion *valid); +\end{verbatim} + +Based on the specified data values, make a guess at the source +classification. + +\subsection{Object Fitting} + +We need a way to fit a particular functional model to an object. +PSLib includes the \code{psMinimizeLMChi2} and \code{psMinimizePowell} +functions, which form the core of this processes. However, additional +support functions and wrapping functions are necessary for the +specific case of source fitting. The operations can be broken down +into discrete steps: + +\begin{enumerate} +\item Identify the pixels of interest + +\item Make a guess at the model parameters. For some models, the +parameters may be guessed based on only the moments. For others, +additional measurements must be made. + +\item Construct the input vectors from the pixels of interest. + +\item Apply fitting function \code{psMinimizeLMChi2()} + +\item Construct model image. + +\item Subtract model from image. +\end{enumerate} + +\begin{verbatim} +bool pmSourceSetPixelsCircle(psSource *source, const psImage *image, float radius); +\end{verbatim} + +Define pixels associated with a source based on a circular aperture. +This operation creates the \code{source.pixels} and \code{source.mask} +entries for the source based on a circular aperture centered on the +source centroid (or peak?). The \code{source.pixels} is a subimage of +the input image. The function returns \code{TRUE} on success or +\code{FALSE} on failure. + +\begin{verbatim} +bool pmSourceModelGuess(psSource *source, const psImage *image); +\end{verbatim} + +Convert available data to an initial guess for the given model. The +method of defining the model parameter guesses are specified for each +model below. The guess values are placed in the model parameters. The +function returns \code{TRUE} on success or \code{FALSE} on failure. + +\begin{verbatim} +psArray *pmSourceContour(const psSource *source, const psImage *image, float level, int mode); +\end{verbatim} + +Find points in a contour for the given source at the given level. If +mode is \code{PS_CONTOUR_CRUDE}, the contour is found by starting at +the source peak, running along each pixel row until the level is +crossed, then interpolating to the level coordinate for that row. +This is done for each row, with the starting point determined by the +midpoint of the previous row, until the starting point has a value +below the contour level. The resulting contour consists of two +vectors giving the x and y coordinates of the contour levels. This +function may be used as part of the model guess inputs. + +\tbd{Other modes may be specified in the future for more refined contours} + +\begin{verbatim} +bool pmSourceFitModel(psSource *source, psImage *image); +\end{verbatim} + +Fit the requested model to the specified source. The starting guess +for the model is given by the input \code{source.model} parameter +values. The pixels of interest are specified by the +\code{source.pixels} and \code{source.mask} entries. This function +calls \code{psMinimizeLMChi2()} on the image data. The function +returns \code{TRUE} on success or \code{FALSE} on failure. + +\begin{verbatim} +bool pmSourceAddModel(psImage *image, psSource *source, bool center); +\end{verbatim} + +Add the given source model flux to the provided image. The boolean +option center selects if the source is recentered to the image center +or if it is placed at its centroid location. The pixel range in the +target image is at most the pixel range specified by the +\code{source.pixels} image. The success status is returned. + +\begin{verbatim} +bool pmSourceSubModel(psSource *source); +\end{verbatim} + +Subtract the model from its image pixels given by +\code{source.pixels}. The success status is returned. + +\subsection{Basic Object Models} +\label{ObjectModels} + +We specify a variety of basic object models which are required. +Details of the model functional forms, parameters, and the derivatives +are specified in the ADD. + +\subsubsection{Real 2D Gaussian} + +\begin{verbatim} +float psMinLM_Gauss2D(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +This function is a two-dimensional Gaussian with an elliptical +cross-section and a constant local background. + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\subsubsection{Pseudo-Gaussian} + +\begin{verbatim} +float psMinLM_PseudoGauss2D(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +This function is a polynomial approximation of a 2D Gaussian otherwise +very similar to the real Gaussian. It is used in place of a real +Gaussian for speed. + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\subsubsection{Waussian} + +\begin{verbatim} +float psMinLM_Wauss2D(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +The Waussian is a modified polynomial approximation of a 2D Gaussian, +with non-linear polynomial terms having variable coefficients, rather +than the Taylor series values of 1/2 and 1/6. + +\subsubsection{Twisted Gaussian} + +\begin{verbatim} +float psMinLM_TwistGauss2D(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +This function describes an object with power-law wings and a flattened +core, where the core has a different contour from the wings. + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\tbd{future galaxy models to be implemented} + +\subsubsection{Sersic Galaxy Model} + +\begin{verbatim} +float psMinLM_Sersic(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +\subsubsection{Sersic with Core Galaxy Model} + +\begin{verbatim} +float psMinLM_SersicCore(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +\subsubsection{Pseudo Sersic Galaxy Model} + +\begin{verbatim} +float psMinLM_PseudoSersic(psVector *deriv, psVector *params, psVector *x); +\end{verbatim} + +\appendix + +\section{Pseudo-C PSPhot} +\label{psphot} + +\begin{verbatim} +# include +# include + +main () { + + psMetadata *header; + psImage *image; + + fd = psFitsOpen (argv[1]); + md = psFitsReadHeader (fd); + image = psFitsReadImage (fd, md); + + stats = psImageStats (NULL, image); + + RDNOISE = psMetadataLookup (md, "RDNOISE"); + GAIN = psMetadataLookup (md, "GAIN"); + INNER = psMetadataLookup (config, "INNER_RADIUS"); + OUTER = psMetadataLookup (config, "OUTER_RADIUS"); + SATURATE = psMetadataLookup (config, "SATURATE"); + NSIGMA = psMetadataLookup (config, "PSF_PEAK_THRESHOLD"); + RADIUS = psMetadataLookup (config, "PSF_MOMENTS_RADIUS"); + XBORDER = psMetadataLookup (config, "XBORDER"); + YBORDER = psMetadataLookup (config, "YBORDER"); + + keep = psRegionAlloc (XBORDER, image->nCol - YBORDER, + YBORDER, image->nRow - YBORDER); + + Sky = stats->median; + Sig = sqrt(Sky/GAIN + SQ(RDNOISE)); + + kernel = psKernelParts (); + smooth = psImageConvolve (NULL, image, kernel, PS_PARTS); + + peaks = pmFindImagePeaks (smooth, NSIGMA*Sig + Sky); + + peaks = pmCullImagePeaks (peaks, SATURATE, keep); + + sources = pmSourceLocalSky (image, peaks, INNER, OUTER); + + sources = pmSourceMoments (image, sources, RADIUS); + + sources = pmSourceRoughClassify (sources, SATURATE, MIN_SN_LIM, keep); + + stars = pmSourceSelectBrightStars (sources); + + stars = pmSourceFitModel ( +} + + + +psArray *pmFindImagePeaks (psImage *image, float threshold) { + + psVector *row; + + row = psVectorAlloc (image[0].Ncol); + + /* find peaks in each row */ + for (i = 0; i < image[0].Nrow; i++) { + rowpeaks = pmFindVectorPeaks (row, threshold); + peaks.x = rowpeaks; + peaks.y = i; + peaks.z = image (i, x); + } + + /* drop non-local peaks (peaks with neighbors) */ + for (n = 0; n < peaks.n; n++) { + if (!local_peak) { + drop_peak; + } + } + return (peaks); +} +\end{verbatim} + +\section{Revision Change Log} +\input{ChangeLogSDRS.tex} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\bibliographystyle{plain} +\bibliography{panstarrs} + +\end{document} + +\subsection{Cosmic rays} + +Given an input image, a choice of a particular algorithm with +corresponding parameters, \code{psPhase2MaskCRs} shall mask cosmic +rays on the input image on the basis of their morphology. The API shall be the following: +\begin{verbatim} +/** Masks Cosmic Rays on the input image on the basis of morphology. */ +psReadout *psPhase2MaskCRs(psReadout *in, ///< Input image to be masked, and output + int algorithm, ///< Algorithm number to use + const void *params ///< Parameters for algorithm + ); +\end{verbatim} + +This is one case in which the best choice of algorithm is not known, +and may change in the future. For this reason, we specify the inputs +as an integer to specify the choice of algorithm in addition to a +pointer to some parameters which will be decoded on the basis of the +choice of algorithm. + +Note that the input image is modified in-place. + +\subsection{psPhase2MaskOpticalDefects} + +Given an input image, a list of nearby stars, and a growing radius, +\code{psPhase2MaskOpticalDefects} shall mask optical defects on the +image. The API shall be the following: +\begin{verbatim} +/** Masks optical defects in the input image */ +psChip *psPhase2MaskOpticalDefects(psChip *in, ///< Image to be masked (with astrometry), and output + const psDlist *catalog, ///< Catalog stars nearby: a list of psObjects + int grow ///< Number of pixels to grow + ); +\end{verbatim} + +\tbd{It's not clear to me how this is accomplished apart from an +optical model of the camera. Put this one on the backburner?} + + +\section{Objects} + +To identify and measure objects, we must measure the PSF, and then +convolve the image by this PSF and apply a threshold. + +\subsection{Measure the PSF} + +Given an input image, a choice of algorithm with corresponding +parameters, \code{psPhase2MeasurePSF} shall return the PSF for the +image. The API shall be the following: +\begin{verbatim} +/** Measures the PSF on the input image. Returns the PSF */ +psImage *psPhase2MeasurePSF(const psReadout *in, ///< Input image for which to measure the PSF + int algorithm, ///< Algorithm number to use + const void *params ///< Parameters for algorithm + ); +\end{verbatim} + +This is another case where the algorithm is not currently clear, and +may change in the future. For this reason, we specify the inputs to +be a choice of algorithm, and a pointer to some parameters, which are +interpreted on the basis of the algorithm choice. + +\subsection{Find and measure objects} + +Given an input image, the PSF of that image, and a list of flux levels +at which to threshold, \code{psPhase2GetObjects} shall return a +readout with the \code{objects} member set to a list of objects. The +API shall be the following: +\begin{verbatim} +/** Find and measure objects on the input image. Fills in the "objects" member of the psReadout. */ +/** THIS NEEDS WORK. */ +psReadout *psPhase2FindObjects(psReadout *in, ///< Input image on which to find objects, and output + const psImage *psf, ///< PSF to use to find objects + const psVector *levels ///< Threshold levels (std dev.s) + ); +\end{verbatim} + +Note that the input image shall be modified in-place, only insofar as +the \code{objects} member shall be set to the list of objects on that +image. + +\tbd{This needs a lot more work.} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\section{Astrometry} + +Given a chip, the elements of which have objects found and measured on +them, a list of catalog stars which lie on (or near) the chip, and +clipping parameters, \code{psPhase2Astrometry} shall fit an +astrometric solution. The API shall be the following: +\begin{verbatim} +/** Corrects astrometry on the input chip */ +psChip *psPhase2Astrometry(psChip *in, ///< Input chip for which to do astrometry, and output + const psDlist *catalog, ///< Catalog stars on the chip: a list of psObjects + int nClips, ///< Number of clipping iterations + float clipLevel ///< Level at which to clip + ); +\end{verbatim} + +Note that the input chip shall be modified in-place, only insofar as +the appropriate astrometry members shall be updated to correspond to +the fit solution (specifically, \code{in->cells[i]->cellToSky}, for +each \code{i}). + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\section{Postage stamps} + +Postage stamps consist of subimages of specific objects of interest, +which are saved for more careful analysis of the pixels. + +\subsection{Specification} + +The postage stamps are specified by a center and size, in celestial +coordinates. We define a \code{psPostageStampSpec} to specify the +parameters for a postage stamp: +\begin{verbatim} +/** Specification of a postage stamp: location on the sky */ +typedef struct { + psSphereCoord *center; ///< Centre of postage stamp + psSphereCoord *size; ///< Size of postage stamp +} psPostageStampSpec; +\end{verbatim} + +The \code{center} shall be specified in ICRS coordinates, which is the +\PS{} standard system. The \code{size} shall be specified in +arcseconds on the sky. + +\subsection{Extracting postage stamps} + +Given an input chip, and a linked-list of regions on or near the input +chip, \code{psPhase2PostageStamps} shall output an array of subimages, +containing each of the regions. The API shall be the following: +\begin{verbatim} +/** Return postage stamps of a set of regions */ +psImageArray *psPhase2PostageStamps(const psChip *in, ///< Chip from which to form postage stamps + const psDlist *regions ///< Regions to postage-stampise: a list of + ///< psPostageStampSpec + ); +\end{verbatim} + +\code{regions} shall be a linked list of \code{psPostageStampSpec}s, +not all of which may correspond to legal positions on the input chip, +\code{in}. +