Changeset 4982

Timestamp:

Sep 9, 2005, 8:30:32 AM (21 years ago)

Author:

eugene

Message:

added updates based on psphot work

Location:

trunk/doc

Files:

• modules/ModulesSDRS.tex (modified) (10 diffs)
• pslib/ChangeLogSDRS.tex (modified) (2 diffs)
• pslib/psLibSDRS.tex (modified) (17 diffs)

trunk/doc/modules/ModulesSDRS.tex

@@ -1 @@
-%%% $Id: ModulesSDRS.tex,v 1.54 2005-08-24 21:27:10 price Exp $
+%%% $Id: ModulesSDRS.tex,v 1.55 2005-09-09 18:30:32 eugene Exp $
 \documentclass[panstarrs]{panstarrs}
 
@@ -1434 +1434 @@
 \subsection{Structures to Describe Sources}
 
+\subsubsection{pmSource and pmPeak}
+
 We start by defining a single source detected in a single band:
 \begin{datatype}
@@ -1447 +1449 @@
 \end{datatype}
 
-This source has the capacity for several types of measurements.  The
+In the object analysis process, we will use specific mask values to
+mark the image pixels.  The following structure defines the relevant
+mask values.
+\begin{datatype}
+enum {
+    PSPHOT_MASK_CLEAR     = 0x00,
+    PSPHOT_MASK_INVALID   = 0x01,
+    PSPHOT_MASK_SATURATED = 0x02,
+    PSPHOT_MASK_MARKED    = 0x08,
+} psphotMaskValues;
+\end{datatype}
+
+A source has the capacity for several types of measurements.  The
 simplest measurement of a source is the location and flux of the peak
 pixel associated with the source:
@@ -1475 +1489 @@
 \end{datatype}
 
+\subsubsection{pmMoments and source description}
+
 The pixels which contain the source may be specified with the
 \code{psImage *pixels} element, and the mask image may be used to
@@ -1496 +1512 @@
   float Peak;               // peak counts above sky
   float Sky;                // sky level (background)
+  float SN;                 // approx signal-to-noise
   int   nPixels;            // number of pixels used
 } pmMoments;
 \end{datatype}
+
+A collection of object moment measurements can be used to determine
+approximate object classes.  The key to this analysis is the location
+and statistics (in the second-moment plane, $\sigma_x$ vs $\sigma_y$)
+of the group of objects which are likely PSF objects.  We define the
+following structure to identify the location and size of the psf clump
+in the second-moment plane.
+\begin{datatype}
+typedef struct {
+    float X;
+    float dX;
+    float Y;
+    float dY;
+} pmPSFClump;
+\end{datatype}
+
+A given source may be identified as most-likely to be one of several
+source types.  The \code{pmSource} entry \code{pmSourceType} defines
+the current best-guess for this source.  \tbd{The values given below
+are currently illustrative and will require some modification as the
+source classification code is developed.}
+
+\begin{datatype}
+typedef enum {
+    PM_SOURCE_DEFECT,              // a cosmic-ray
+    PM_SOURCE_SATURATED,           // random saturated pixels
+
+    PM_SOURCE_SATSTAR,             // a saturated star
+    PM_SOURCE_PSFSTAR,             // a PSF star
+    PM_SOURCE_GOODSTAR,            // a good-quality star
+
+    PM_SOURCE_POOR_FIT_PSF,        // poor quality PSF fit
+    PM_SOURCE_FAIL_FIT_PSF,        // failed to get a good PSF fit
+    PM_SOURCE_FAINTSTAR,           // below S/N cutoff
+
+    PM_SOURCE_GALAXY,              // an extended object (galaxy)
+    PM_SOURCE_FAINT_GALAXY,        // a galaxy below S/N cutoff
+    PM_SOURCE_DROP_GALAXY,         // ?
+    PM_SOURCE_FAIL_FIT_GAL,        // failed on the galaxy fit
+    PM_SOURCE_POOR_FIT_GAL,        // poor quality galaxy fit
+
+    PM_SOURCE_OTHER,               // unidentified
+} pmSourceType; 
+\end{datatype}
+
+\subsubsection{pmModel Source Model and Abstraction} 
 
 An object's flux distribution may be modelled with some analytical
@@ -1521 +1584 @@
 parameters are specified for the object by the PSF, not by the fit.
 The FLT model represents the best fit of the given model to the
-object, with all parameters floating in the fit.
+object, with all parameters floating in the fit.  
 
 \begin{datatype}
 typedef struct {
-  psS32 type;               // model to be used
+  pmModelType type;         // model to be used
   psVector *params;         // parameter values
   psVector *dparams;        // parameter errors
@@ -1531 +1594 @@
   psS32 nDOF;               // number of degrees of freedom
   psS32 nIter;              // number of iterations
+  float radius;             // fit radius actually used
 } pmModel;
 \end{datatype}
 
-A given source may be identified as most-likely to be one of several
-source types.  The \code{pmSource} entry \code{pmSourceType} defines
-the current best-guess for this source.  \tbd{The values given below
-are currently illustrative and will require some modification as the
-source classification code is developed.}
+Every model instance belongs to a class of models, defined by the
+value of the \code{pmModelType type} entry.  Various functions need
+access to information about each of the models.  Some of this
+information varies from model to model, and may depend on the current
+parameter values or other data quantities.  In order to keep the code
+from requiring the information about each model to be coded into the
+low-level fitting routines, we define a collection of functions which
+allow us to abstract this type of model-dependent information.  These
+generic functions take the model type and return the corresponding
+function pointer for the specified model.  Each
+model is defined by creating this collection of specific functions,
+and placing them in a single file for each model.  We define the
+following structure to carry the collection of information about the
+models. 
 
 \begin{datatype}
-typedef enum {
-  PM_SOURCE_PSFSTAR;
-  PM_SOURCE_GALAXY;
-  PM_SOURCE_DEFECT;
-  PM_SOURCE_SATURATED;
-  PM_SOURCE_SATSTAR;
-  PM_SOURCE_FAINTSTAR;
-  PM_SOURCE_BRIGHTSTAR;
-  PM_SOURCE_OTHER;
-} pmSourceType; 
+typedef struct {
+    char *name;
+    int nParams;
+    pmModelFunc          modelFunc;
+    pmModelFlux          modelFlux;
+    pmModelRadius        modelRadius;
+    pmModelLimits        modelLimits;
+    pmModelGuessFunc     modelGuessFunc;
+    pmModelFromPSFFunc   modelFromPSFFunc;
+    pmModelFitStatusFunc modelFitStatusFunc;
+} pmModelGroup;
 \end{datatype}
+
+Each entry in the \code{pmModelGroup} defines the information needed
+by the system to specify a model.  The function types define above are
+\begin{prototype}
+typedef psMinimizeLMChi2Func pmModelFunc;
+typedef psF64 (*pmModelFlux)(const psVector *params);
+typedef psF64 (*pmModelRadius)(const psVector *params, double flux);
+typedef bool (*pmModelLimits)(psVector **beta_lim, psVector **params_min, psVector **params_max);
+typedef bool (*pmModelGuessFunc)(pmModel *model, pmSource *source);
+typedef bool (*pmModelFromPSFFunc)(pmModel *modelPSF, pmModel *modelFLT, pmPSF *psf);
+typedef bool (*pmModelFitStatusFunc)(pmModel *model);
+\end{prototype}
+
+Each of these functions is found for a given model by calling the
+corresponding lookup function:
+\begin{prototype}
+pmModelFunc          pmModelFunc_GetFunction (pmModelType type);
+pmModelFlux          pmModelFlux_GetFunction (pmModelType type);
+pmModelRadius        pmModelRadius_GetFunction (pmModelType type);
+pmModelLimits        pmModelLimits_GetFunction (pmModelType type);
+pmModelGuessFunc     pmModelGuessFunc_GetFunction (pmModelType type);
+pmModelFromPSFFunc   pmModelFromPSFFunc_GetFunction (pmModelType type);
+pmModelFitStatusFunc pmModelFitStatusFunc_GetFunction (pmModelType type);
+\end{prototype}
+
+\code{pmModelFunc} is the function used to determine the value of the
+mdoel at a specific coordinate, and is the one used by
+\code{psMinimizeLMChi2}.  
+
+\code{pmModelFlux} returns the total integrated flux for the given
+input parameters.
+
+\code{pmModelRadius} returns the scaling radius at which the flux of
+the model matches the specified flux.  This presumes that the model is
+a function of an elliptical contour.  
+
+\code{pmModelLimits} sets the parameter limit vectors for the
+function.
+
+\code{pmModelGuessFunc} generates an initial guess for the model based
+on the provided source statistics (moments and pixel values as
+needed).
+
+\code{pmModelFromPSFFunc} takes as input a representation of the psf
+and a value for the model and fills in the PSF parameters of the
+model.  The input primarily relies upon the centroid coordinates of
+the input model, thought the normalization may potentially be used.
+
+\code{pmModelFitStatusFunc} returns a true or false values based on
+the success or failure of a model fit.  the success is determined by
+quantities such as the chisq or the signal-to-noise.
+
+In addition, the following functions are useful for interacting with
+the collection of models:
+\begin{prototype}
+int                  pmModelParameterCount (pmModelType type);
+\end{prototype}
+This function returns the number of parameters used by the listed
+function.
+
+\begin{prototype}
+char                *pmModelGetType (pmModelType type);
+pmModelType          pmModelSetType (char *name);
+\end{prototype}
+These two functions provide translations between the user-space model
+names and the internal model type codes.  The model type codes are not
+necessarily maintained between compilations of the program; the name
+should be used to transfer models between programs or systems.
+
+\subsubsection{pmPSF, pmPSFtry, and PSF model} 
 
 It is useful to generate a model to define the point-spread-function
@@ -1572 +1716 @@
 \end{datatype}
 
+\begin{prototype}
+pmModel     *pmModelFromPSF (pmModel *model, pmPSF *psf);
+\end{prototype}
+This function constructs a \code{pmModel} instance based on the
+\code{pmPSF} description of the PSF.  The input is a \code{pmModel}
+with at least the values of the centroid coordinates (possibly
+normalization if this is needed) defined.  The values of the
+PSF-dependent parameters are specified for the specific realization
+based on the coordinates of the object.  
+
+\begin{prototype}
+bool         pmPSFFromModels (pmPSF *psf, psArray *models, psVector *mask);
+\end{prototype}
+This function takes a collection of \code{pmModel} fitted models from
+across a single image and builds a \code{pmPSF} representation of the
+PSF.  The input array of model fits may consist of entries to be
+ignored (noted by a non-zero \code{mask} entry).  The analysis of the
+models fits a 2D polynomial for each parameter to the collection of
+model parameters as a function of position (and normalization?).  In
+this process, some of the input models may be marked as outliers and
+excluded from the fit.  These elements will be marked with a specific
+mask value (1 == \code{PSFTRY_MASK_OUTLIER}).  
+
+We have the capability to test several different model functions in an
+attempt to build an accurate PSF for an image.  The complete set of
+data needed to build and test as specific PSF model is carried by the
+\code{pmPSFtry} structure:
+\begin{datatype}
+typedef struct {
+    pmModelType modelType;
+    pmPSF      *psf;
+    psArray    *sources;      // pointers to the original sources
+    psArray    *modelFLT;     // model fits, floating parameters 
+    psArray    *modelPSF;     // model fits, PSF parameters
+    psVector   *mask;
+    psVector   *metric;
+    psVector   *fitMag;
+    float       ApResid;
+    float       dApResid;
+    float       skyBias;
+} pmPSFtry;
+\end{datatype}
+This structure contains a pointer to the collection of \code{sources}
+which will be used to test the PSF model form.  It lists the
+\code{pmModelType type} of model being tests, and contains an element
+to store the resulting \code{psf} representation.  In addition, this
+structure carries the complete collection of FLT (floating parameter)
+and PSF (fixed parameter) model fits to each of the sources
+\code{modelFLT} and \code{modelPSF}.  It also contains a mask which is
+set by the model fitting and psf fitting steps.  For each model, the
+value of the quality metric is stored in the vector \code{metric} and
+the fitted instrumental magntiude is stored in \code{fitMag}.  The
+quality metric for the PSF model is the aperture magnitude minus the
+fitted magnitude for each source.  This collection of aperture
+residuals is examined in the analysis process, and a linear trend of
+the residual with the inverse object flux (ie, $10^{0.4*mag)$) is
+fitted.  The result of this fit is a measured sky bias (systematic
+error in the sky measured by the fits), an effective
+infinite-magnitude aperture correction (\code{ApResid}), and the
+scatter of the aperture correction for the ensemble of PSF stars
+(\code{dApResid}).  The ultimate metric to intercompare multiple types
+of PSF models is the value of the aperture correction scatter.
+
+The following functions are used to try out a single PSF model.
+\begin{prototype}
+pmPSFtry *pmPSFtryModel (psArray *sources, char *modelName, float RADIUS);
+\end{prototype}
+This function takes the input collection of sources and performs a
+complete analysis to determine a PSF model of the given type
+(specified by model name).  The result is a \code{pmPSFtry} with the
+results of the analysis.
+
+\begin{prototype}
+bool pmPSFtryMetric (pmPSFtry *try, float RADIUS);
+\end{prototype}
+This function is used to measure the PSF model metric for the set of
+results contained in the \code{pmPSFtry} structure.
+
+The following datatype defines the masks used by the \code{pmPSFtry}
+anslysis to identify sources which should or should not be included in
+the analysis.
+\begin{datatype}
+enum {
+    PSFTRY_MASK_CLEAR    = 0x00,
+    PSFTRY_MASK_OUTLIER  = 0x01, // 1: outlier in psf polynomial fit (provided by psPolynomials)
+    PSFTRY_MASK_FLT_FAIL = 0x02, // 2: flt model failed to converge 
+    PSFTRY_MASK_PSF_FAIL = 0x04, // 3: psf model failed to converge 
+    PSFTRY_MASK_BAD_PHOT = 0x08, // 4: invalid source photometry           
+    PSFTRY_MASK_ALL      = 0x0f,
+} pmPSFtryMaskValues;
+\end{datatype}
+
 \begin{datatype}
 typedef enum {
@@ -1587 +1823 @@
 
 \subsection{Object Model Abstraction}
+
+\tbd{this section is duplicating the section above}
 
 The object model functions are defined to allow for the flexible
@@ -1731 +1969 @@
 circular radius of the \code{source.peak} coordinates.  The return
 value indicates the success (TRUE) of the operation.
+
+\tbd{add pmSourcePSFClump}
+
+\tbd{fix pmSourceRoughClass}
+
+\tbd{add pmSourcePhotometry}
+
+\tbd{pmSourceDophotType}
+
+\tbd{pmSourceSextractType}
+
+\tbd{pmModelFitStatus}
 
 \begin{prototype}

trunk/doc/pslib/ChangeLogSDRS.tex

@@ -1 @@
-%%% $Id: ChangeLogSDRS.tex,v 1.169 2005-09-02 22:29:08 price Exp $
+%%% $Id: ChangeLogSDRS.tex,v 1.170 2005-09-09 18:30:23 eugene Exp $
 
 \subsection{Changes from version 00 to version 01}
@@ -779 +779 @@
   and an optional error vector.
 \item Changed \code{complex double} to \code{complex} throughout.
-\end{itemize}
-
+
+\item Added \code{psImageCountPixelMask}
+\item Added \code{psVectorCountPixelMask}
+\item Added \code{psVectorCreate}
+\item Added \code{psImageRow}
+\item Added \code{psImageColumn}
+\item moved \code{paramMask} to \code{psMinimization}
+\item added \code{paramMin} to \code{psMinimization}
+\item added \code{paramMax} to \code{psMinimization}
+\item added \code{paramDelta} to \code{psMinimization}
+\item adjusted \code{psMinimizeLMChi2} to drop \code{paramMask}
+
+\end{itemize}
+

trunk/doc/pslib/psLibSDRS.tex

@@ -1 @@
-%%% $Id: psLibSDRS.tex,v 1.333 2005-09-02 22:29:08 price Exp $
+%%% $Id: psLibSDRS.tex,v 1.334 2005-09-09 18:30:23 eugene Exp $
 \documentclass[panstarrs,spec]{panstarrs}
 
@@ -3405 +3405 @@
 
 \begin{prototype}
-bool psVectorInit(psVector *image, ...);
+bool psVectorInit(psVector *vector, ...);
 \end{prototype}
 
@@ -3423 +3423 @@
 value of the \code{input} vector at the specified \code{position}.  A
 negative \code{position} means index from the end.
+
+\begin{prototype}
+psVector psVectorCreate(psVector *input, double lower, double upper, double delta, psElemType type);
+\end{prototype}
+
+This function creates a new vector, or reallocates the provided vector
+if \code{input} is not \code{NULL}.  The created vector consists of
+the data range starting at \code{lower}, running to \code{upper}, in
+steps of \code{delta}.  The upper-end value is {\em exclusive}; the
+sequence is equivalent to \code{for (x = lower; x < upper; x +=
+  delta)}.  
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
@@ -4378 +4389 @@
 \end{prototype}
 
+\begin{prototype}
+long psVectorCountPixelMask (psVector *mask, psMaskValue value);
+\end{prototype}
+This function returns the number of pixels in the \code{vector} which
+satisfy any of the mask bits.  An error (eg, invalid vector) results
+in a return value of -1.  The vector must be of the same type as
+psMaskValue.
+
 \subsubsection{Histograms}
 \label{sec:histograms}
@@ -4640 +4659 @@
 We specify two minimization engines, Levenberg-Marquardt and Powell,
 since the former is relatively robust, but requires derivatives to be
-known, while the latter does not always obtain the best fit but does
-not require derivatives to be known.
-
-In both cases, we need to be able to set the maximum number of
-iterations (\code{maxIter}) and the maximum tolerance (\code{tol}) for
-convergence, in addition to having some basic information on the
-results and quality of the minimization: the value of the function at
-the minimum (\code{value}), the number of iterations performed
-(\code{iter}) and last change in tolerance before returning
-(\code{lastDelta}).
+known, while the latter does not require derivatives to be known, but
+since it needs to calculate the derivatives on the fly, is more
+computationally intensive.  
+
+We define the structure \code{psMinimization} to carry in user-defined
+limits on the minimization process, and to carry out information about
+the minimization analysis.  The maximum number of iterations is
+specified by \code{maxIter}, while the maximum tolerance for
+convergence is \code{tol}.  Four parameter vectors describe the
+behavior of individual parameters.  The \code{paramMask} vector
+defines the free (0) or frozen (not 0) parameters.  The
+\code{paramMin} defines the minimum allowed value for each parameter,
+while \code{paramMax} defines the maximum allowed value for each
+parameter.  During the analysis, parameters which would trend beyond
+these limits saturate to the limit.  The \code{paramDelta} specifies
+the maximum allowed absolute value of the swing of a given parameter.
+If the delta for a specific parameter would be larger than this, the
+swing is saturated to the limit (with the correct sign).  Any of these
+parameter vectors may be set to \code{NULL}, in which case the concept
+is ignored in the analysis.  The output information carried by the
+structure consists of the value of the function at the minimum
+(\code{value}), the number of iterations performed (\code{iter}) and
+last change in tolerance before returning (\code{lastDelta}).
 
 \begin{datatype}
@@ -4658 +4690 @@
     int iter;                           ///< Actual number of iterations performed
     float lastDelta;                    ///< Last change before quitting
+    psVector *paramMask;                ///< valid / invalid parameters
+    psVector *paramMax;                 ///< max allowed parameters
+    psVector *paramMin;                 ///< min allowed parameters
+    psVector *paramDelta;               ///< max allowed param swing
 } psMinimization;
 \end{datatype}
@@ -4665 +4701 @@
 psMinimization *psMinimizationAlloc(int maxIter, float tol);
 \end{prototype}
+and the parameter vectors are initially set to \code{NULL}.  
 
 \subsubsection{Levenberg-Marquardt}
@@ -4685 +4722 @@
                       psImage *covar, 
                       psVector *params,
-                      const psVector *paramMask, 
                       const psArray  *x, 
                       const psVector *y,
@@ -4694 +4730 @@
 The function shall return \code{false} in the event that there was an
 error (bad input, too many iterations), and also call \code{psError}.
+It is not an error for the minimization to reach one of the parameter
+limits.  
 
 The minimization specification, \code{min}, shall be modified with the
@@ -4741 +4779 @@
 the goodness of a fit.  This function returns the distances of the
 current parameter set from the minimization parameters expected from
-Gauss-Newton minimization.  The arguments are identical to those of
-\code{psMinimizeLMChi2}, except only the single vector \code{delta},
-consisting of the distances, is returned.  This vector must be
-pre-allocated to the dimenstions of \code{params}.
+Gauss-Newton minimization.  The arguments have the same meaning as for
+\code{psMinimizeLMChi2}.  The returned vector, \code{delta}, consists
+of the distances.  This vector must be pre-allocated to the
+dimenstions of \code{params}.
 
 %% \subsubsubsection{Pre-defined Functions for LM}
@@ -4785 +4823 @@
 
 \begin{prototype}
-bool psMinimizePowell(psMinimization *min, psVector *params, const psVector *paramMask,
-                      const psArray *coords, psMinimizePowellFunc func);
+bool psMinimizePowell(psMinimization *min, 
+                      psVector *params, 
+                      const psVector *paramMask,
+                      const psArray *coords, 
+                      psMinimizePowellFunc func);
 \end{prototype}
 
 The inputs and general behavior of this function is the same as for
 \code{psMinimizeLM}, except for the absence of the covariance matrix,
-\code{covar}.
+\code{covar} \note{why does this not have a covar matrix?}
 
 \subsubsubsection{Minimizing $\chi^2$ with Powell}
@@ -4803 +4844 @@
 
 \begin{prototype}
-bool psMinimizeChi2Powell(psMinimization *min, psVector *params, const psVector *paramMask,
-                          const psArray *coords, const psVector *value, const psVector *error,
+bool psMinimizeChi2Powell(psMinimization *min, 
+                          psVector *params, 
+                          const psVector *paramMask,
+                          const psArray *coords, 
+                          const psVector *value, 
+                          const psVector *error,
                           psMinimizeChi2PowellFunc model);
 \end{prototype}
@@ -4812 +4857 @@
 instead of being provided the function to be minimized, it is provided
 a model to fit and must calculate the $\chi^2$ to be minimized itself.
+
+\note{why does this not have a covar matrix?}
+
+\note{unify the param names with psMinimizeLMChi2Func?}
 
 For this purpose, \code{value} shall contain measured values at the
@@ -5009 +5058 @@
 
 \begin{prototype}
+psVector *psImageRow(psVector *out, 
+                     const psImage *input,
+                     psU32 row);
+psVector *psImageCol(psVector *out, 
+                     const psImage *input,
+                     psU32 column);
+\end{prototype}
+
+These two functions extract a single complete \code{row} or
+\code{column} from the \code{image} and return it to the provided
+\code{vector}, allocating it if it is \code{NULL}.  These are provided
+as fast, simple implementations of data extraction.  For more
+sophisticated operations, the following two functions should be used. 
+
+\begin{prototype}
 psVector *psImageSlice(psVector *out, 
                        psPixels *coords,
@@ -5257 +5321 @@
 following types: \code{psS8}, \code{psU16}, \code{psF32},
 \code{psF64}.
+
+\begin{prototype}
+long psImageCountPixelMask (psImage *mask, psRegion region, psMaskValue value);
+\end{prototype}
+This function returns the number of pixels in the image region which
+satisfy any of the mask bits.  An error (eg, invalid image, invalid
+region) results in a return value of -1.
 
 \begin{prototype}
@@ -6230 +6301 @@
 bool psTimerStart(char *name);
 psF64 psTimerMark(char *name);
+psF64 psTimerClear(char *name);
 psF64 psTimerStop(void);
 \end{prototype}
@@ -6236 +6308 @@
 timers, under the supplied \code{name}.  \code{psTimerMark} shall
 return the elapsed time, in seconds, for the timer specified by
-\code{name}.  \code{psTimerStop} shall free all memory associated with
-the timers, so that no memory is leaked, and return the maximum time
-expended.
-
+\code{name}.  \code{psTimerClear} resets the named timer.
+\code{psTimerStop} shall free all memory associated with all timers,
+so that no memory is leaked, and return the maximum time expended.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%