Index: trunk/doc/pslib/ChangeLogADD.tex =================================================================== --- trunk/doc/pslib/ChangeLogADD.tex (revision 3069) +++ trunk/doc/pslib/ChangeLogADD.tex (revision 3070) @@ -17,3 +17,4 @@ \item Added short section on histograms in the presence of errors. \item Added short note on inverse spherical transformations. +\item Added section on astronomical object models \end{itemize} Index: trunk/doc/pslib/ChangeLogSDRS.tex =================================================================== --- trunk/doc/pslib/ChangeLogSDRS.tex (revision 3069) +++ trunk/doc/pslib/ChangeLogSDRS.tex (revision 3070) @@ -1,3 +1,3 @@ -%%% $Id: ChangeLogSDRS.tex,v 1.57 2005-01-20 08:25:45 price Exp $ +%%% $Id: ChangeLogSDRS.tex,v 1.58 2005-01-22 01:57:42 eugene Exp $ \subsection{Changes from version 00 to version 01} @@ -390,5 +390,5 @@ \end{itemize} -\subsection{Changes from Revision 10 (30 November 2004) to present} +\subsection{Changes from Revision 10 (30 November 2004) to Revision 11 (21 January 2005)} \begin{itemize} @@ -399,6 +399,5 @@ \item fixed error of psHash to psMetadata in psFitsReadHeaderSet \item added psFitsWriteImage -\item changed psFitsWriteImageSection to psFitsUpdateImage {\bf verify} -\item changed psFitsWriteHeader to psFitsUpdateHeader {\bf verify} +\item changed psFitsWriteImageSection to psFitsUpdateImage \item added header entry to psFitsWriteTable \item added psFitsUpdateTable @@ -434,3 +433,4 @@ PS_INTERPOLATE_BICUBIC_VARIANCE, PS_INTERPOLATE_SINC_VARIANCE}. \item Added \code{psImageTransform}. -\end{itemize} +\item Added section of Database Functions +\end{itemize} Index: trunk/doc/pslib/psLibADD.tex =================================================================== --- trunk/doc/pslib/psLibADD.tex (revision 3069) +++ trunk/doc/pslib/psLibADD.tex (revision 3070) @@ -1,3 +1,3 @@ -%%% $Id: psLibADD.tex,v 1.56 2004-12-21 21:37:08 price Exp $ +%%% $Id: psLibADD.tex,v 1.57 2005-01-22 01:57:42 eugene Exp $ \documentclass[panstarrs]{panstarrs} @@ -1646,5 +1646,5 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsubsection{Missing and Todo} +\subsection{Missing and Todo} \tbd{define SINC, LAGRANGE interpolation} @@ -1659,4 +1659,263 @@ \tbd{define Brent's method \& minimization bracketing} + +\section{Pan-STARRS Modules} + +\subsection{Object Models} + +\subsubsection{Real 2D Gaussian} + +This function is a two-dimensional Gaussian with an elliptical +cross-section and a constant local background: +\[ +f(x,y) = Z_o e^{-z} + S_o +\] +where +\[ +z = \frac{(x - x_o)^2}{2\sigma_x^2} + \frac{(y-y_o)^2}{2\sigma_y^2} + (x-x_o) (y - y_o) \sigma_{xy} +\] + +Below is the relationship between the \code{psModel} parameters and +the function parameters, sample C-code implementing the function +efficiently, and the value of the derivatives: + +\begin{verbatim} + param[0] = So; + param[1] = Zo; + param[2] = Xo; + param[3] = Yo; + param[4] = sqrt(2) / SigmaX; + param[5] = sqrt(2) / SigmaY; + param[6] = Sxy; + + X = x[0] - param[2]; + Y = x[1] - param[3]; + + px = param[4]*X; + py = param[5]*Y; + + z = 0.5*SQ(px) + 0.5*SQ(py) + param[6]*X*Y; + r = exp(-z); + f = param[1]*r + param[0]; + /* f is the function value */ + + q = param[1]*r; + deriv[0] = +1; + deriv[1] = +r; + deriv[2] = q*(2*px*param[4] + param[6]*Y); + deriv[3] = q*(2*py*param[5] + param[6]*X); + deriv[4] = -2*q*px*X; + deriv[5] = -2*q*py*Y; + deriv[6] = -q*X*Y; +\end{verbatim} + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\subsubsection{Pseudo-Gaussian} + +This function is a polynomial approximation of a 2D Gaussian. The +function is very similar to the real Gaussian: +\[ +f(x,y) = Z_o (1 + z + z^2/2 + z^3/6)^{-1} + S_o +\] +where +\[ +z = \frac{(x - x_o)^2}{2\sigma_x^2} + \frac{(y-y_o)^2}{2\sigma_y^2} + (x-x_o) (y - y_o) \sigma_{xy} +\] + +Below is the relationship between the \code{psModel} parameters and +the function parameters, sample C-code implementing the function +efficiently, and the value of the derivatives: + +\begin{verbatim} + param[0] = So; + param[1] = Zo; + param[2] = Xo; + param[3] = Yo; + param[4] = sqrt(2) / SigmaX; + param[5] = sqrt(2) / SigmaY; + param[6] = Sxy; + + X = x[0] - param[2]; + Y = x[1] - param[3]; + + px = param[4]*X; + py = param[5]*Y; + + z = 0.5*SQ(px) + 0.5*SQ(py) + param[6]*X*Y; + t = 1 + z + 0.5*z*z; + r = 1.0 / (t*(1 + z/3)); /* ~ exp (-Z) */ + f = param[1]*r + param[0]; + /* f is the function value */ + + /* note difference from a pure gaussian: q = param[1]*r */ + q = param[1]*r*r*t; + deriv[0] = +1; + deriv[1] = +r; + deriv[2] = q*(2*px*param[4] + param[6]*Y); + deriv[3] = q*(2*py*param[5] + param[6]*X); + deriv[4] = -2*q*px*X; + deriv[5] = -2*q*py*Y; + deriv[6] = -q*X*Y; +\end{verbatim} + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\subsubsection{Waussian} + +The Waussian is a modified polynomial approximation of a 2D Gaussian, +with non-linear polynomial terms having variable coefficients, rather +than the Taylor series values of 1/2 and 1/6. The +function is very similar to the pseudo-Gaussian: +\[ +f(x,y) = Z_o (1 + z + B_2 (z^2/2 + B_3 z^3/6))^{-1} + S_o +\] +where +\[ +z = \frac{(x - x_o)^2}{2\sigma_x^2} + \frac{(y-y_o)^2}{2\sigma_y^2} + (x-x_o) (y - y_o) \sigma_{xy} +\] + +Below is the relationship between the \code{psModel} parameters and +the function parameters, sample C-code implementing the function +efficiently, and the value of the derivatives. Note the fudge factors +of 100 in the derivatives of $B_2$ and $B_3$: these are included to +slow the variation of these parameters, which are otherwise very +sensitive to small errors. + +\begin{verbatim} + param[0] = So; + param[1] = Zo; + param[2] = Xo; + param[3] = Yo; + param[4] = Sx; + param[5] = Sy; + param[6] = Sxy; + param[7] = B2; + param[8] = B3; + + X = x - param[2]; + Y = y - param[2]; + + px = param[4]*X; + py = param[5]*Y; + + z = 0.5*SQ(px) + 0.5*SQ(py) + param[6]*X*Y; + t = 0.5*z*z*(1 + param[8]*z/3); + r = 1.0 / (1 + z + param[7]*t); /* ~ exp (-Z) */ + f = param[1]*r + param[0]; + + /* note difference from gaussian: q = param[1]*r */ + q = param[1]*r*r*(1 + param[7]*z*(1 + param[8]*z/2)); + deriv[0] = +1; + deriv[1] = +r; + deriv[2] = q*(2*px*param[4] + param[6]*Y); + deriv[3] = q*(2*py*param[5] + param[6]*X); + deriv[4] = -2*q*px*X; + deriv[5] = -2*q*py*Y; + deriv[6] = -q*X*Y; + deriv[7] = -100*param[1]*r*r*t; + deriv[8] = -100*param[1]*r*r*param[7]*(z*z*z)/6; + /* the values of 100 dampen the swing of param[7,8] */ +\end{verbatim} + +\subsubsection{Twisted Gaussian} + +This function describes an object with power-law wings and a flattened +core, where the core has a different contour from the wings. + +\[ +f(x,y) = Z_{\rm pk} (1 + z_1 + z_2^M)^{-1} + Sky +\] +where +\[ +z_1 = \frac{x^2}{2\sigma_{x,in}^2} + \frac{y^2}{2\sigma_{y,in}^2} + x y \sigma_{xy,in} +z_2 = \frac{x^2}{2\sigma_{x,out}^2} + \frac{y^2}{2\sigma_{y,out}^2} + x y \sigma_{xy,out} +\] + +\begin{verbatim} + param[0] = So; + param[1] = Zo; + param[2] = Xo; + param[3] = Yo; + param[4] = SxInner; + param[5] = SyInner; + param[6] = SxyInner; + param[7] = SxOuter; + param[8] = SyOuter; + param[9] = SxyOuter; + param[10] = N; + + X = x - param[2]; + Y = y - param[3]; + + px1 = param[4]*X; + py1 = param[5]*Y; + px2 = param[7]*X; + py2 = param[8]*Y; + + z1 = 0.5*SQ(px1) + 0.5*SQ(py1) + param[4]*X*Y; + z2 = 0.5*SQ(px2) + 0.5*SQ(py2) + param[9]*X*Y; + + r = 1.0 / (1 + z1 + pow(z2,param[10])); + f = param[5]*r + param[6]; + + q1 = param[5]*SQ(r); + q2 = param[5]*SQ(r)*param[10]*pow(z2,(param[10]-1)); + + deriv[0] = +1; + deriv[1] = +r; + deriv[2] = q1*(2*px1*param[4] + param[6]*Y) + q2*(2*px2*param[7] + param[9]*Y); + deriv[3] = q1*(2*py1*param[5] + param[6]*X) + q2*(2*py2*param[8] + param[9]*X); + + /* these fudge factors impede the growth of param[4] beyond param[7] */ + f1 = fabs(param[7]) / fabs(param[4]); + f2 = (f1 < FSCALE) ? 1 : FFACTOR*(f1 - FSCALE) + 1; + deriv[4] = -2*q1*px1*X*f2; + + /* these fudge factors impede the growth of param[5] beyond param[8] */ + f1 = fabs(param[8]) / fabs(param[5]); + f2 = (f1 < FSCALE) ? 1 : FFACTOR*(f1 - FSCALE) + 1; + deriv[5] = -2*q1*py1*Y*f2; + + deriv[6] = -q1*X*Y; + + deriv[7] = -2*q2*px2*X; + deriv[8] = -2*q2*py2*Y; + deriv[9] = -q2*X*Y; + deriv[10] = -q1*ln(z2); +\end{verbatim} + +The intial guess for the Gaussian parameters may be taken from the +moments, peak value, and local sky. + +\tbd{future galaxy models to be implemented} + +\begin{verbatim} +float Sersic() + param[0] = So; + param[1] = Zo; + param[2] = Xo; + param[3] = Yo; + param[4] = Sx; + param[5] = Sy; + param[6] = Sxy; + param[7] = Nexp; + +float SersicBulge() + param[0] So; + param[1] Zo; + param[2] Xo; + param[3] Yo; + param[4] SxInner; + param[5] SyInner; + param[6] SxyInner; + param[7] Zd; + param[8] SxOuter; + param[9] SyOuter; + param[10] = SxyOuter; + param[11] = Nexp; +\end{verbatim} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Index: trunk/doc/pslib/psLibSDRS.tex =================================================================== --- trunk/doc/pslib/psLibSDRS.tex (revision 3069) +++ trunk/doc/pslib/psLibSDRS.tex (revision 3070) @@ -1,3 +1,3 @@ -%%% $Id: psLibSDRS.tex,v 1.168 2005-01-20 08:25:55 price Exp $ +%%% $Id: psLibSDRS.tex,v 1.169 2005-01-22 01:57:42 eugene Exp $ \documentclass[panstarrs,spec]{panstarrs} @@ -11,5 +11,5 @@ \project{Pan-STARRS Image Processing Pipeline} \organization{Institute for Astronomy} -\version{10} +\version{11} \docnumber{PSDC-430-007} @@ -41,4 +41,5 @@ 09 & 2004 Nov 15 & final for cycle 4 \\ \hline 10 & 2004 Nov 30 & update for cycle 4 \\ +11 & 2005 Jan 21 & draft for cycle 5 \\ \RevisionsEnd @@ -97,5 +98,5 @@ search path, PSLib may be used within a program by including the line \code{#include } into the C code and linking with -\code{-lpslib}. +\code{-lpslib.} This document describes the data structures and details the functions @@ -109,5 +110,5 @@ principle relevance and VerbPhase describes the operation applied to that data type. For example, the function which copies an image (of -type \code{psImage}) is called \code{psImageCopy()}. +type \code{psImage}) is called \code{psImageCopy().} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -4193,5 +4194,5 @@ typedef xmlDocPtr psXMLDoc; void psXMLDocFree(psXMLDoc *doc); -\end{verbatim}. +\end{verbatim} The next pair of functions convert a \code{psMetadata} data structure @@ -4200,5 +4201,5 @@ psXMLDoc *psMetadataToXMLDoc(const psMetadata *metadata); psMetadata *psXMLDocToMetadata(const psXMLDoc *doc); -\end{verbatim}. +\end{verbatim} The next pair of functions loads the data in a named file into a @@ -4208,5 +4209,5 @@ psXMLDoc *psXMLParseFile(const char *filename); int psXMLDocToFile(const psXMLDoc *doc, const char *filename); -\end{verbatim}. +\end{verbatim} The next pair of functions accepts a block of memory and parses it @@ -4215,5 +4216,5 @@ psXMLDoc *psXMLParseMemory(const char *buffer, const int size); int psXMLDocToMemory(const psXMLDoc *doc, char *buffer); -\end{verbatim}. +\end{verbatim} The next pair of functions read from and write to a file descriptor. @@ -4224,5 +4225,174 @@ psXMLDoc *psXMLParseFD(int fd); int psXMLDocToFD(const psXMLDoc *doc, int fd); -\end{verbatim}. +\end{verbatim} + +\subsection{Database Functions} + +Many of the applications that PSLib will be used for will require +access to a simple relational database. PSLib includes generic +database-independent interface mechanisms as part of its API set. The +most important aspect of PSLib's database support is to abstract as +much database specific complexity as is feasible. As almost all RDBMS +provide at least a simple transactional model, commit and rollback +support should be provided. + +Currently, only support for MySQL 4.1.x is required but other backends +may be added as options in the future. As a particular example which +has implications for the database interaction model, support for +SQLite may be required in the future. Currently, the choice of +backend database interface may be made as a compile option. Details +of the specified APIs in the discussion below refer to the relevant +MySQL functions. + +Database errors must be trapped and placed onto the psError stack. +The complete error message should be retrieved with the database's +error function. + +\subsubsection{Managing the Database Connection} + +We specify a database handle which carries the information about the +database connection: + +\begin{verbatim} + typedef struct { + MYSQL mysql; + } psDB; +\end{verbatim} + +The following collection of functions provides basic database functionality: + +\begin{verbatim} + // wraps mysql_init() & mysql_real_connect() + psDB *psDBInit(const char *host, const char *user, const char *passwd, const char *dbname); + + // wraps mysql_close() + void psDBCleanup(psDB *dbh); + + // wraps mysql_create_db() + bool psDBCreate(psDB *dbh, const char *dbname); + + // wraps mysql_select_db() + bool psDBChange(psDB *dbh, const char *dbname); + + // wraps mysql_drop_db() + bool psDBDrop(psDB *dbh, const char *dbname); +\end{verbatim} + +For MySQL support, \code{psDBInit()} wraps \code{mysql_init()} and +\code{mysql_real_connect()} in order to initialize a psDB structure and +establish a database connection. A null pointer should be returned on +failure. + +When implementing support for SQLite, or other DB which is purely +file-based, the \code{host}, \code{user}, and \code{passwd} arguments +would be ignored while \code{dbname} would specify the path to the +SQLite db file. + +\subsubsection{Interacting with Database Tables} + +The functions in this section perform high level interactions with the +database tables. All of them should behave ``atomically'' with +respect to the state of the database. Specifically, all interactions +with the database should be done as a part of a transaction that is +rolled-back on failure and committed only after all queries used by +the API have been run. In general, this API set attempts to treat a +database table as a 2D matrix where columns can be represented by a +\code{psVector} and rows as a \code{psMetadata} type. A +\code{psMetadata} collection is also used to define the columns of a +table and as part of the query restrictions. + +\begin{verbatim} + bool psDBCreateTable(psDB *dbh, const char *tableName, psMetadata *md); +\end{verbatim} + +This function generates and executes the SQL needed to create a table +named \code{tableName}, with the column names and datatypes as +described in \code{md}. Each data item in the \code{psMetadata} +collection represents a single table field. The name of the field is +given by the name of the \code{psMetadataItem} and the data type is +give by the \code{psMetadataItem.type} and \code{psMetadataItem.ptype} +entries. A lookup table should be used to convert from PSLib types +into MySQL compatible SQL data types. For example, a +\code{PS_META_STR} would map to an SQL99 varchar. If value of +\code{type} is \code{PS_META_STR} then the \code{psMetadataItem.data} +element is set to a string with the length for the field written as a +text string. The value of the \code{psMetadataItem.data} element is +unused for the \code{PS_META_PRIMITIVE} types. Other metadata types +beyond \code{PS_META_STR} and \code{PS_META_PRIMITIVE} are not allowed +in a table definition metadata collection. + +Database indexes can be specified setting the \code{comment} field to +``\code{Primary Key}'' or ``\code{Key}''. Comment are otherwise +ignored. + +\begin{verbatim} + bool psDBDropTable(psDB *dbh, const char *tableName); +\end{verbatim} + +This function deletes the specified table. + +\begin{verbatim} + psArray *psDBSelectColumn(psDB *dbh, const char *tableName, const char *col, const psU64 limit); + psVector *psDBSelectColumnNum(psDB *dbh, const char *tableName, const char *col, const psU64 limit); +\end{verbatim} + +These functions generates and executes the SQL needed to select an +entire column from a table or up to \code{limit} rows from it. If +\code{limit} is 0, the entire range is returned. The database +response is processed and a \code{psArray} of elements of the native +type is returned. The Num version of the function returns the data in +a \code{psVector}. It returns an error (NULL) if the requested field +is not a numerical type. + +\begin{verbatim} + psArray *psDBSelectRows(psDB *dbh, const char *tableName, psMetadata *where); +\end{verbatim} + +This function returns rows from the specified table which match +the restrictions given by \code{where}. The restrictions are +specified as field / value pairs. The \code{psMetadata} collection +where must consist of valid database fields, though the database query +checking functions may be used to validate the fields as part of the +query. If \code{where} is \code{NULL}, then there are no restrictions +on the rows selected. The selected rows are returned as a +\code{psArray} of \code{psMetadata} values, one per row. + +\begin{verbatim} + bool psDBInsertRow(psDB *dbh, const char *tableName, psMetadata *row); +\end{verbatim} + +Insert the data from \code{row} into \code{tableName}. It should be noted in +the API reference that if fields are specified in \code{row} that do not exist +in \code{tablename}, the insert will fail. + +\begin{verbatim} + psArray *psDBDumpRows(psDB *dbh, const char *tableName); +\end{verbatim} + +Fetch all rows as an psArray of psMetadata. + +\begin{verbatim} + psArray *psDBDumpCols(psDB *dbh, const char *tableName); +\end{verbatim} + +Fetch all columns as an psArray of psVector + +\begin{verbatim} + psU64 psDBUpdateRow(psDB *dbh, const char *tableName, psMetadata *where, psMetadata *values); +\end{verbatim} + +Update the columns contained in \code{values} in the row(s) that have a field +with the value indicated by \code{where} (note that this is only allows very +limited use of SQL99's ``where'' semantics). The number of rows modified is +returned. If there are multiple psMetadataItems in \code{where} then each item +should be considered as an additional constraint. e.g. ``where foo = x and +where bar = y'' + +\begin{verbatim} + psU64 psDBDeleteRow(psDB *dbh, const char *tableName, psMetadata *where); +\end{verbatim} + +Delete the rows that are matched by \code{where} using the same semantics for +\code{where} as in psDBUpdateRow(). \subsection{FITS I/O Functions} @@ -4598,10 +4768,7 @@ \begin{verbatim} -psPlaneTransform *psPlaneTransformInvert(psPlaneTransform *out, const psPlaneTransform *in, float xMin, - float xMax, float yMin, float yMax, int nSamples); -psPlaneTransform *psPlaneTransformCombine(psPlaneTransform *out, const psPlaneTransform *trans1, - const psPlaneTransform *trans2); -bool psPlaneTranformFit(psPlaneTransform *trans, const psArray *source, const psArray *dest, int nRejIter, - float sigmaClip); +psPlaneTransform *psPlaneTransformInvert(psPlaneTransform *out, const psPlaneTransform *in, psRegion *region, int nSamples); +psPlaneTransform *psPlaneTransformCombine(psPlaneTransform *out, const psPlaneTransform *trans1, const psPlaneTransform *trans2); +bool psPlaneTranformFit(psPlaneTransform *trans, const psArray *source, const psArray *dest, int nRejIter, float sigmaClip); \end{verbatim} @@ -4612,24 +4779,22 @@ the forward transformation. In the event that the input transformation is linear, an exact solution may be calculated; -otherwise \code{nSamples} samples in each axis, ranging from -\code{xMin} to \code{xMax} and \code{yMin} to \code{yMax} shall be -used as a grid to fit the best inverse transformation. The function -shall return \code{NULL} if it was unable to generate the inverse -transformation; otherwise it shall return the inverse transformation. -In the event that \code{out} is \code{NULL}, a new -\code{psPlaneTransform} shall be allocated and returned. - -\code{psPlaneTransformSubsume} takes two transformations +otherwise \code{nSamples} samples in each axis, covering the region +specified by \code{region} shall be used as a grid to fit the best +inverse transformation. The function shall return \code{NULL} if it +was unable to generate the inverse transformation; otherwise it shall +return the inverse transformation. In the event that \code{out} is +\code{NULL}, a new \code{psPlaneTransform} shall be allocated and +returned. + +\code{psPlaneTransformCombine} takes two transformations (\code{trans1} and \code{trans2}) and returns a single transformation that has the effect of performing \code{trans1} followed by \code{trans2}. The function shall return \code{NULL} if it was unable to generate the transformation; otherwise it shall return the -transformation. \tbd{Not sure on the algorithm yet --- it may be the -same as for \code{psPlaneTransformInvert}, in which case we will need -the ranges and number of samples as well.} - -\code{psPlaneTransform} takes two arrays containing matched -coordinates (i.e., coordinates the \code{source} correspond to the -appropriate coordinates in the \code{dest}) and returns the +transformation. + +\code{psPlaneTransformFit} takes two arrays containing matched +coordinates (i.e., coordinates in the \code{source} array correspond +to the coordinates in the \code{dest} array) and returns the best-fitting transformation. The \code{source} and \code{dest} will contain \code{psCoord}s. In the event that the number of coordinates