Index: trunk/doc/pslib/psLibSDRS.tex =================================================================== --- trunk/doc/pslib/psLibSDRS.tex (revision 4253) +++ trunk/doc/pslib/psLibSDRS.tex (revision 4254) @@ -1,3 +1,3 @@ -%%% $Id: psLibSDRS.tex,v 1.282 2005-06-15 00:30:28 eugene Exp $ +%%% $Id: psLibSDRS.tex,v 1.283 2005-06-15 00:52:45 price Exp $ \documentclass[panstarrs,spec]{panstarrs} @@ -148,18 +148,45 @@ containers (doubly-linked lists, hashes, arrays). -\subsection{Threads} - -Pan-STARRS does not have a strong requirement for multithreading -capability. The memory management functions, defined below, must be -written to be thread-safe. The use of \code{static} variables should -be kept to an absolute minimum and then only when protected by a -``mutex''. Cross-thread synchronization for PSLib's fundamental -datatypes (\code{psArray}, \code{psList}, etc.) is left to the end-user. - -PSLib shall provide some functions and structures that assist in -thread-safety, but it is left up to the end-user to employ these. -That is, these conveniences are not used internally by PSLib, and so -the user should make no assumptions about thread-safety (outside the -memory management functions). +\subsection{Threads and Re-entrancy} + +Due to current developments in CPU architecture, we must assume that +PSLib will be used in a threaded environment. However, coding the +library to be thread-safe may have implications for the speed of the +library and for the simplicity of use. We therefore make the +following policies: + +\begin{itemize} +\item The memory management functions, defined below, must be written + to be thread-safe, since we cannot risk this crucial area being + unstable. +\item Re-entrant versions of system calls and external library + functions should be used. We expect that these cases are + sufficiently small that we are prepared to err on the side of + caution. +\item The practise of using \code{static} variable to achieve high + efficiency (e.g., so that subsequent calls do not have to repeat a + large memory allocation) should be kept to an absolute minimum. + Where it has been justified (i.e., through code profiling), the + \code{static} variable must be protected by a ``mutex''. +\item Cross-thread synchronization for PSLib's fundamental datatypes + (\code{psArray}, \code{psList}, etc.) is left to the end-user + (although some convenience is provided --- see below). +\end{itemize} + +As a convenience to the user in achieving thread-safe operation, each +of the data structures classified as a ``collection'' (i.e., +\code{psList, psHash, psMetadata, psArray, psPixels, psVector, +psBitSet}) and \code{psImage} shall contain a member, \code{void +*lock}, which provides a place for the user to carry around a mutex or +semaphore. This is provided so that the user doesn't have to pass +around both the structure and a mutex, or wrap PSLib structures in +their own thread-safe structures that contain a mutex. PSLib is not +responsible for allocating, setting, checking or freeing the +\code{lock} --- these are entirely the responsibility of the user. +PSLib provides only a place to hang it. Your mileage may vary. + +If these policies become a burden on the processing speed, we can +investigate alternative measures, such as defining specifically +re-entrant versions of select speed-critical functions. \subsection{Angles} @@ -343,10 +370,10 @@ struct psMemBlock* previousBlock; ///< previous block in allocation list struct psMemBlock* nextBlock; ///< next block allocation list - psFreeFunc freeFunc; ///< deallocator. If NULL, use generic deallocation. + psFreeFunc freeFunc; ///< deallocator. If NULL, use generic deallocation. size_t userMemorySize; ///< the size of the user-portion of the memory block const psMemId id; ///< a unique ID for this allocation const char* file; ///< set from __FILE__ in e.g. p_psAlloc const unsigned int lineno; ///< set from __LINE__ in e.g. p_psAlloc - pthread_mutex_t refCounterMutex; ///< mutex to ensure exclusive access to reference counter + pthread_mutex_t refCounterMutex; ///< mutex to ensure exclusive access to reference counter psReferenceCount refCounter; ///< how many times pointer is referenced bool persistent; ///< marks if non-user persistent data like error stack, etc. @@ -787,4 +814,33 @@ \code{psStringCopy} or \code{psStringNCopy}). +\subsubsection{Strings} + +The use of strings within the PSLib memory management system requires +that they be allocated with \code{psAlloc}. This means that +substrings cannot be placed on containers such as \code{psArray} or +\code{psMetadata}, since these check for the existence of the attached +\code{psMemBlock}. To get around this, we specify functions that copy +a string into \code{psAlloc}-ed memory: +\begin{prototype} +psString psStringCopy(const char *string); +psString psStringNCopy(const char *string, unsigned int nChar); +\end{prototype} + +\code{psStringCopy} shall perform a deep copy of the \code{string} +into \code{psAlloc}-ed memory. \code{psStringNCopy} shall do the +same, up to a maximum of \code{nChar} characters. + +We also specify two useful functions: + +\begin{prototype} +ssize_t psStringAppend(char **dest, const char *format, ...) +ssize_t psStringPrepend(char **dest, const char *format, ...) +\end{prototype} + +\code{psStringAppend} shall append, according to the \code{format} +values into the \code{dest} string. \code{dest} shall be allocated if +\code{NULL}, and the length of the new string (excluding the +terminator) shall be returned. \code{psStringPrepend} shall do +similarly, except it shall prepend to the \code{dest} string. \subsubsection{Type information} @@ -943,49 +999,34 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{Threads} - -As pointed out earlier, PSLib makes no guarantees for thread-safety -outside of the memory management functions. Nevertheless, the following -facilities are provided as a convenience to the user. - -Each of the data structures classified as a ``collection'' (i.e., -\code{psList, psHash, psMetadata, psArray, psPixels, psVector, -psBitSet}) and \code{psImage} shall contain an additional member, -\code{void *lock}, which provides a place for the user to carry around -a mutex or semaphore. This is provided so that the user doesn't have -to pass around both the structure and a mutex, or wrap PSLib -structures in their own thread-safe structures that contain a mutex. -PSLib is not responsible for freeing the \code{lock} (it may not even -be allocated using the PSLib memory management system) --- it is -entirely the responsibility of the user and PSLib provides only a -place to hang it. - -We also define the following conveniences: -\begin{datatype} -typedef struct { - pthread_mutex_t mutex; -} psMutex; -\end{datatype} - -\begin{prototype} -psMutex *psMutexAlloc(void); -bool psMutexLock(psMutex *mutex); -bool psMutexUnlock(psMutex *mutex); -\end{prototype} - -\code{psMutex} simply wraps a POSIX thread mutex (this is necessary in -order to use the PS memory management system). - -\code{psMutexAlloc} shall allocate memory for a \code{psMutex}, and -initialise the mutex. \code{psMutexLock} shall lock the mutex in -\code{thread}, and \code{psMutexUnlock} shall unlock the mutex in -\code{thread}. - -These functions, in the interests of speed, should be implemented as -preprocessor macros. - -These functions and the \code{void *lock} in the collection structures -and \code{psImage} should only be defined if \code{_REENTRANT} is -defined; otherwise the functions should be poisoned. +%%% +%%% Not sure we need these --- all we do is wrap pthread_mutex, so why bother? +%%% + +%% \subsection{Locks} +%% +%% We define the following conveniences: +%% \begin{datatype} +%% typedef pthread_mutex_t psMutex; +%% \end{datatype} +%% +%% \begin{prototype} +%% psMutex *psMutexAlloc(void); +%% bool psMutexLock(psMutex *mutex); +%% bool psMutexUnlock(psMutex *mutex); +%% \end{prototype} +%% +%% \code{psMutex} simply wraps a POSIX thread mutex (this is necessary in +%% order to use the PS memory management system). +%% +%% \code{psMutexAlloc} shall allocate memory for a \code{psMutex}, and +%% initialise the mutex. \code{psMutexLock} shall lock the \code{mutex}, +%% and \code{psMutexUnlock} shall unlock the \code{mutex}. +%% +%% These functions, in the interests of speed, should be implemented as +%% preprocessor macros. +%% +%% These functions and the \code{void *lock} in the collection structures +%% and \code{psImage} should only be defined if \code{_REENTRANT} is +%% defined; otherwise the functions should be poisoned. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -3095,5 +3136,5 @@ psC32 **C32; ///< Pointers to complex floating-point data psC64 **C64; ///< Pointers to complex floating-point data - psPtr V; ///< Pointers to raw data + psPtr V; ///< Pointers to raw data } data; const struct psImage *parent; ///< parent, if a subimage @@ -3796,4 +3837,17 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\subsection{Masks} + +Several functions --- especially the statistical and image functions +--- use a mask vector and a mask value to specify values in an input +list that should be excluded from the calculations. We define a mask +type, which we will initially set to U8. In the event that our masks +exceed eight bits, we can then change the mask definition without +major changes throughout the code. + +\begin{datatype} +typedef psU8 psMask; +\end{datatype} + \subsection{Statistical Functions} @@ -3830,5 +3884,5 @@ const psVector *errors, const psVector *mask, - unsigned int maskVal + psMask maskVal ); \end{prototype} @@ -3859,5 +3913,5 @@ clipped statistics are modified according to the ADD; the robust median and quartiles are modified as specified in the ADD. The mask -must be of type \code{psU8}. +must be of type \code{psMask}. The \code{psStats} structure is defined with entries for each of the @@ -3974,5 +4028,5 @@ const psVector *errors, const psVector *mask, - unsigned int maskVal); + psMask maskVal); \end{prototype} The \code{values} vector may be of types \code{psU8, psU16, psF32, @@ -4006,5 +4060,5 @@ typedef struct { psPolynomialType type; ///< Polynomial type - psElemType ctype; ///< Polynomial precision + psElemType ctype; ///< Polynomial precision int n; ///< Number of terms union { @@ -4024,5 +4078,5 @@ typedef struct { psPolynomialType type; ///< Polynomial type - psElemType ctype; ///< Polynomial precision + psElemType ctype; ///< Polynomial precision int nX, nY; ///< Number of terms in x and y union { @@ -4034,5 +4088,4 @@ psF64 **F64; ///< Error in coefficients } coeffErr; - char *mask; ///< Coefficient mask char **mask; ///< Coefficients mask } psPolynomial2D; @@ -4077,15 +4130,15 @@ psF64 x); psF64 psPolynomial2DEval(const psPolynomial2D *poly, - psF64 x, - psF64 y); + psF64 x, + psF64 y); psF64 psPolynomial3DEval(const psPolynomial2D *poly, - psF64 x, - psF64 y, - psF64 z); + psF64 x, + psF64 y, + psF64 z); psF64 psPolynomial4DEval(const psPolynomial2D *poly, - psF64 x, - psF64 y, - psF64 z, - psF64 t); + psF64 x, + psF64 y, + psF64 z, + psF64 t); \end{prototype} @@ -4247,10 +4300,10 @@ bool psMinimizeLMChi2(psMinimization *min, psImage *covar, - psVector *params, + psVector *params, const psVector *paramMask, - const psArray *x, - const psVector *y, + const psArray *x, + const psVector *y, const psVector *yErr, - psMinimizeLMChi2Func func); + psMinimizeLMChi2Func func); \end{prototype} @@ -4287,7 +4340,7 @@ identical. -\code{paramMask} must be of type \code{psU8}, while \code{params} must -be of type \code{psF32}. The \code{func} function must be valid only -for types \code{psF32}, \code{psF64}. +\code{paramMask} must be of type \code{psMask}, while \code{params} +must be of type \code{psF32}. The \code{func} function must be valid +only for types \code{psF32}, \code{psF64}. \begin{prototype} @@ -4396,20 +4449,20 @@ \begin{prototype} psPolynomial1D *psVectorFitPolynomial1D(psPolynomial1D *poly, - const psVector *mask, - unsigned int maskValue - const psVector *f, + const psVector *mask, + psMask maskValue + const psVector *f, const psVector *fErr const psVector *x); psPolynomial2D *psVectorFitPolynomial2D(psPolynomial1D *poly, - const psVector *mask, - unsigned int maskValue - const psVector *f, + const psVector *mask, + psMask maskValue + const psVector *f, const psVector *fErr const psVector *x, const psVector *y); psPolynomial3D *psVectorFitPolynomial3D(psPolynomial1D *poly, - const psVector *mask, - unsigned int maskValue - const psVector *f, + const psVector *mask, + psMask maskValue + const psVector *f, const psVector *fErr const psVector *x, @@ -4417,7 +4470,7 @@ const psVector *z); psPolynomial4D *psVectorFitPolynomial4D(psPolynomial1D *poly, - const psVector *mask, - unsigned int maskValue - const psVector *f, + const psVector *mask, + psMask maskValue + const psVector *f, const psVector *fErr const psVector *x, @@ -4442,37 +4495,37 @@ \begin{prototype} psPolynomial1D *psVectorClipFitPolynomial1D(psPolynomial1D *poly, - psStats *stats, - const psVector *mask, - unsigned int maskValue - const psVector *f, - const psVector *fErr - const psVector *x); + psStats *stats, + const psVector *mask, + psMask maskValue + const psVector *f, + const psVector *fErr + const psVector *x); psPolynomial2D *psVectorClipFitPolynomial2D(psPolynomial1D *poly, - psStats *stats, - const psVector *mask, - unsigned int maskValue - const psVector *f, - const psVector *fErr - const psVector *x, - const psVector *y); + psStats *stats, + const psVector *mask, + psMask maskValue + const psVector *f, + const psVector *fErr + const psVector *x, + const psVector *y); psPolynomial3D *psVectorClipFitPolynomial3D(psPolynomial1D *poly, - psStats *stats, - const psVector *mask, - unsigned int maskValue - const psVector *f, - const psVector *fErr - const psVector *x, - const psVector *y, - const psVector *z); + psStats *stats, + const psVector *mask, + psMask maskValue + const psVector *f, + const psVector *fErr + const psVector *x, + const psVector *y, + const psVector *z); psPolynomial4D *psVectorClipFitPolynomial4D(psPolynomial1D *poly, - psStats *stats, - const psVector *mask, - unsigned int maskValue - const psVector *f, - const psVector *fErr - const psVector *x, - const psVector *y, - const psVector *z, - const psVector *t); + psStats *stats, + const psVector *mask, + psMask maskValue + const psVector *f, + const psVector *fErr + const psVector *x, + const psVector *y, + const psVector *z, + const psVector *t); \end{prototype} These functions replicate the capability of the vector fitting @@ -4575,5 +4628,5 @@ const psImage *input, const psImage *mask, - unsigned int maskVal, + psMask maskVal, psRegion region, psImageCutDirection direction, @@ -4610,5 +4663,5 @@ const psImage *input, const psImage *mask, - unsigned int maskVal, + psMask maskVal, psRegion region, unsigned int nSamples, @@ -4630,5 +4683,5 @@ const psImage *input, const psImage *mask, - unsigned int maskVal, + psMask maskVal, float x, float y, @@ -4681,5 +4734,5 @@ psImage *psImageRebin(psImage *out, const psImage *in, const psImage *mask, - unsigned int maskVal, + psMask maskVal, int scale, const psStats *stats); \end{prototype} @@ -4755,5 +4808,5 @@ const psImage *input, const psImage *inputMask, - int inputMaskVal, + psMask inputMaskVal, const psPlaneTransform *outToIn, psRegion region, @@ -4771,5 +4824,5 @@ If the \code{inputMask} is non-\code{NULL}, those pixels in the \code{inputMask} matching \code{inputMaskVal} are to be ignored in the -transformation. The \code{inputMask} must be of type \code{psU8}, and +transformation. The \code{inputMask} must be of type \code{psMask}, and of the same size as the \code{input}, otherwise the function shall generate an error and return \code{NULL}. The transformation @@ -4797,5 +4850,5 @@ const psImage *in, const psImage *mask, - unsigned int maskVal); + psMask maskVal); \end{prototype} Determine statistics for image (or subimage). The statistics to be @@ -4810,5 +4863,5 @@ const psImage *in, const psImage *mask, - unsigned int maskVal); + psMask maskVal); \end{prototype} Construct a histogram from an image (or subimage). The histogram to @@ -4838,5 +4891,5 @@ \begin{prototype} complex double psImagePixelInterpolate(const psImage *input, float x, float y, - const psImage *mask, unsigned int maskVal, + const psImage *mask, psMask maskVal, complex double unexposedValue, psImageInterpolateMode mode); \end{prototype} @@ -4921,6 +4974,6 @@ \begin{prototype} -psImage *psImageGrowMask(psImage *out, const psImage *in, unsigned int maskVal, - unsigned int growSize, unsigned int growValue); +psImage *psImageGrowMask(psImage *out, const psImage *in, psU16 maskVal, + unsigned int growSize, psU16 growVal); \end{prototype} @@ -4935,13 +4988,16 @@ \code{maskVal} shall have the corresponding pixel in the \code{out} image set to the \code{growValue}. The function must be defined for -the following types: \code{psU8}, \code{psU16}. - -\begin{prototype} -psImage *psPixelsToMask(psImage *out, const psPixels *pixels, psRegion region, unsigned int maskVal); -psPixels *psPixelsFromMask(psPixels *out, const psImage *mask, unsigned int maskVal); -\end{prototype} - -\code{psPixelsToMask} shall return an image of type U8 with the -\code{pixels} lying within the specified \code{region} set to the +the following types: \code{psU8}, \code{psU16}. Note that +\code{maskVal} and \code{growVal} are not of type \code{psMask}, but +\code{psU16} so that they match the maximum required type for the +\code{in} image. + +\begin{prototype} +psImage *psPixelsToMask(psImage *out, const psPixels *pixels, psRegion region, psMask maskVal); +psPixels *psPixelsFromMask(psPixels *out, const psImage *mask, psMask maskVal); +\end{prototype} + +\code{psPixelsToMask} shall return an image of type \code{psMask} with +the \code{pixels} lying within the specified \code{region} set to the \code{maskVal}. The \code{out} image shall be modified if supplied, or allocated and returned if \code{NULL}. The size of the output