IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Ignore:
Timestamp:
Jun 14, 2005, 2:52:45 PM (21 years ago)
Author:
Paul Price
Message:

Updating thread policy, removing psMutex (what's the point?), added psMask, psString functions

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/pslib/psLibSDRS.tex

    r4253 r4254  
    1 %%% $Id: psLibSDRS.tex,v 1.282 2005-06-15 00:30:28 eugene Exp $
     1%%% $Id: psLibSDRS.tex,v 1.283 2005-06-15 00:52:45 price Exp $
    22\documentclass[panstarrs,spec]{panstarrs}
    33
     
    148148containers (doubly-linked lists, hashes, arrays).
    149149
    150 \subsection{Threads}
    151 
    152 Pan-STARRS does not have a strong requirement for multithreading
    153 capability.  The memory management functions, defined below, must be
    154 written to be thread-safe.  The use of \code{static} variables should
    155 be kept to an absolute minimum and then only when protected by a
    156 ``mutex''.  Cross-thread synchronization for PSLib's fundamental
    157 datatypes (\code{psArray}, \code{psList}, etc.) is left to the end-user.
    158 
    159 PSLib shall provide some functions and structures that assist in
    160 thread-safety, but it is left up to the end-user to employ these.
    161 That is, these conveniences are not used internally by PSLib, and so
    162 the user should make no assumptions about thread-safety (outside the
    163 memory management functions).
     150\subsection{Threads and Re-entrancy}
     151
     152Due to current developments in CPU architecture, we must assume that
     153PSLib will be used in a threaded environment.  However, coding the
     154library to be thread-safe may have implications for the speed of the
     155library and for the simplicity of use.  We therefore make the
     156following policies:
     157
     158\begin{itemize}
     159\item The memory management functions, defined below, must be written
     160  to be thread-safe, since we cannot risk this crucial area being
     161  unstable.
     162\item Re-entrant versions of system calls and external library
     163  functions should be used.  We expect that these cases are
     164  sufficiently small that we are prepared to err on the side of
     165  caution.
     166\item The practise of using \code{static} variable to achieve high
     167  efficiency (e.g., so that subsequent calls do not have to repeat a
     168  large memory allocation) should be kept to an absolute minimum.
     169  Where it has been justified (i.e., through code profiling), the
     170  \code{static} variable must be protected by a ``mutex''.
     171\item Cross-thread synchronization for PSLib's fundamental datatypes
     172  (\code{psArray}, \code{psList}, etc.) is left to the end-user
     173  (although some convenience is provided --- see below).
     174\end{itemize}
     175
     176As a convenience to the user in achieving thread-safe operation, each
     177of the data structures classified as a ``collection'' (i.e.,
     178\code{psList, psHash, psMetadata, psArray, psPixels, psVector,
     179psBitSet}) and \code{psImage} shall contain a member, \code{void
     180*lock}, which provides a place for the user to carry around a mutex or
     181semaphore.  This is provided so that the user doesn't have to pass
     182around both the structure and a mutex, or wrap PSLib structures in
     183their own thread-safe structures that contain a mutex.  PSLib is not
     184responsible for allocating, setting, checking or freeing the
     185\code{lock} --- these are entirely the responsibility of the user.
     186PSLib provides only a place to hang it.  Your mileage may vary.
     187
     188If these policies become a burden on the processing speed, we can
     189investigate alternative measures, such as defining specifically
     190re-entrant versions of select speed-critical functions.
    164191
    165192\subsection{Angles}
     
    343370    struct psMemBlock* previousBlock;   ///< previous block in allocation list
    344371    struct psMemBlock* nextBlock;       ///< next block allocation list
    345     psFreeFunc freeFunc;                 ///< deallocator.  If NULL, use generic deallocation.
     372    psFreeFunc freeFunc;                ///< deallocator.  If NULL, use generic deallocation.
    346373    size_t  userMemorySize;             ///< the size of the user-portion of the memory block
    347374    const psMemId id;                   ///< a unique ID for this allocation
    348375    const char* file;                   ///< set from __FILE__ in e.g. p_psAlloc
    349376    const unsigned int lineno;          ///< set from __LINE__ in e.g. p_psAlloc
    350     pthread_mutex_t   refCounterMutex;  ///< mutex to ensure exclusive access to reference counter
     377    pthread_mutex_t refCounterMutex;    ///< mutex to ensure exclusive access to reference counter
    351378    psReferenceCount refCounter;        ///< how many times pointer is referenced
    352379    bool persistent;                    ///< marks if non-user persistent data like error stack, etc.
     
    787814\code{psStringCopy} or \code{psStringNCopy}).
    788815
     816\subsubsection{Strings}
     817
     818The use of strings within the PSLib memory management system requires
     819that they be allocated with \code{psAlloc}.  This means that
     820substrings cannot be placed on containers such as \code{psArray} or
     821\code{psMetadata}, since these check for the existence of the attached
     822\code{psMemBlock}.  To get around this, we specify functions that copy
     823a string into \code{psAlloc}-ed memory:
     824\begin{prototype}
     825psString psStringCopy(const char *string);
     826psString psStringNCopy(const char *string, unsigned int nChar);
     827\end{prototype}
     828
     829\code{psStringCopy} shall perform a deep copy of the \code{string}
     830into \code{psAlloc}-ed memory.  \code{psStringNCopy} shall do the
     831same, up to a maximum of \code{nChar} characters.
     832
     833We also specify two useful functions:
     834
     835\begin{prototype}
     836ssize_t psStringAppend(char **dest, const char *format, ...)
     837ssize_t psStringPrepend(char **dest, const char *format, ...)
     838\end{prototype}
     839
     840\code{psStringAppend} shall append, according to the \code{format}
     841values into the \code{dest} string.  \code{dest} shall be allocated if
     842\code{NULL}, and the length of the new string (excluding the
     843terminator) shall be returned.  \code{psStringPrepend} shall do
     844similarly, except it shall prepend to the \code{dest} string.
    789845
    790846\subsubsection{Type information}
     
    943999%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    9441000
    945 \subsection{Threads}
    946 
    947 As pointed out earlier, PSLib makes no guarantees for thread-safety
    948 outside of the memory management functions.  Nevertheless, the following
    949 facilities are provided as a convenience to the user.
    950 
    951 Each of the data structures classified as a ``collection'' (i.e.,
    952 \code{psList, psHash, psMetadata, psArray, psPixels, psVector,
    953 psBitSet}) and \code{psImage} shall contain an additional member,
    954 \code{void *lock}, which provides a place for the user to carry around
    955 a mutex or semaphore.  This is provided so that the user doesn't have
    956 to pass around both the structure and a mutex, or wrap PSLib
    957 structures in their own thread-safe structures that contain a mutex.
    958 PSLib is not responsible for freeing the \code{lock} (it may not even
    959 be allocated using the PSLib memory management system) --- it is
    960 entirely the responsibility of the user and PSLib provides only a
    961 place to hang it.
    962 
    963 We also define the following conveniences:
    964 \begin{datatype}
    965 typedef struct {
    966     pthread_mutex_t mutex;
    967 } psMutex;
    968 \end{datatype}
    969 
    970 \begin{prototype}
    971 psMutex *psMutexAlloc(void);
    972 bool psMutexLock(psMutex *mutex);
    973 bool psMutexUnlock(psMutex *mutex);
    974 \end{prototype}
    975 
    976 \code{psMutex} simply wraps a POSIX thread mutex (this is necessary in
    977 order to use the PS memory management system).
    978 
    979 \code{psMutexAlloc} shall allocate memory for a \code{psMutex}, and
    980 initialise the mutex.  \code{psMutexLock} shall lock the mutex in
    981 \code{thread}, and \code{psMutexUnlock} shall unlock the mutex in
    982 \code{thread}.
    983 
    984 These functions, in the interests of speed, should be implemented as
    985 preprocessor macros.
    986 
    987 These functions and the \code{void *lock} in the collection structures
    988 and  \code{psImage} should  only  be defined  if \code{_REENTRANT}  is
    989 defined; otherwise the functions should be poisoned.
     1001%%%
     1002%%% Not sure we need these --- all we do is wrap pthread_mutex, so why bother?
     1003%%%
     1004
     1005%% \subsection{Locks}
     1006%%
     1007%% We define the following conveniences:
     1008%% \begin{datatype}
     1009%% typedef pthread_mutex_t psMutex;
     1010%% \end{datatype}
     1011%%
     1012%% \begin{prototype}
     1013%% psMutex *psMutexAlloc(void);
     1014%% bool psMutexLock(psMutex *mutex);
     1015%% bool psMutexUnlock(psMutex *mutex);
     1016%% \end{prototype}
     1017%%
     1018%% \code{psMutex} simply wraps a POSIX thread mutex (this is necessary in
     1019%% order to use the PS memory management system).
     1020%%
     1021%% \code{psMutexAlloc} shall allocate memory for a \code{psMutex}, and
     1022%% initialise the mutex.  \code{psMutexLock} shall lock the \code{mutex},
     1023%% and \code{psMutexUnlock} shall unlock the \code{mutex}.
     1024%%
     1025%% These functions, in the interests of speed, should be implemented as
     1026%% preprocessor macros.
     1027%%
     1028%% These functions and the \code{void *lock} in the collection structures
     1029%% and  \code{psImage} should  only  be defined  if \code{_REENTRANT}  is
     1030%% defined; otherwise the functions should be poisoned.
    9901031
    9911032%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     
    30953136        psC32 **C32;                    ///< Pointers to complex floating-point data
    30963137        psC64 **C64;                    ///< Pointers to complex floating-point data
    3097         psPtr V;                        ///< Pointers to raw data
     3138        psPtr V;                        ///< Pointers to raw data
    30983139    } data;
    30993140    const struct psImage *parent;       ///< parent, if a subimage
     
    37963837%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    37973838
     3839\subsection{Masks}
     3840
     3841Several functions --- especially the statistical and image functions
     3842--- use a mask vector and a mask value to specify values in an input
     3843list that should be excluded from the calculations.  We define a mask
     3844type, which we will initially set to U8.  In the event that our masks
     3845exceed eight bits, we can then change the mask definition without
     3846major changes throughout the code.
     3847
     3848\begin{datatype}
     3849typedef psU8 psMask;
     3850\end{datatype}
     3851
    37983852\subsection{Statistical Functions}
    37993853
     
    38303884                       const psVector *errors,
    38313885                       const psVector *mask,
    3832                        unsigned int maskVal
     3886                       psMask maskVal
    38333887                       );
    38343888\end{prototype}
     
    38593913clipped statistics are modified according to the ADD; the robust
    38603914median and quartiles are modified as specified in the ADD.  The mask
    3861 must be of type \code{psU8}.
     3915must be of type \code{psMask}.
    38623916
    38633917The \code{psStats} structure is defined with entries for each of the
     
    39744028                               const psVector *errors,
    39754029                               const psVector *mask,
    3976                                unsigned int maskVal);
     4030                               psMask maskVal);
    39774031\end{prototype}
    39784032The \code{values} vector may be of types \code{psU8, psU16, psF32,
     
    40064060typedef struct {
    40074061    psPolynomialType type;              ///< Polynomial type
    4008     psElemType ctype;               ///< Polynomial precision
     4062    psElemType ctype;                   ///< Polynomial precision
    40094063    int n;                              ///< Number of terms
    40104064    union {
     
    40244078typedef struct {
    40254079    psPolynomialType type;              ///< Polynomial type
    4026     psElemType ctype;               ///< Polynomial precision
     4080    psElemType ctype;                   ///< Polynomial precision
    40274081    int nX, nY;                         ///< Number of terms in x and y
    40284082    union {
     
    40344088      psF64 **F64;                      ///< Error in coefficients
    40354089    } coeffErr;
    4036     char *mask;                         ///< Coefficient mask
    40374090    char **mask;                        ///< Coefficients mask
    40384091} psPolynomial2D;
     
    40774130                         psF64 x);
    40784131psF64 psPolynomial2DEval(const psPolynomial2D *poly,
    4079                         psF64 x,
    4080                         psF64 y);               
     4132                        psF64 x,
     4133                        psF64 y);               
    40814134psF64 psPolynomial3DEval(const psPolynomial2D *poly,
    4082                         psF64 x,
    4083                         psF64 y,
    4084                         psF64 z);                 
     4135                        psF64 x,
     4136                        psF64 y,
     4137                        psF64 z);                 
    40854138psF64 psPolynomial4DEval(const psPolynomial2D *poly,
    4086                         psF64 x,
    4087                         psF64 y,
    4088                         psF64 z,
    4089                         psF64 t);
     4139                        psF64 x,
     4140                        psF64 y,
     4141                        psF64 z,
     4142                        psF64 t);
    40904143\end{prototype}
    40914144
     
    42474300bool psMinimizeLMChi2(psMinimization *min,
    42484301                      psImage *covar,
    4249                       psVector *params,
     4302                      psVector *params,
    42504303                      const psVector *paramMask,
    4251                       const psArray  *x,
    4252                       const psVector *y,
     4304                      const psArray  *x,
     4305                      const psVector *y,
    42534306                      const psVector *yErr,
    4254                       psMinimizeLMChi2Func func);
     4307                      psMinimizeLMChi2Func func);
    42554308\end{prototype}
    42564309
     
    42874340identical.
    42884341
    4289 \code{paramMask} must be of type \code{psU8}, while \code{params} must
    4290 be of type \code{psF32}.  The \code{func} function must be valid only
    4291 for types \code{psF32}, \code{psF64}.
     4342\code{paramMask} must be of type \code{psMask}, while \code{params}
     4343must be of type \code{psF32}.  The \code{func} function must be valid
     4344only for types \code{psF32}, \code{psF64}.
    42924345
    42934346\begin{prototype}
     
    43964449\begin{prototype}
    43974450psPolynomial1D *psVectorFitPolynomial1D(psPolynomial1D *poly,
    4398                                         const psVector *mask,
    4399                                         unsigned int    maskValue
    4400                                         const psVector *f,
     4451                                        const psVector *mask,
     4452                                        psMask          maskValue
     4453                                        const psVector *f,
    44014454                                        const psVector *fErr
    44024455                                        const psVector *x);
    44034456psPolynomial2D *psVectorFitPolynomial2D(psPolynomial1D *poly,
    4404                                         const psVector *mask,
    4405                                         unsigned int    maskValue
    4406                                         const psVector *f,
     4457                                        const psVector *mask,
     4458                                        psMask          maskValue
     4459                                        const psVector *f,
    44074460                                        const psVector *fErr
    44084461                                        const psVector *x,
    44094462                                        const psVector *y);
    44104463psPolynomial3D *psVectorFitPolynomial3D(psPolynomial1D *poly,
    4411                                         const psVector *mask,
    4412                                         unsigned int    maskValue
    4413                                         const psVector *f,
     4464                                        const psVector *mask,
     4465                                        psMask          maskValue
     4466                                        const psVector *f,
    44144467                                        const psVector *fErr
    44154468                                        const psVector *x,
     
    44174470                                        const psVector *z);
    44184471psPolynomial4D *psVectorFitPolynomial4D(psPolynomial1D *poly,
    4419                                         const psVector *mask,
    4420                                         unsigned int    maskValue
    4421                                         const psVector *f,
     4472                                        const psVector *mask,
     4473                                        psMask          maskValue
     4474                                        const psVector *f,
    44224475                                        const psVector *fErr
    44234476                                        const psVector *x,
     
    44424495\begin{prototype}
    44434496psPolynomial1D *psVectorClipFitPolynomial1D(psPolynomial1D *poly,
    4444                                         psStats            *stats,
    4445                                         const psVector     *mask,
    4446                                         unsigned int        maskValue
    4447                                         const psVector     *f,
    4448                                         const psVector     *fErr
    4449                                         const psVector     *x);
     4497                                        psStats            *stats,
     4498                                        const psVector     *mask,
     4499                                        psMask              maskValue
     4500                                        const psVector     *f,
     4501                                        const psVector     *fErr
     4502                                        const psVector     *x);
    44504503psPolynomial2D *psVectorClipFitPolynomial2D(psPolynomial1D *poly,
    4451                                         psStats            *stats,
    4452                                         const psVector     *mask,
    4453                                         unsigned int        maskValue
    4454                                         const psVector     *f,
    4455                                         const psVector     *fErr
    4456                                         const psVector     *x,
    4457                                         const psVector     *y);
     4504                                        psStats            *stats,
     4505                                        const psVector     *mask,
     4506                                        psMask              maskValue
     4507                                        const psVector     *f,
     4508                                        const psVector     *fErr
     4509                                        const psVector     *x,
     4510                                        const psVector     *y);
    44584511psPolynomial3D *psVectorClipFitPolynomial3D(psPolynomial1D *poly,
    4459                                         psStats            *stats,
    4460                                         const psVector     *mask,
    4461                                         unsigned int        maskValue
    4462                                         const psVector     *f,
    4463                                         const psVector     *fErr
    4464                                         const psVector     *x,
    4465                                         const psVector     *y,
    4466                                         const psVector     *z);
     4512                                        psStats            *stats,
     4513                                        const psVector     *mask,
     4514                                        psMask              maskValue
     4515                                        const psVector     *f,
     4516                                        const psVector     *fErr
     4517                                        const psVector     *x,
     4518                                        const psVector     *y,
     4519                                        const psVector     *z);
    44674520psPolynomial4D *psVectorClipFitPolynomial4D(psPolynomial1D *poly,
    4468                                         psStats            *stats,
    4469                                         const psVector     *mask,
    4470                                         unsigned int        maskValue
    4471                                         const psVector     *f,
    4472                                         const psVector     *fErr
    4473                                         const psVector     *x,
    4474                                         const psVector     *y,
    4475                                         const psVector     *z,
    4476                                         const psVector     *t);
     4521                                        psStats            *stats,
     4522                                        const psVector     *mask,
     4523                                        psMask              maskValue
     4524                                        const psVector     *f,
     4525                                        const psVector     *fErr
     4526                                        const psVector     *x,
     4527                                        const psVector     *y,
     4528                                        const psVector     *z,
     4529                                        const psVector     *t);
    44774530\end{prototype}
    44784531These functions replicate the capability of the vector fitting
     
    45754628                       const psImage *input,
    45764629                       const psImage *mask,
    4577                        unsigned int maskVal,
     4630                       psMask maskVal,
    45784631                       psRegion region,
    45794632                       psImageCutDirection direction,
     
    46104663                     const psImage *input,
    46114664                     const psImage *mask,
    4612                      unsigned int maskVal,
     4665                     psMask maskVal,
    46134666                     psRegion region,
    46144667                     unsigned int nSamples,
     
    46304683                           const psImage *input,
    46314684                           const psImage *mask,
    4632                            unsigned int maskVal,
     4685                           psMask maskVal,
    46334686                           float x,
    46344687                           float y,
     
    46814734psImage *psImageRebin(psImage *out, const psImage *in,
    46824735                      const psImage *mask,
    4683                       unsigned int maskVal,
     4736                      psMask maskVal,
    46844737                      int scale, const psStats *stats);
    46854738\end{prototype}
     
    47554808                          const psImage *input,
    47564809                          const psImage *inputMask,
    4757                           int inputMaskVal,
     4810                          psMask inputMaskVal,
    47584811                          const psPlaneTransform *outToIn,
    47594812                          psRegion region,
     
    47714824If the \code{inputMask} is non-\code{NULL}, those pixels in the
    47724825\code{inputMask} matching \code{inputMaskVal} are to be ignored in the
    4773 transformation.  The \code{inputMask} must be of type \code{psU8}, and
     4826transformation.  The \code{inputMask} must be of type \code{psMask}, and
    47744827of the same size as the \code{input}, otherwise the function shall
    47754828generate an error and return \code{NULL}.  The transformation
     
    47974850                      const psImage *in,
    47984851                      const psImage *mask,
    4799                       unsigned int maskVal);
     4852                      psMask maskVal);
    48004853\end{prototype}
    48014854Determine statistics for image (or subimage).  The statistics to be
     
    48104863                              const psImage *in,
    48114864                              const psImage *mask,
    4812                               unsigned int maskVal);
     4865                              psMask maskVal);
    48134866\end{prototype}
    48144867Construct a histogram from an image (or subimage).  The histogram to
     
    48384891\begin{prototype}
    48394892complex double psImagePixelInterpolate(const psImage *input, float x, float y,
    4840                               const psImage *mask, unsigned int maskVal,
     4893                              const psImage *mask, psMask maskVal,
    48414894                              complex double unexposedValue, psImageInterpolateMode mode);
    48424895\end{prototype}
     
    49214974
    49224975\begin{prototype}
    4923 psImage *psImageGrowMask(psImage *out, const psImage *in, unsigned int maskVal,
    4924                          unsigned int growSize, unsigned int growValue);
     4976psImage *psImageGrowMask(psImage *out, const psImage *in, psU16 maskVal,
     4977                         unsigned int growSize, psU16 growVal);
    49254978\end{prototype}
    49264979
     
    49354988\code{maskVal} shall have the corresponding pixel in the \code{out}
    49364989image set to the \code{growValue}.  The function must be defined for
    4937 the following types: \code{psU8}, \code{psU16}.
    4938 
    4939 \begin{prototype}
    4940 psImage *psPixelsToMask(psImage *out, const psPixels *pixels, psRegion region, unsigned int maskVal);
    4941 psPixels *psPixelsFromMask(psPixels *out, const psImage *mask, unsigned int maskVal);
    4942 \end{prototype}
    4943 
    4944 \code{psPixelsToMask} shall return an image of type U8 with the
    4945 \code{pixels} lying within the specified \code{region} set to the
     4990the following types: \code{psU8}, \code{psU16}.  Note that
     4991\code{maskVal} and \code{growVal} are not of type \code{psMask}, but
     4992\code{psU16} so that they match the maximum required type for the
     4993\code{in} image.
     4994
     4995\begin{prototype}
     4996psImage *psPixelsToMask(psImage *out, const psPixels *pixels, psRegion region, psMask maskVal);
     4997psPixels *psPixelsFromMask(psPixels *out, const psImage *mask, psMask maskVal);
     4998\end{prototype}
     4999
     5000\code{psPixelsToMask} shall return an image of type \code{psMask} with
     5001the \code{pixels} lying within the specified \code{region} set to the
    49465002\code{maskVal}.  The \code{out} image shall be modified if supplied,
    49475003or allocated and returned if \code{NULL}.  The size of the output
Note: See TracChangeset for help on using the changeset viewer.