Changeset 4208

Timestamp:

Jun 9, 2005, 5:56:03 PM (21 years ago)

Author:

Paul Price

Message:

General tidy up through to section ~ 2. Still need to check psString etc.

Location:

trunk/doc/pslib

Files:

• ChangeLogSDRS.tex (modified) (2 diffs)
• psLibSDRS.tex (modified) (19 diffs)

trunk/doc/pslib/ChangeLogSDRS.tex

@@ -1 @@
-%%% $Id: ChangeLogSDRS.tex,v 1.134 2005-06-09 22:21:06 jhoblitt Exp $
+%%% $Id: ChangeLogSDRS.tex,v 1.135 2005-06-10 03:56:03 price Exp $
 
 \subsection{Changes from version 00 to version 01}
@@ -695 +695 @@
 \item moved image hierarchy section to ModulesSDRS
 \item moved photometry section to ModulesSDRS
+\item Updating section on thread safety, and added \code{psMutex}.
 \item add \code{psFitsOpenFD()}
-
-\end{itemize}
+\end{itemize}

trunk/doc/pslib/psLibSDRS.tex

@@ -1 @@
-%%% $Id: psLibSDRS.tex,v 1.273 2005-06-09 22:21:06 jhoblitt Exp $
+%%% $Id: psLibSDRS.tex,v 1.274 2005-06-10 03:56:03 price Exp $
 \documentclass[panstarrs,spec]{panstarrs}
 
@@ -127 +127 @@
 Library (GSL):
 
-\href{www.gnu.org/software/gsl/}{\tt www.gnu.org/software/gsl});
+\href{www.gnu.org/software/gsl/}{\tt www.gnu.org/software/gsl};
 
 \item The sort functions should wrap the system \code{qsort} call
@@ -145 +145 @@
 fashion in the Pilot Project.  Also, RHL has provided functional
 prototype code for the memory management, tracing and message logging
-functions (some of which need to be revised), and the data containers
-(doubly-linked lists, hashes, arrays).
+functions (some of which need to be revised), and some of the data
+containers (doubly-linked lists, hashes, arrays).
 
 \subsection{Threads}
 
-Pan-STARRS does not have a strong requirement for multithreading capability.
-The memory management functions, defined below, must be written to be
-thread-safe.  The use of \code{static} variables should be keep to an absolute
-minimum and then only when protected by a ``mutex''.  Cross-thread
-synchronization for PSLib's fundamental datatypes (\code{psArray},
-\code{psVector}, etc.) is left as an exercise for the end-user.
+Pan-STARRS does not have a strong requirement for multithreading
+capability.  The memory management functions, defined below, must be
+written to be thread-safe.  The use of \code{static} variables should
+be kept to an absolute minimum and then only when protected by a
+``mutex''.  Cross-thread synchronization for PSLib's fundamental
+datatypes (\code{psArray}, \code{psList}, etc.) is left to the end-user.
+
+PSLib shall provide some functions and structures that assist in
+thread-safety, but it is left up to the end-user to employ these.
+That is, these conveniences are not used internally by PSLib, and so
+the user should make no assumptions about thread-safety (outside the
+memory management functions).
 
 \subsection{Conventions}
@@ -204 +210 @@
 \subsection{Initialization}
 
-PSLib shall be initialized by calling \code{psLibInit} by the user
+PSLib may be initialized by calling \code{psLibInit} by the user
 before use.  This enables the library to perform the following tasks
 that are required before general use:
@@ -219 +225 @@
 
 \code{timeConfig} specifies the filename of the configuration file
-for \code{psTime} metadata
+for \code{psTime} metadata.
 
 \subsection{Finalization}
@@ -228 +234 @@
 
 \code{psLibCleanup} shall free memory that was allocated by
-\code{psLibInit}, allowing a subsequent search for leaked memory.
+\code{psLibInit}, allowing a subsequent search for leaked memory (even
+memory marked as \code{persistent}).
 
 \subsection{Memory Management}
@@ -272 +279 @@
 \item
   While debugging complex scientific code, it is very useful to be
-  able to trace a given data structure as it passes through the
+  able to trace a specific data structure as it passes through the
   processing pipeline.
 
@@ -359 +366 @@
 list.
 
-The element \code{freeFunc} specifies the deallocator associated with a
-specific block of memory.  If this element is \code{NULL}, the basic
-deallocator is used and the memory block must not be a rich data
-structure which requires additional freeing functionality.
+The element \code{freeFunc} specifies the deallocator associated with
+a specific block of memory.  If this element is \code{NULL}, the basic
+deallocator (\code{psFree}) is used and the memory block must not be a
+rich data structure which requires additional freeing functionality
+(otherwise memory leaks will occur).
 
 The element \code{id} in the structure is a sequential memory block
@@ -384 +392 @@
 \code{persistent} to be ignored when identifing memory leaks with
 \code{psMemCheckLeaks}.  Private functions shall be provided to test
-and set the value of \code{persistent}.
+and set the value of \code{persistent}:
+\begin{prototype}
+void p_psMemSetPersistent(psPtr ptr, bool value);
+bool p_psMemGetPersistent(psPtr ptr);
+\end{prototype}
+
 
 The \code{psMemBlock} structure element \code{refCounter} is provided
@@ -417 +430 @@
 components that employ any \code{persistent} memory shall provide a
 function that frees all of its \code{persistent} memory (and only that
-persistent memory that belongs to that component).
+persistent memory that belongs to that component), all of which should
+be called from \code{psLibCleanup}.
 
 \subsubsection{APIs for Allocating and Freeing}
@@ -768 +782 @@
 points to a data block with multiple references, this call would only
 decrement the counter.  
+
+\subsection{Conventions adopted}
+
+Only pointers allocated with the PSLib memory functions are compatible
+with the various PSLib container types (e.g., \code{psList,
+psMetadata}), because the functions working with the container types
+search for the attached \code{psMemBlock}.  If a pointer allocated
+with another memory system (e.g., the system \code{malloc}), or
+generated by offsetting from another pointer that was allocated with
+\code{psAlloc}, is used with PSLib, the PSLib functions would falsely
+determine that memory is corrupted, because of the missing
+\code{psMemBlock}.
+
+To pilot our way through the potential confusion, instead of calling
+all pointers (of unspecified type) a ``\code{void*}'', we adopt a
+convention, both in this document and in the source, of referring to a
+pointer that has a \code{psMemBlock} attached as a \code{psPtr}:
+
+\begin{datatype}
+typedef void* psPtr;
+\end{dataType}
+
+For the same reason, we also adopt a convention of referring to a string
+that has a \code{psMemBlock} attached as a \code{psString}:
+
+\begin{datatype}
+typedef char* psString;
+\end{datatype}
+
+That is, \code{psPtr} is used in cases where the function requires the
+use of a pointer of unspecified type allocated with \code{psAlloc},
+and \code{psString} is used in cases where the function requires the
+use of a \code{char*} allocated with \code{psAlloc} (e.g., via
+\code{psStringCopy} or \code{psStringNCopy}).
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{Threads}
+
+As pointed out earlier, PSLib makes no guarantees for thread-safety
+outside of the memory management functions.  Nevertheless, the following
+facilities are provided as a convenience to the user.
+
+Each of the data structures classified as a ``collection'' (i.e.,
+\code{psList, psHash, psMetadata, psArray, psPixels, psVector,
+psBitSet}) and \code{psImage} shall contain an additional member,
+\code{void *lock}, which provides a place for the user to carry around
+a mutex or semaphore.  This is provided so that the user doesn't have
+to pass around both the structure and a mutex, or wrap PSLib
+structures in their own thread-safe structures that contain a mutex.
+
+We also define the following conveniences:
+\begin{datatype}
+typedef struct {
+    pthread_mutex_t mutex;
+} psMutex;
+\end{datatype}
+
+\begin{prototype}
+psMutex *psMutexAlloc(void);
+bool psMutexLock(psMutex *mutex);
+bool psMutexUnlock(psMutex *mutex);
+\end{prototype}
+
+\code{psMutex} simply wraps a POSIX thread mutex (this is necessary in
+order to use the PS memory management system).
+
+\code{psMutexAlloc} shall allocate memory for a \code{psMutex}, and
+initialise the mutex.  \code{psMutexLock} shall lock the mutex in
+\code{thread}, and \code{psMutexUnlock} shall unlock the mutex in
+\code{thread}.  No distinction is made between read and write locks.
+
+These functions, in the interests of speed, should be implemented as
+preprocessor macros.
+
+These functions should only be defined if \code{_REENTRANT} is
+defined, and poisoned otherwise.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \subsection{Tracing and Logging}
@@ -1511 +1604 @@
         psC64 *C64;                     ///< Pointers to complex floating-point data
     } data;
+    void *lock;                         ///< Lock for thread safety
 } psVector;
 \end{datatype}
@@ -1625 +1719 @@
     const struct psImage *parent;       ///< parent, if a subimage 
     psArray *children;                  ///< children of this region
+    void *lock;                         ///< Lock for thread safety
 } psImage;
 \end{datatype}
@@ -1756 +1851 @@
     const psU32 nalloc;         // Number allocated
     psPixelCoord *data;         // The pixel coordinates
+    void *lock;                         ///< Lock for thread safety
 } psPixels;
 \end{datatype}
@@ -1833 +1929 @@
     const long nalloc;                  ///< allocated data block
     void **data;                        ///< pointer to data block
-} psArray;
+    void *lock;                         ///< Optional lock for thread safety
+}} psArray;
 \end{datatype}
 %
@@ -1918 +2015 @@
    psListElem *tail;                    ///< last element on list (may be NULL)
    psArray *iterators;                  ///< array of psListIterator: iteration cursors
+   void *lock;                          ///< Optional lock for thread safety
 } psList;
 \end{datatype}
@@ -2126 +2224 @@
     long n;                             ///< number of buckets
     psHashBucket **buckets;             ///< the buckets themselves
-} psHash;
+    void *lock;                         ///< Optional lock for thread safety
+}} psHash;
 \end{datatype}
 %
@@ -2334 +2433 @@
     long n;                             ///< Number of chars that form the bitset
     char *bits;                         ///< The bits
-} psBitSet;
+    void *lock;                         ///< Optional lock for thread safety
+}} psBitSet;
 \end{datatype}
 
@@ -2561 +2661 @@
     psList *list;                       ///< list of psMetadataItem
     psHash *hash;                       ///< hash table of the same metadata
-} psMetadata;
+    void *lock;                         ///< Optional lock for thread safety
+}} psMetadata;
 \end{datatype}
 The type \code{psMetadata} is a container class for metadata. Note