Changeset 27840 for branches/simtest_nebulous_branches/doc/stamps/stampsICD.tex

Timestamp:

May 3, 2010, 8:50:52 AM (16 years ago)

Author:

eugene

Message:

updates from trunk

Location:

branches/simtest_nebulous_branches

Files:

• . (modified) (1 prop)
• doc/stamps/stampsICD.tex (modified) (15 diffs)

branches/simtest_nebulous_branches/doc/stamps/stampsICD.tex

@@ -76 +76 @@
 \end{table}
 
-The IPP will be configured to periodicly query a set of Data Stores for new request file sets.
-We will refer to each Data Store as a \emph{requestor}.
-Each request table submitted by a given requstor must have a unique value for 
-the keyword \code{REQ_NAME}. This value is used for the name of the request's output file set.
+The IPP will be configured to periodically query a set of Data Stores for new request file sets.
+We will refer to each Data Store as a \emph{requester}.
+Each request table submitted must have a unique value for the keyword \code{REQ_NAME}.
+This value is used for the name of the request's output file set.
 
 The postage stamp server supports three types of "postage stamp" job specified by the value
 the column \code{JOB_TYPE}. If the value is \emph{stamp} postage stamp images will be extracted
-from the images of interest.  If the value of \code{JOB_TYPE} is \emph{get\_image} a copy
-of the selected image(s) with (satelite streaks removed) will be placed on the output file set.
-Finally if the \code{JOB_TYPE} is \emph{list} the results file will list the input selected images
-but no postage stamps will be made.
+from the images of interest.  If the value of \code{JOB_TYPE} is \emph{get\_image} 
+the data from the selected images' "runs"  with (satellite streaks removed) are bundled
+in and placed on the output file set.
 
 \begin{table}[h]
@@ -97 +96 @@
 \hline
 \code{ROWNUM}   & U32       &                           &        must be unique in file\\
-\code{PROJECT}  & string    & TBD                       &        \\
+\code{PROJECT}  & string    & gpc1                      &        \\
 \code{JOB_TYPE} & string    & stamp or get\_image       &        \\
 \\
-\code{STAMP_NAME}  & string & letters, numbers, '.',  '\-' and '\_'  &        \\
-\code{OPTION_MASK} & U32    & \code{[0-5]} see text     &      \\ 
-\\
-\code{REQ_TYPE} & string    & bycoord, byexp, or byid   &        \\
+\code{OPTION_MASK} & U32    & see text                  &      \\ 
+\\
+\code{REQ_TYPE} & string    & bycoord, byexp, byid, bydiff, or byskycell   &        \\
 \code{IMG_TYPE} & string    & raw, chip, warp, diff, or stack  & \\
-\code{ID} & string          & depends on project and REQ\_TYPE& \\
-\code{CLASS_ID} & string    & see text                  &        not null for byid raw \& byexp raw \\
+\code{ID} & string          & depends on REQ\_TYPE& \\
+\code{TESS_ID}  & string    & see text                  &        \\
+\code{COMPONENT} & string   & see text                  &        \\
 \\
 \code{COORD_MASK} &  U32    & \code{[0-3]} see text     &        not null if \code{JOB_TYPE} = stamp \\
@@ -114 +113 @@
 \code{HEIGHT}   &  F64      & see text                  &        not null if \code{JOB_TYPE} = stamp \\
 \\
-\code{REQFILT}  &   string  & any                       &        used only for \code{REQ_TYPE} = bycoord \\
-\code{MJD_MIN}  &   F64     & Modified Julian Day       &        used only for \code{REQ_TYPE} = bycoord \\
-\code{MJD_MAX}  &   F64     &                           &        used only for \code{REQ_TYPE} = bycoord \\
-\\
+\code{DATA_GROUP} &   string  & any                      & \\
+\code{REQFILT}  &   string  & any                       & \\
+\code{MJD_MIN}  &   F64     & Modified Julian Day       & \\
+\code{MJD_MAX}  &   F64     &                           & \\
+\\
+\code{COMMENT}  &   string  & copied to the results flie & \\
 \hline
 \end{tabular}
@@ -130 +131 @@
 \subsubsection{Request Types}
 The method used to select the images of interest is determined by the value of the REQ\_TYPE column.
-There are three alllowed values. 
+There are three allowed values. 
 
 \begin{itemize}
 
-\item{\textbf{bycoord}} The center of the region of interest is specifed in celestial coordinates.
+\item{\textbf{bycoord}} The center of the region of interest is specified in celestial coordinates.
 Images are selected by querying the project's DVO database for images with the provided
 IMG\_TYPE that contain the center.
 The set of images of interest may be limited by specifying certain criteria as described
-in Section \ref{ImageFilters}.
+in Section \ref{ImageCuts}. NOTE: this request type is not yet implemented.
 
 \item{\textbf{byexp}}  The image of interest is the image resulting from a particular exposure 
@@ -146 +147 @@
 The ID parameter refers to IPP image database key that corresponds to the IMG\_TYPE. 
 e.g: either exp\_id, chip\_id, warp\_id, diff\_id, or stack\_id.
+
+\item{\textbf{bydiff}}  The image with given IMG\_TYPE corresponding to the difference image
+whose IMAGEID is equal to ID. This type is useful for requesting stamps based on difference detections.
+
+\item{\textbf{byskycell}}  Images are selected from the skycell with skycell\_id equal to the
+value for COMPONENT from the tessellation identified by TESS\_ID. This request type may be
+used only with IMG\_TYPE warp, stack, and diff.
+The set of images may be limited by filter and date as described in Section \ref{ImageCuts}.
+NOTE: this request type is not yet implemented.
+
 \end{itemize}
 
-CLASS\_ID must also be supplied for request specifications where all of the following are true
+Note that at this time the REQ\_TYPE byexp is not implemented for IMG\_TYPE stack.
+
+COMPONENT must also be supplied for request specifications where all of the following are true
 
 \begin{itemize}
@@ -158 +171 @@
 \item REQ\_TYPE=byexp or REQ\_TYPE=byid 
 
-\item IMG\_TYPE=chip or IMG\_TYPE=raw
-
 \end{itemize}
 
-
-\subsubsection{Image Selection Filters}\label{ImageFilters}
-
-A request for given celestial coordinates may refer to many images. The scope of the search can be
+The special value 'all' may be used to select all component images but this is not recommended.
+
+\subsubsection{Image Selection Cuts}\label{ImageCuts}
+
+A request specification may refer to many images. The scope of the search can be
 limited to certain dates with the parameters MJD\_MIN and MJD\_MAX. The specified range is inclusive.
+Note: these cuts have no effect for stack stage requests.
+
+DATA\_GROUP restricts the search to runs with the given value. Note that the SQL database lookups
+are done with a "LIKE" comparison so SQL wild cards may be used.
 
 The REQFILT parameter may be used to restrict the search for images taken with a given filter.
-XXX: What is the namespace for filter names.  For example Megacam has filter names like U.9301 which
-IPP maps to u.  Which name do we allow? Probably should used the mapped names g r i etc.
+If the filter is a single character (g, r, i, z, y) the server appends a '\%' character to the
+sql query.
+
 
 \subsection{The Region of Interest}
@@ -177 +194 @@
 a center,  width, and height.
 
-The center is defined by the columns CENTER\_X and CENTER\_Y and the dimenions of the rectangle are
+The center is defined by the columns CENTER\_X and CENTER\_Y and the dimensions of the rectangle are
 the values of WIDTH and HEIGHT. The two least signficant bits in value for the column COORD\_MASK
 determine the type for the coordinates
@@ -189 +206 @@
 {\bf COORD\_MASK value} &  {\bf Description } \\
 \hline
-0                       &   center in RA/DEC; width \& height in arcseconds\\
-1                       &   center in pixel coordinates; width and height in arcseconds\\
+0                       &   center in RA/DEC; width \& height in arc seconds\\
+1                       &   center in pixel coordinates; width and height in arc seconds\\
 2                       &   center in RA/DEC; width and height in pixels\\
 3                       &   center x/y, width, and height in pixel coordinates\\
@@ -198 +215 @@
 \end{table}
 
-Another way to describe this is that COORD\_MASK is two bit mask that bitmask that
+Another way to describe this is that COORD\_MASK is two bit mask that bit mask that
 determines the format of the ROI values. In the C language:
 
@@ -212 +229 @@
 
 When pixel coordinates are used the coordinate system is that of the image. (For chip level images, before
-the stamps are created the cells are mosaiced into a single image.) 
-
-Any fractional parts in pixel coordinate values are ignored.
-
-The value of \code{OPTION_MASK} is used to select various other options. It is a bitwise or of the following
-values.
+the stamps are created the cells are mosaicked into a single image.) 
+
+Any fractional parts in pixel coordinate values are ignored. (XXX: is that true?)
+
+For JOB\_TYPE stamp the value of \code{OPTION\_MASK} is used to select various other options.
+It is a bitwise OR of the following values.
 
 \begin{table}[h]
@@ -229 +246 @@
 1                       &   create postage stamp of the image pixels \\
 2                       &   create postage stamp of the mask pixels \\
-4                       &   create postage stamp of the weight pixels \\
-8                       &   convert WCS to approximate linear transformation in the stamp images\\
-\hline
-\end{tabular}
-\end{center}
-\end{table}
+4                       &   create postage stamp of the variance pixels \\
+8                       &   return the sources file (.cmf) for the images' run \\
+16                      &   return the psf file for the images' run \\
+32                      &   return the background model file for the images chipRun (applies only for chip) \\
+1024                    &   make stamps from inverse difference images (applies only for diff) \\
+2048                    &   make stamps from unconvolved stack images (applies only for stack) \\
+\hline
+\end{tabular}
+\end{center}
+\end{table}
+
+If a bit is set in the OPTION\_MASK that doesn't apply to a particular IMG\_TYPE it is silently ignored.
+
+Note that bit zero is also currently ignored. A stamp of the image pixels is always included in the
+results.
+
+For requests with IMG\_TYPE diff or if \code{REQ_TYPE} is bydiff and 
+the corresponding difference run is a 'bothways' diff the images are selected based on the inverse difference
+images if \code{(OPTION_MASK & 1024)} is non-zero. Otherwise from the images are selected based
+on the 'positive' images.
 
 
@@ -293 +324 @@
 \code{REQ_TYPE} & string    & \\
 \code{IMG_TYPE} & string    & \\
-\code{CLASS_ID} & string    & \\
+\code{TESS_ID} & string    & \\
+\code{COMPONENT} & string    & \\
 \\
 \code{COORD_MASK} &  U32    & \\
@@ -304 +336 @@
 \code{MJD_MIN}  &   F64     & \\
 \code{MJD_MAX}  &   F64     & \\
+\code{DATA_GROUP}  &   string     & \\
+\code{COMMENT}  &   string     & \\
 \\
 \hline
@@ -317 +351 @@
 IMG\_NAME give the name of the image in the Data Store file set.
 
-If no STAMP\_NAME is provided in the request specifiction the image file names will be in the form
-\code{ROWNUM_I.fits}  where I is an integer that ranges from 1 to the number of jobs that the request
-specification generated. Mask images will be named \code{ROWNUM_I.mk.fits} and weight images will
+The image file names will be in the form \code{ROWNUM_I.fits}  where I is an integer that ranges from 1
+to the number of jobs generated for the row's request
+specification.
+
+Mask images will be named \code{ROWNUM_I.mk.fits} and variance images will
 have the name \code{ROWNUM_I.mk.fits}
 
-If STAMP\_NAME is provided in the resquest specification, the name of the image, mask, and weight files
-will be 
-\begin{itemize}
-\item\code{ROWNUM_I_STAMP_NAME.fits}
-\item\code{ROWNUM_I_STAMP_NAME.mk.fits},
-\item\code{ROWNUM_I_STAMP_NAME.wt.fits}
-\end{itemize}
-
-JOB\_ID is the value of the postage stamp server's internal job id. This is provided primarily as a debugging
-aid.
+JOB\_ID is the value of the postage stamp server's internal job id. (This is provided primarily as a debugging
+aid.)
 
 The column named PROJECT and following columns are copies from the input request specification.
 
-If ERROR\_CODE is non-zero one or more errors occured processing the request specification.
-The value of ERROR\_CODE is a bitwise or of the values given in the table\ref{Rerrorcodes}.
+If ERROR\_CODE is non-zero an error occurred when processing the request specification or the job.
+The values for ERROR\_CODE are listed in table\ref{Rerrorcodes}.
 
 
@@ -348 +376 @@
 {\bf Name} & {\bf ERROR\_CODE value} &  {\bf Description } \\
 \hline
-NO\_ERROR            &   0   &   \\
-NO\_MATCHING\_IMAGE   &   1   & No image matches the request specification \\
-RA\_DEC\_NOT\_ON\_PIXEL &   2   & The central RA, DEC are on a blind area (for example a inter-chip gap) \\
-RA\_DEC\_NOT\_IN\_FOV   &   4   & Wrong position on sky for given MJD \\
-NO\_IMAGE\_FOR\_FILTER &   8   & Image is not in the filter requested \\
-NOT\_AVAILABLE\_PERM  &   16  & Image matched but is permanantly unavailable\\
-NOT\_AVAILABLE\_TEMP  &   32  & Image matched but is currently unaccessible\\
-INVALID\_REQUEST     &   64  & request specification was invalid \\
-SYSTEM\_ERROR        &   128 & system error \\
-\hline
-\end{tabular}
-\end{center}
-\end{table}
-
-In the case of an error parsing the request table a file containing a textual description of the errror
-may be added to the resulting file set.
+PSTAMP\_SUCCESS           & 0  &    No errors \\
+PSTAMP\_SYSTEM\_ERROR     & 10 &    some unspecified system error occurred during processing\\
+PSTAMP\_NOT\_IMPLEMENTED  & 11 &    Feature not yet implemented \\
+PSTAMP\_UNKNOWN\_ERROR    & 12 &    Unknown error \\
+PSTAMP\_DUP\_REQUEST      & 20 &    Request name is a duplicate \\
+PSTAMP\_INVALID\_REQUEST  & 21 &    Error in a request specification. See parse\_error.txt \\
+PSTAMP\_UNKNOWN\_PRODUCT  & 22 &    Unknown product in request specification \\
+PSTAMP\_NO\_IMAGE\_MATCH  & 23 &   No images matched the request \\
+PSTAMP\_NOT\_DESTREAKED   & 24 &    Image matched, but not yet de-streaked \\
+PSTAMP\_NOT\_AVAILABLE    & 25 &    Image not available (temporary) \\
+PSTAMP\_GONE              & 26 &     Image is no longer available \\
+PSTAMP\_NO\_JOBS\_QUEUED  & 27 &   Request specification yielded no jobs. See parse\_error.txt \\
+PSTAMP\_NO\_OVERLAP       & 28 &   Center not contained in any image of interest \\
+
+\hline
+\end{tabular}
+\end{center}
+\end{table}
+
+If parsing of the request specification file results in error output, the contents will be placed
+in the file 'parse\_error.txt' in the resulting file set.
 
 \end{document}