Changeset 40026

Timestamp:

May 8, 2017, 11:06:39 AM (9 years ago)

Author:

eugene

Message:

more edits

File:

• trunk/doc/release.2015/ps1.datasystem/datasystem.tex (modified) (8 diffs)

trunk/doc/release.2015/ps1.datasystem/datasystem.tex

@@ -1111 +1111 @@
 \label{sec:postprocessing}
 
+\begin{table}[hb]
+\begin{center}
+\caption{DVO Database Tables\label{tab:DVOtables}}
+\begin{tabular}{ll}
+\hline
+\hline
+{\bf Table Name} & {\bf Description} \\
+\hline
+Images               & The images that have objects in the DB. \\
+Image Overlaps       & Image regions which are touched by specific images. \\
+Objects              & The objects --- average properties of multiple detections of the same object. \\
+Average Magnitudes   & Average photometry in multiple filters \\
+Solar System Objects & Identification of solar system objects \\
+Matched Detections   & Detections of sources in an image identified with an Object. \\
+Orphaned Detections  & Detections of sources in an image not identified with an Object. \\
+Non-detections       & Non-detections of objects in an image. \\
+SkyRegions           & spatial distribution of tables \\
+Filters              & Filters understood by the system. \\
+Photcodes            & Transformations between different photometric systems \\
+Zero Points          & History of Zero-point \& Airmass terms \\
+Distortion Models    & History of Optical Distortion terms \\
+Database Hosts       & computers used to store the tables \\
+\hline
+\end{tabular}
+\end{center}
+\end{table}
+
 \begin{verbatim}
 DVO section outline or list of topics:
@@ -1132 +1159 @@
 \label{sec:DVO}
 
+\subsubsection{Overview}
+
+% intro
 The Pan-STARRS IPP uses an internal database system, distinct from the
 publically visible database system, to determine the association
@@ -1142 +1172 @@
 context.  
 
+% overview
+DVO tracks three main classes of information: 1) properties of
+astronomical objects; 2) measurements of those objects (from which the
+properties are derived); 3) properties of image which provided some or
+all of the measuements.  Figure~\ref{fig:DVO_schema} illustrates the
+schematic relationship between these types of measurements.  
+
+In the most basic implementation, a collection of measurements from a
+set of images are loaded into DVO along with the metadata describing
+the images.  The latter includes properties such as the exposure time,
+airmass, filter, time \& date of the exposure, etc.  Critically, the
+image metadata includes an astrometric transformation relating the
+detection coordinate on the image to the coordinate on the sky.  As
+the collection of measurements are loaded into DVO, the software
+constructs astronomical objects based on those detections.  If
+images overlapped, multiple observations of the same astronomical
+object are grouped together.  Thus, a single DVO database will contain
+a one-to-many relationship between the images and the measurements and
+a many-to-one relationship between the measurements and the derived
+astronomical objects.
+
+Table~\ref{tab:DVO_schema} lists the full collection of database
+tables used by DVO.  These tables fall into one of several classes:
+those which store information about the average properties of
+astronomical objects; those which store information about individual
+measurements; those which store information about the images; those
+which store supporting information.
+
+Beyond that basic use, DVO has the ability to accept data from other
+kinds of data sources in which measurements are not clearly associated
+with specific images.  DVO ingest methods are defined for several
+large-scale surveys for which the published data represent average
+properties derived from multiple measurements, and for which the
+measurement-to-image relationship is not provided.  Ingets methods
+have been defined for example for 2MASS, WISE, Gaia, USNO-B.  In each
+of these cases, the astrometric and photometric measurements are
+stored in the \table{Measure} table, with the data source identified
+by the photcode of the measurement.
+
+% object -> detection
 One of the main purposes of DVO is to define the relationship between
 individual detections of an astronomical object and the definition of
@@ -1152 +1222 @@
 database, the detection is associated with the closest object.
 
+% photcodes
 Detections in DVO have a special piece of metadata called the
 \ippdbcolumn{photcode} which identifies the source of the measurement.
@@ -1174 +1245 @@
 photcodes.
 
+% FITS table + compression
 In the implementation of DVO used for the PV3 calibration analysis,
 the database tables are stored on disk using binary FITS tables.  Each
@@ -1183 +1255 @@
 volume (70TB compressed) drove the decision to compress the tables.
 
+% FITS table compression details
 The FITS binary table compression scheme uses a strategy similar to
 that used for FITS image compression (\note{REF}).  The binary tabular
@@ -1196 +1269 @@
 TTYPE1) and units (e.g., TUNIT1) are retained in their original form.
 
+% FITS table compression details
 The compression algorithm can treat the entire column as a single
 block of data, or it may be broken into a number of chunks, each
@@ -1262 +1336 @@
   and the calibration paper?}
 
+% parallel partitions
 The DVO software system allows the tables which are partitioned across
 the sky to also be distributed across multiple computers, which we