IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Ignore:
Timestamp:
Jan 26, 2019, 12:26:08 PM (7 years ago)
Author:
eugene
Message:

submitted to arxiv

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/release.2015/ps1.datasystem/datasystem.tex

    r40612 r40613  
    1 % \documentclass[iop,floatfix]{emulateapj}
     1% \documentclass[preprint2]{emulateapj} % works for 2-column
     2\documentclass[iop,floatfix]{emulateapj}
    23% \documentclass[iop,floatfix,onecolumn]{emulateapj}
    34% \documentclass[12pt,preprint]{aastex}
    4 \documentclass[10pt,preprint]{aastex}
     5% \documentclass[10pt,preprint]{aastex} % use for 1-column
     6% \documentclass[preprint]{aastex}
    57% \pdfoutput=1
    68
    79%\RequirePackage{deluxetable} -- included by aastex?
    8 \RequirePackage{nsfprop}
     10%\RequirePackage{nsfprop} % defines \subsubsubsection but breaks 2-col
    911\RequirePackage{color}
    1012\RequirePackage{code}
     
    12261228entry as such.
    12271229
    1228 \begin{table}[hb]
     1230\section{Post-Processing : Database Ingest and Calibration}
     1231\label{sec:postprocessing}
     1232
     1233\subsection{DVO}
     1234\label{sec:DVO}
     1235
     1236\subsubsection{Overview}
     1237
     1238% intro
     1239The Pan-STARRS IPP uses an internal database system, distinct from the
     1240publicly visible database system, to determine the association
     1241between multiple detections of the same astronomical object and as
     1242part of the astrometric and photometric calibration process.  This
     1243database system, called the ``Desktop Virtual Observatory'' (DVO) was
     1244developed originally for the LONEOS project
     1245\citep{1995DPS....27.0110B}, and used as part of the CFHT Elixir
     1246system \citep{2004PASP..116..449M}.  The capabilities of this
     1247databasing system have been somewhat expanded for the Pan-STARRS
     1248context.
     1249
     1250% overview
     1251DVO tracks three main classes of information: 1) average properties of
     1252astronomical objects; 2) measurements of those objects (from which the
     1253average properties are derived); 3) properties of the images which
     1254provided some or all of the measurements.  In addition, certain
     1255metadata tables define general features of the database.
     1256Table~\ref{tab:DVO_schema} lists the full collection of database
     1257tables used by DVO.
     1258
     1259%Figure~\ref{fig:DVO_schema}
     1260%illustrates the schematic relationship between these types of
     1261%measurements.
     1262
     1263In the most basic implementation, a collection of measurements for
     1264detections from a set of images is loaded into DVO along with the
     1265metadata describing the images.  The latter includes properties such
     1266as the exposure time, airmass, filter, time \& date of the exposure,
     1267etc.  Critically, the image metadata includes an astrometric
     1268transformation relating the detection coordinate on the image to the
     1269coordinate on the sky.  As the collection of measurements are loaded
     1270into DVO, the software constructs astronomical objects based on those
     1271detections.  If images overlap, multiple observations of the same
     1272astronomical object are grouped together.  Thus, a single DVO database
     1273will contain a one-to-many relationship between the images and the
     1274measurements and a many-to-one relationship between the measurements
     1275and the derived astronomical objects.
     1276
     1277%
     1278%% These tables fall into one of several classes:
     1279%% those which store information about the average properties of
     1280%% astronomical objects; those which store information about individual
     1281%% measurements; those which store information about the images; those
     1282%% which store supporting information (metadata).
     1283
     1284%% DVO includes two major classes of database tables: those containing
     1285%% information about astronomical objects in the sky and those containing
     1286%% other supporting information.  The object-related tables are
     1287%% partitioned on the basis of position in the sky: objects within a
     1288%% region bounded by lines of constant RA,DEC are contained in a specific
     1289%% file.  The boundaries and the associated partition names are stored in
     1290%% one of the supporting tables, \ippdbtable{SkyTable}.  This table
     1291%% contains the definitions of the boundaries for each sky region
     1292%% (\ippdbcolumn{R_MIN}, \ippdbcolumn{R_MAX}, \ippdbcolumn{D_MIN},
     1293%% \ippdbcolumn{D_MAX}), the name of the sky region, an ID
     1294%% (\ippdbcolumn{INDEX}, equal to the sequence number of the region in
     1295%% the table), and index entries to enable navigation within the table.
     1296%% The regions are defined in a hierarchical sense, with a series of
     1297%% levels each containing a finer mesh of regions covering the sky.
     1298
     1299\subsubsection{DVO Schema}
     1300
     1301\begin{table*}[hb]
    12291302\begin{center}
    12301303\caption{DVO Database Tables\label{tab:DVO_schema}}
     
    12481321\end{tabular}
    12491322\end{center}
    1250 \end{table}
    1251 
    1252 \section{Post-Processing : Database Ingest and Calibration}
    1253 \label{sec:postprocessing}
    1254 
    1255 \subsection{DVO}
    1256 \label{sec:DVO}
    1257 
    1258 \subsubsection{Overview}
    1259 
    1260 % intro
    1261 The Pan-STARRS IPP uses an internal database system, distinct from the
    1262 publicly visible database system, to determine the association
    1263 between multiple detections of the same astronomical object and as
    1264 part of the astrometric and photometric calibration process.  This
    1265 database system, called the ``Desktop Virtual Observatory'' (DVO) was
    1266 developed originally for the LONEOS project
    1267 \citep{1995DPS....27.0110B}, and used as part of the CFHT Elixir
    1268 system \citep{2004PASP..116..449M}.  The capabilities of this
    1269 databasing system have been somewhat expanded for the Pan-STARRS
    1270 context.
    1271 
    1272 % overview
    1273 DVO tracks three main classes of information: 1) average properties of
    1274 astronomical objects; 2) measurements of those objects (from which the
    1275 average properties are derived); 3) properties of the images which
    1276 provided some or all of the measurements.  In addition, certain
    1277 metadata tables define general features of the database.
    1278 Table~\ref{tab:DVO_schema} lists the full collection of database
    1279 tables used by DVO.
    1280 
    1281 %Figure~\ref{fig:DVO_schema}
    1282 %illustrates the schematic relationship between these types of
    1283 %measurements.
    1284 
    1285 In the most basic implementation, a collection of measurements for
    1286 detections from a set of images is loaded into DVO along with the
    1287 metadata describing the images.  The latter includes properties such
    1288 as the exposure time, airmass, filter, time \& date of the exposure,
    1289 etc.  Critically, the image metadata includes an astrometric
    1290 transformation relating the detection coordinate on the image to the
    1291 coordinate on the sky.  As the collection of measurements are loaded
    1292 into DVO, the software constructs astronomical objects based on those
    1293 detections.  If images overlap, multiple observations of the same
    1294 astronomical object are grouped together.  Thus, a single DVO database
    1295 will contain a one-to-many relationship between the images and the
    1296 measurements and a many-to-one relationship between the measurements
    1297 and the derived astronomical objects.
    1298 
    1299 %
    1300 %% These tables fall into one of several classes:
    1301 %% those which store information about the average properties of
    1302 %% astronomical objects; those which store information about individual
    1303 %% measurements; those which store information about the images; those
    1304 %% which store supporting information (metadata).
    1305 
    1306 %% DVO includes two major classes of database tables: those containing
    1307 %% information about astronomical objects in the sky and those containing
    1308 %% other supporting information.  The object-related tables are
    1309 %% partitioned on the basis of position in the sky: objects within a
    1310 %% region bounded by lines of constant RA,DEC are contained in a specific
    1311 %% file.  The boundaries and the associated partition names are stored in
    1312 %% one of the supporting tables, \ippdbtable{SkyTable}.  This table
    1313 %% contains the definitions of the boundaries for each sky region
    1314 %% (\ippdbcolumn{R_MIN}, \ippdbcolumn{R_MAX}, \ippdbcolumn{D_MIN},
    1315 %% \ippdbcolumn{D_MAX}), the name of the sky region, an ID
    1316 %% (\ippdbcolumn{INDEX}, equal to the sequence number of the region in
    1317 %% the table), and index entries to enable navigation within the table.
    1318 %% The regions are defined in a hierarchical sense, with a series of
    1319 %% levels each containing a finer mesh of regions covering the sky.
    1320 
    1321 \subsubsection{DVO Schema}
    1322 
    1323 \subsubsubsection{Photcodes}
     1323\end{table*}
     1324
     1325\paragraph{Photcodes}
    13241326
    13251327% photcodes
     
    13661368photcode of the measurement.
    13671369
    1368 \subsubsubsection{Measurement Tables}
     1370\paragraph{Measurement Tables}
    13691371
    13701372In most cases, the individual measurements of the astronomical objects
     
    14381440% \note{Average used above but defined below}
    14391441
    1440 \subsubsubsection{Object Tables}
     1442\paragraph{Object Tables}
    14411443\label{sec:object}
    14421444
     
    15011503calculated.
    15021504
    1503 \subsubsubsection{Image Tables}
     1505\paragraph{Image Tables}
    15041506
    15051507Measurements which are loaded into DVO may be associated with a
     
    15361538%% \ippdbtable{Measure} and similar tables,
    15371539
    1538 \subsubsubsection{Other Tables}
     1540\paragraph{Other Tables}
    15391541
    15401542Other tables are used to track information used by the calibration
     
    27672769\rfloor + 1$, where $\mathrm{nodes}_\mathrm{max}$ is the maximum
    27682770number of nodes that can be requested in a single job (1000 for
    2769 Mustang).  Table \ref{tab:SC processing parameters} contains the cost
     2771Mustang).  Table \ref{tab:SC_processing_parameters} contains the cost
    27702772values used for the various IPP processing stages.
    27712773
    2772 %% \begin{table}
    2773 %% \caption{\label{tab:SC_processing_parameters} Cost values for remote processing}\vspace{-0.5cm}
    2774 %% \begin{center}
    2775 %% \begin{tabular}{lcc}
    2776 %% \hline
    2777 %% \hline
    2778 %% {\bf IPP Stage} & {\bf $t_\mathrm{task}$ (s)} & {\bf $S_\mathrm{task}$} \\
    2779 %% \hline
    2780 %%   \ippstage{chip} & 150 & 2 \\
    2781 %%   \ippstage{camera} & 1700 & 2 \\
    2782 %%   \ippstage{warp} & 110 & 2 \\
    2783 %%   \ippstage{stack} & 1500 & 6 \\
    2784 %%   \ippstage{staticsky} & 7200 & 6 \\
    2785 %% %  \ippstage{diff} & 300 & 2 \\
    2786 %%   \ippstage{fullforce} & 300 & 2 \\
    2787 %% \hline
    2788 %% \end{tabular}
    2789 %% \end{center}
    2790 %% \end{table}
    2791 
    2792 \begin{deluxetable}{lcc}
    2793   \tablecolumns{3}
    2794   \tablewidth{0pc}
    2795   \tablecaption{Cost values for remote processing}
    2796   \tablehead{\colhead{IPP Stage}&\colhead{$t_\mathrm{task}$ (s)}&\colhead{$S_\mathrm{task}$}}
    2797   \startdata
     2774\begin{table*}
     2775\caption{\label{tab:SC_processing_parameters} Cost values for remote processing}
     2776\begin{center}
     2777\begin{tabular}{lcc}
     2778\hline
     2779\hline
     2780{\bf IPP Stage} & {\bf $t_\mathrm{task}$ (s)} & {\bf $S_\mathrm{task}$} \\
     2781\hline
    27982782  \ippstage{chip} & 150 & 2 \\
    27992783  \ippstage{camera} & 1700 & 2 \\
     
    28022786  \ippstage{staticsky} & 7200 & 6 \\
    28032787%  \ippstage{diff} & 300 & 2 \\
    2804   \ippstage{fullforce} & 300 & 2
    2805   \enddata
    2806   \label{tab:SC processing parameters}
    2807 \end{deluxetable}
     2788  \ippstage{fullforce} & 300 & 2 \\
     2789\hline
     2790\end{tabular}
     2791\end{center}
     2792\end{table*}
     2793
     2794%%\begin{deluxetable}{lcc}
     2795%%  \tablecolumns{3}
     2796%%  \tablewidth{0pc}
     2797%%  \tablecaption{Cost values for remote processing}
     2798%%  \tablehead{\colhead{IPP Stage}&\colhead{$t_\mathrm{task}$ (s)}&\colhead{$S_\mathrm{task}$}}
     2799%%  \startdata
     2800%%  \ippstage{chip} & 150 & 2 \\
     2801%%  \ippstage{camera} & 1700 & 2 \\
     2802%%  \ippstage{warp} & 110 & 2 \\
     2803%%  \ippstage{stack} & 1500 & 6 \\
     2804%%  \ippstage{staticsky} & 7200 & 6 \\
     2805%%%  \ippstage{diff} & 300 & 2 \\
     2806%%  \ippstage{fullforce} & 300 & 2
     2807%%  \enddata
     2808%%  \label{tab:SC processing parameters}
     2809%%\end{deluxetable}
    28082810
    28092811Once the preparation for the job is complete, the input and output
     
    29002902
    29012903\bibliographystyle{apj}
    2902 \bibliography{lib}{}
    2903 %\input{datasystem.bbl}
     2904%\bibliography{lib}{}
     2905\input{datasystem.bbl}
    29042906
    29052907% \appendix
Note: See TracChangeset for help on using the changeset viewer.