Changes between Version 10 and Version 11 of ippToPspsOrr

Timestamp:

Feb 16, 2012, 4:14:45 PM (14 years ago)

Author:

rhenders

Comment:

--

ippToPspsOrr

@@ -56 +56 @@
 == Anatomy of a batch ==
 
-To give an idea of what takes time during loading with ippToPsps, this section gives a timing breakdown of a run-of-the-mill batch for a PS1 exposure with 83,542 detections. Below is a section of the ippToPsps log detailing all stages of processing for this batch.
+To give an idea of where the time is spent when loading with ippToPsps, this section gives a timing breakdown of a run-of-the-mill batch for a PS1 exposure with 83,542 detections. Below is a section of the ippToPsps log detailing all stages of processing for this batch. This is an example using the standard DVO on-the-fly access, i.e. not using pre-ingested DVO data.
 
 {{{
@@ -118 +118 @@
 Clearly DVO is the bottleneck. This example is from a client running on the same machine as the DVO database (accessing over the network slows down DVO access substantially). Also, during this batch creation, two other clients were running on the same machine using the same MySQL database.
 
+Using pre-ingested DVO data speeds up batch creation time, but the ingest itself takes time so a ratio of smfs-available/size-of-DVO-data-on-disk needs to be considered for each region of sky. Basically, for MD fields it makes sense to pre-ingest, whereas for usual 3PI survey areas it does not.
+
 == Monitoring ippToPsps ==