Changes between Version 168 and Version 169 of ippToPsps

Timestamp:

Oct 5, 2011, 3:11:11 PM (15 years ago)

Author:

rhenders

Comment:

--

ippToPsps

@@ -154 +154 @@
  * it will ignore certain missing values, which would otherwise fail the batch
 
+=== Multiple 'clients' ===
+
+It is possible, and usually necessary, to run multiple versions of {{{load.py}}}. The code has been designed so that the user need simply run the program as described above. Any number of clients can run concurrently on any number of hosts, so long as they have a local 'scratch' database set up properly (see above).
+
+Note that clients on machines where the DVO database is not on a local disk will run more slowly due to the increased network reading of DVO data files.
+
 == Running {{{dvoToMySQL}}} ==
 
@@ -170 +176 @@
 When loading, three copies of the data exist: the original files on disk, the batches on the datastore and a third copy on the PSPS loading machine. The deletion policy at present is:
 
- * when a batch has been loaded to the ODM and has a status of 'merge worthy', then the copy on the datastore and the DXLayer copy can be removed
- * when a batch has been successfully merged into the PSPS database, the final copy on local disk is deleted
-
-The logic for this is that errors may occur during the merge phase and it is useful to have local copies of offending batches for debugging purposes.
-
-This standard behavior can be changed by setting the appropriate values in the ''deletion'' section of the config file.
+ * when a batch has been loaded to the ODM and has a status of 'merge worthy' '''or''' 'failed', then the copy on the datastore and the DXLayer's copy can be removed
+ * when a batch has been successfully merged into the PSPS database, then final copy on local disk may be deleted
+
+The logic for this is that errors may occur during the merge phase and it is useful to have local copies of offending batches for debugging purposes. This standard behavior can be changed by setting the appropriate values in the ''deletion'' section of the config file.
+
+To actually perform the clean, run the {{{cleanup.py}}} program like this:
+
+{{{
+./run.sh cleanup.py  configs/lap20110809.xml
+}}}
 
 == Metrics ==