| | 136 | |
| | 137 | {{{cleanup.py}}} monitors the status of each batch for a given config and deletes the three existing copies at the appropriate time. |
| | 138 | |
| | 139 | {{{ |
| | 140 | ./run.sh cleanup.py someConfigName |
| | 141 | }}} |
| | 142 | |
| | 143 | |
| | 144 | The deletion policy is as follows. |
| | 145 | |
| | 146 | When loading, three copies of the data exist: the original files on disk, the batches on the datastore and a third copy on the PSPS loading machine. The deletion policy at present is: |
| | 147 | |
| | 148 | * when a batch has been loaded to the ODM and has a status of 'merge worthy' '''or''' 'failed', then the copy on the datastore and the DXLayer's copy can be removed |
| | 149 | * when a batch has been successfully merged into the PSPS database, then final copy on local disk may be deleted |
| | 150 | |
| | 151 | The logic for this is that errors may occur during the merge phase and it is useful to have local copies of offending batches for debugging purposes. This standard behavior can be changed by setting the appropriate values in the ''deletion'' section of the [wiki:ippToPsps_Database#Theconfigtable config]. |
| | 152 | |
| | 153 | |
