| | 14 | |
| | 15 | * Put PSPS loading data into czartool for easier monitoring. |
| | 16 | * removed revision/db upgrade stuff from czartool: not necessary, as I will maintain a dump of schema shape instead. |
| | 17 | * added tables to CzarDb of same shape as IPP ones: 'pending', 'processed', 'faults' for each stage |
| | 18 | * changes to ippToPsps metrics program to write new czarDb tables |
| | 19 | * adapted czarplot to deal with ipptopsps stages |
| | 20 | * put ipptopsps metrics program on a timer so it can routinely write to CzarDb |
| | 21 | * updated czardb maintenance code to trim and optmize new tables |
| | 22 | * updated czarpoll to produce ippToPsps plots |
| | 23 | * created new PHP page in czartool for display |
| | 24 | |
| | 25 | * Duplicates broke PSPS merge. Investigated and found that duplicate frames were in DVO. Helped Sue remove them before re-merging. This prompted some extra duplicate protection measures: |
| | 26 | * created ipptopsp 'test' database to reduce confusion when I'm testing new code |
| | 27 | * increased 'processing now' limit from 2 to 4 hours, as when network is slow it is feasible that a batch could take longer than 2 hours |
| | 28 | * removed 'loaded_to_datastore' constraint when checking if a stage_id has already been processed as it is a little dangerous (could be called just after a batch is processed by one client, but before it uploads that batch to the datastore) |
| | 29 | |
| | 30 | * Other ippToPsps coding: |
| | 31 | * found bug in loading code: batches that failed at the tar and zip stage prior to publication to datastore were being labelled as successfully processed, then failing to publish |
| | 32 | * populated some new fields for stacks |
| | 33 | |
| | 34 | * Other |
| | 35 | * my crontab'd backup script for various Dbs (ipptopsps, czadb, mailing lists) now dumps a copy of the schema as well |
| | 36 | * cleaned-up some failed batches |
| | 37 | * work setting-up re-loading of SAS data to PSPS |
