== IPP Progress Report for the week 2011.11.21 - 2011.11.25 == 
[[PageOutline]]

(Up to [wiki:IPP_Progress_Reports IPP Progress Reports])

=== Eugene Magnier ===

=== Serge Chastel ===

=== Heather Flewelling === 

=== Roy Henderson ===


 * Put PSPS loading data into czartool for easier monitoring.
  * removed revision/db upgrade stuff from czartool: not necessary, as I will maintain a dump of schema shape instead.
  * added tables to !CzarDb of same shape as IPP ones: 'pending', 'processed', 'faults' for each stage
  * changes to ippToPsps metrics program to write new !CzarDb tables
  * adapted czarplot to deal with ipptopsps stages
  * put ipptopsps metrics program on a timer so it can routinely write to !CzarDb
  * updated !CzarDb maintenance code to trim and optmize new tables
  * updated czarpoll to produce ippToPsps plots
  * created new PHP page in czartool for display

 * Duplicates broke PSPS merge. Investigated and found that duplicate frames were in DVO. Helped Sue remove them before re-merging. This prompted some extra duplicate protection measures:
   * created ipptopsp 'test' database to reduce confusion when I'm testing new code
   * increased 'processing now' limit from 2 to 4 hours, as when network is slow it is feasible that a batch could take longer than 2 hours
   * removed 'loaded_to_datastore' constraint when checking if a stage_id has already been processed as it is a little dangerous (could be called just after a batch is processed by one client, but before it uploads that batch to the datastore)

 * Other ippToPsps coding:
  * found bug in loading code: batches that failed at the tar and zip stage prior to publication to datastore were being labelled as successfully processed, then failing to publish
  * populated some new fields for stacks

 * Other
  * my crontab'd backup script for various Dbs (ipptopsps, czadb, mailing lists) now dumps a copy of the schema as well
  * cleaned-up some failed batches
  * work setting-up re-loading of SAS data to PSPS
=== Mark Huber ===

=== Bill Sweeney ===

=== Chris Waters ===