Changes between Version 8 and Version 9 of PS1_IPP_Czarlog_20151005

Timestamp:

Oct 8, 2015, 10:11:59 AM (11 years ago)

Author:

eugene

Comment:

--

PS1_IPP_Czarlog_20151005

@@ -30 +30 @@
  * 17:40 EAM: stopping and restarting pantasks for the night
 
-=== Thursday : YYYY.MM.DD ===
+=== Thursday : 2015.10.08 ===
+ * 09:30 EAM: running updates on the outstanding ws diff inputs.  I've generated a scripts in ~ipp to handle this situation so we don't have to keep re-inventing the commands.  The script is called "~ipp/fix.ws.nightlyscience".  Here are examples using it to find and fix the WS diff run inputs and then the diff runs:
+   * list state of inputs to the outstanding WS diff runs:
+{{{
+ipp@ippc19:/home/panstarrs/ipp>fix.ws.nightlyscience ippuser list
++---------+---------+---------+------------------+---------+---------+-------+
+| chip_id | warp_id | diff_id | data_group       | state   | state   | state |
++---------+---------+---------+------------------+---------+---------+-------+
+| 1676749 | 1624170 | 1218073 | SNIa.20151002    | cleaned | full    | new   | 
+| 1677272 | 1624691 | 1222131 | ThreePi.20151003 | full    | cleaned | new   | 
+| 1677283 | 1624702 | 1222307 | ThreePi.20151003 | full    | cleaned | new   | 
+| 1677582 | 1624994 | 1222718 | OSS.20151003     | cleaned | cleaned | new   | 
++---------+---------+---------+------------------+---------+---------+-------+
+}}}
+   * generate chiptool and warptool commands to fix the problems above:
+{{{
+ipp@ippc19:/home/panstarrs/ipp>fix.ws.nightlyscience ippuser update
+chiptool -dbname gpc1 -setimfiletoupdate  -set_label ws_nightly_update -chip_id 1676749
+chiptool -dbname gpc1 -setimfiletoupdate  -set_label ws_nightly_update -chip_id 1677582
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ws_nightly_update -warp_id 1624691
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ws_nightly_update -warp_id 1624702
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ws_nightly_update -warp_id 1624994
+}}}
+You can cut and paste the above lines, redirect them into a script to be sourced, or pipe the output directly to csh.  After running the above, you will see the states get updated:
+{{{
+ipp@ippc19:/home/panstarrs/ipp>fix.ws.nightlyscience ippuser list
++---------+---------+---------+------------------+--------+--------+-------+
+| chip_id | warp_id | diff_id | data_group       | state  | state  | state |
++---------+---------+---------+------------------+--------+--------+-------+
+| 1676749 | 1624170 | 1218073 | SNIa.20151002    | update | full   | new   | 
+| 1677272 | 1624691 | 1222131 | ThreePi.20151003 | full   | update | new   | 
+| 1677283 | 1624702 | 1222307 | ThreePi.20151003 | full   | update | new   | 
+| 1677582 | 1624994 | 1222718 | OSS.20151003     | update | update | new   | 
++---------+---------+---------+------------------+--------+--------+-------+
+}}}
+The runs get set to the label 'ws_nightly_update' which I've added to the input script for stdscience, so it is always available.
+
+After the inputs are processed, you may find the diff runs still fail (likely, as this is why they never completed in the first place).  You can list the diff runs like this:
+{{{
+ipp@ippc19:/home/panstarrs/ipp>fix.ws.nightlyscience ippuser diffstate
++---------+------------------+---------------------------+------------------+-------+-------+
+| diff_id | skycell_id       | label                     | data_group       | state | fault |
++---------+------------------+---------------------------+------------------+-------+-------+
+| 1222131 | skycell.1565.069 | ThreePi.WS.nightlyscience | ThreePi.20151003 | new   |     2 | 
+| 1222307 | skycell.1566.012 | ThreePi.WS.nightlyscience | ThreePi.20151003 | new   |     2 | 
+| 1218073 | skycell.1051.065 | SNIa.WS.nightlyscience    | SNIa.20151002    | new   |     5 | 
+| 1222718 | skycell.0885.058 | OSS.WS.nightlyscience     | OSS.20151003     | new   |     5 | 
++---------+------------------+---------------------------+------------------+-------+-------+
+}}}
+You can also examine the logs with the command:
+{{{
+fix.ws.nightlyscience ippuser difflogs   
+}}}
+Finally, the script will generate difftool commands to set the quality flags for the failures:
+{{{
+ipp@ippc19:/home/panstarrs/ipp>fix.ws.nightlyscience ippuser difffix
+difftool -dbname gpc1 -updatediffskyfile -fault 0 -set_quality 66 -diff_id 1222131 -skycell_id skycell.1565.069
+difftool -dbname gpc1 -updatediffskyfile -fault 0 -set_quality 66 -diff_id 1222307 -skycell_id skycell.1566.012
+difftool -dbname gpc1 -updatediffskyfile -fault 0 -set_quality 42 -diff_id 1218073 -skycell_id skycell.1051.065
+difftool -dbname gpc1 -updatediffskyfile -fault 0 -set_quality 42 -diff_id 1222718 -skycell_id skycell.0885.058
+}}}
+In this case, the fault 2 entries had missing stack inputs, so they get quality 66 while the fault 5s were one of the more traditional psphot failures.
 
 === Friday : YYYY.MM.DD ===