PS1 IPP Czar Logs for the week 2012.05.28 - 2012.06.03

(Up to PS1 IPP Czar Logs)

Monday : 2012.05.28

• few MD08 exposures last night, look to be through okay. looking at if the new masking Chris setup is being used and if a problem.

​http://ippmonitor.ipp.ifa.hawaii.edu/ippMonitor/showimage.php?name=neb://any/gpc1/MD08.nt/2012/05/28//o6075g0034o.492281/o6075g0034o.492281.cm.443364&rule=PPIMAGE.JPEG1&camera=GPC1&class_id=NONE

​http://ippmonitor.ipp.ifa.hawaii.edu/ippMonitor/showimage.php?name=neb://any/gpc1/MD08.nt/2012/05/20//o6067g0308o.490670/o6067g0308o.490670.cm.439923&rule=PPIMAGE.JPEG1&camera=GPC1&class_id=NONE

• looks like more masking so should probably revert masks for now

  | exp_id | exp_name    | label                      | ref_static | ref_dyna | ref_advis | max_static | max_dyna | max_advis |
  +--------+-------------+----------------------------+------------+----------+-----------+------------+----------+-----------+
  | 490670 | o6067g0308o | MD08.20120520              | 0.197      | 0.000234 | 0.0201    | 0.211      | 0.000223 | 0.0196    | 
  | 490672 | o6067g0309o | MD08.20120520              | 0.197      | 0.000280 | 0.0213    | 0.211      | 0.000300 | 0.0208    | 
  
  | 491784 | o6072g0163o | MD08.20120525              | 0.198      | 0.000255 | 0.0153    | 0.212      | 0.000275 | 0.0149    | 
  | 491785 | o6072g0164o | MD08.20120525              | 0.199      | 0.000246 | 0.0164    | 0.213      | 0.000243 | 0.0159    | 
  
  | 492281 | o6075g0034o | MD08.20120528              | 0.240      | 0.000245 | 0.0060    | 0.253      | 0.000234 | 0.0058    | 
  | 492282 | o6075g0035o | MD08.20120528              | 0.238      | 0.000258 | 0.0079    | 0.252      | 0.000247 | 0.0076    | 
  

Tuesday : 2012.05.29

Mark is czar

• no data last night. reverted to previous mask set using a script Chris setup since his new ones to remove static issues due to crosstalk bleeds/etc was masking more than expected/desired.
• 11:00 Serge: stopped replication save on ippdb02 to give mysqldump a chance to finish. Changed nebulous backup copy on ipp@ipp001 to 20:05 (instead of 12:05).
• 15:40 re-ran SAS.Footprint to verify masks reverted. looks like they are.

  | exp_id | exp_name    | label                      | ref_static | ref_dyna | ref_advis | max_static | max_dyna | max_advis |
  +--------+-------------+----------------------------+------------+----------+-----------+------------+----------+-----------+
  | 356282 | o5744g0329o | czw.footprint.test21       | 0.195      | 0.000031 | 0.0102    | 0.209      | 0.000029 | 0.0099    | 
  | 356282 | o5744g0329o | czw.footprint.test120525   | 0.225      | 0.000029 | 0.0096    | 0.238      | 0.000027 | 0.0094    | 
  | 356282 | o5744g0329o | czw.footprint.test120525v2 | 0.238      | 0.000029 | 0.0094    | 0.251      | 0.000027 | 0.0092    | 
  | 356282 | o5744g0329o | czw.footprint.test120529   | 0.195      | 0.000031 | 0.0102    | 0.209      | 0.000029 | 0.0099    | 
  | 356283 | o5744g0330o | czw.footprint.test21       | 0.195      | 0.000036 | 0.0106    | 0.209      | 0.000034 | 0.0104    | 
  | 356283 | o5744g0330o | czw.footprint.test120525   | 0.224      | 0.000034 | 0.0101    | 0.237      | 0.000032 | 0.0099    | 
  | 356283 | o5744g0330o | czw.footprint.test120525v2 | 0.238      | 0.000034 | 0.0099    | 0.251      | 0.000031 | 0.0097    | 
  | 356283 | o5744g0330o | czw.footprint.test120529   | 0.195      | 0.000036 | 0.0106    | 0.209      | 0.000034 | 0.0104    | 
  

• 15:50 shutting down all pantasks and turning apache off on ippc17 while Serge does some DB work.
• 16:15 Serge is now in charge.
• 16:30 Serge: restarted slave on ippdb02. All apache nebulous conf files are now pointing towards ippdb02. I will wait till 8pm for jt scripts running on ippc19 to finish.
• 16:35 Serge: nebulous slave is now synchronized with its master.
• 17:10 Serge: the things to be done are described here (just in case I have a moped accident)

Wednesday : 2012-05-30

Serge is czar

• No observation last night due to a problem with the primary mirror actuator system. Repair is scheduled for 5-30-2012 HST (quoted from ps-obs report)
• Nebulous maintenance is happening and details can be seen in the same reporting wikipage as yesterday
• 12:00 Started optimization of gpc1 on ippdb01
• 15:25 All but the last two versions of src/ipp-<tag> are being moved to /export/ippc18.1/ipp/src/Archives/
• 16:35 Moved ~ipp/iasc to ippc18.1. Symbolic link in ~ipp
• 17:08 End of gpc1 optimization: 195GB before. 167GB now.
• 17:10 No change for the ippdb00 ingestion completion: 6am.

Thursday :2012-05-31

Serge is czar

• 09:30 ippdb00 ingestion is still happening
• 11:00 Processing has been restarted
• 11:57 Processing stopped. The new tag is being built.
• 13:05 All pantasks servers (but ippdvo related) have been restarted
• 14:05 mysql in ippdb00 has crashed. From /var/log/message:

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD