ISP SAS Processing

Heather is attempting to process the SAS area of sky in skyprobe.

Issues

• ippMonitor / isp is broken - it looks at an old database? Heather doesn't know how to fix this
• there are multiple dark detrends for isp and heather doesn't remember why. The one for 2011 (det_id 14) is missing, so heather set that to drop at the moment to continue processing
  • assume they were not replicated and was on a machine that is dead or out of nebulous...
• need to fix summitcopy/registration to ingest the comment from isp data, data after a certain date does not have a comment.

First steps

• compiled ipp trunk as of 9-7-2012 as heather, running ispstdscience as heather on ipp007 (neither of these matter, just for reference)
• we want warps and stacks - likely the gpc1 skycells are not idea. Heather (with help of gene) created new skycells for isp, called ISP.V0. They are essentially identical to gpc1, except the pixel size is much larger (5.57), and each projection cell is not subdivided (gpc1 subdivides by 10x10)
  • these are located at /local/ipp/isp/ISP.V0 on all relevant machines
  • the magical command: skycells -mode RINGS -scale 5.57 -nx 1 -ny 1 -fix-ns -D CATDIR ISP.V0 -overlap 30 30 -skyparity
• queued (repeatedly) sas.v0 stuff. this overlaps with MD09, so did the following to queue:
  • chiptool -dbname isp -definebyquery -set_label sas.v0 -dateobs_begin 2010-01-01T00:00:00 -comment '%ps1%' -dateobs_end 2012-10-02T00:00:00 -set_workdir neb://@HOST@.0/isp/sas.v0 -ra_min 328 -ra_max 339 -decl_min -6 -decl_max 6 -set_end_stage warp -set_tess_id ISP.V0 -simple

• here's the status of processing:

   mysql> select count(*), state from chipRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |     3300 | full  | 
  |       21 | new   | 
  +----------+-------+
  2 rows in set (0.02 sec)
  
  mysql> select count(*), state from camRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |     3300 | full  | 
  +----------+-------+
  1 row in set (0.01 sec)
  
  mysql> select count(*), state from warpRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |      143 | full  | 
  |     2335 | new   | 
  +----------+-------+
  2 rows in set (0.01 sec)
  
  

hmmm... warp is still running.... This seems unexpected... What do the statistics look like?

first look at some warps.

mysql> select uri from warpRun join warpSkyfile using (warp_id) where label = 'SAS.v0' and state = 'full' and warp_id = 29;
+---------------------------------------------------------------------------------------+
| uri                                                                                   |
+---------------------------------------------------------------------------------------+
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1404.fits | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1405.fits | 
| NULL                                                                                  | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1493.fits | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1494.fits | 
| NULL                                                                                  | 
| NULL                                                                                  | 
| NULL                                                                                  | 
+---------------------------------------------------------------------------------------+
8 rows in set (0.00 sec)

database says this one is processed, and it tried 8 skycells (?). I loaded up the 4 good ones in ds9:

 ds9 -mosaicwcs `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1404.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1405.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1493.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1494.fits`

Hey, this doesn't look so bad, except for it being out of focus. D'oh...

according to the database, teh fwhm is 17... yuuuuck!

here are some statistics on the camRun:

mysql> select count(*), min(fwhm_major), max(fwhm_major), min(sigma_ra), max(sigma_ra)  from camRun join camProcessedExp using (cam_id) where label = 'SAS.v0' limit 1;
+----------+------------------+-----------------+------------------+-----------------+
| count(*) | min(fwhm_major)  | max(fwhm_major) | min(sigma_ra)    | max(sigma_ra)   |
+----------+------------------+-----------------+------------------+-----------------+
|     3300 | 0.89642798900604 | 28.676900863647 | 0.44685900211334 | 37.357189178467 | 
+----------+------------------+-----------------+------------------+-----------------+

and here's what I think are potentially good: I require sigma_ra < 1 (is that reasonable?), n_astrom > 0 (otherwise, astrometry failed), and fwhm_major < 5 (no ideas if this is reasonable or not...)

mysql> select count(*), min(fwhm_major), max(fwhm_major)  from camRun join camProcessedExp using (cam_id) where label = 'SAS.v0' and sigma_ra < 1 and n_astrom > 0 and fwhm_major < 5;
+----------+-----------------+-----------------+
| count(*) | min(fwhm_major) | max(fwhm_major) |
+----------+-----------------+-----------------+
|     1356 |  1.417240023613 | 4.9119601249695 | 
+----------+-----------------+-----------------+
1 row in set (0.01 sec)