ISP SAS Processing

Heather is attempting to process the SAS area of sky in skyprobe.

Issues

• ippMonitor / isp is broken - it looks at an old database? Heather doesn't know how to fix this
• there are multiple dark detrends for isp and heather doesn't remember why. The one for 2011 (det_id 14) is missing, so heather set that to drop at the moment to continue processing
  • assume they were not replicated and was on a machine that is dead or out of nebulous...
• need to fix summitcopy/registration to ingest the comment from isp data, data after a certain date does not have a comment.

First steps

• compiled ipp trunk as of 9-7-2012 as heather, running ispstdscience as heather on ipp007 (neither of these matter, just for reference)
• we want warps and stacks - likely the gpc1 skycells are not idea. Heather (with help of gene) created new skycells for isp, called ISP.V0. They are essentially identical to gpc1, except the pixel size is much larger (5.57), and each projection cell is not subdivided (gpc1 subdivides by 10x10)
  • these are located at /local/ipp/isp/ISP.V0 on all relevant machines
  • the magical command: skycells -mode RINGS -scale 5.57 -nx 1 -ny 1 -fix-ns -D CATDIR ISP.V0 -overlap 30 30 -skyparity
• queued (repeatedly) sas.v0 stuff. this overlaps with MD09, so did the following to queue:
  • chiptool -dbname isp -definebyquery -set_label sas.v0 -dateobs_begin 2010-01-01T00:00:00 -comment '%ps1%' -dateobs_end 2012-10-02T00:00:00 -set_workdir neb://@HOST@.0/isp/sas.v0 -ra_min 328 -ra_max 339 -decl_min -6 -decl_max 6 -set_end_stage warp -set_tess_id ISP.V0 -simple

• here's the status of processing:

   mysql> select count(*), state from chipRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |     3300 | full  | 
  |       21 | new   | 
  +----------+-------+
  2 rows in set (0.02 sec)
  
  mysql> select count(*), state from camRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |     3300 | full  | 
  +----------+-------+
  1 row in set (0.01 sec)
  
  mysql> select count(*), state from warpRun where label = 'SAS.v0' group by state;
  +----------+-------+
  | count(*) | state |
  +----------+-------+
  |      143 | full  | 
  |     2335 | new   | 
  +----------+-------+
  2 rows in set (0.01 sec)
  
  

hmmm... warp is still running.... This seems unexpected... What do the statistics look like?

first look at some warps.

mysql> select uri from warpRun join warpSkyfile using (warp_id) where label = 'SAS.v0' and state = 'full' and warp_id = 29;
+---------------------------------------------------------------------------------------+
| uri                                                                                   |
+---------------------------------------------------------------------------------------+
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1404.fits | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1405.fits | 
| NULL                                                                                  | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1493.fits | 
| neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1494.fits | 
| NULL                                                                                  | 
| NULL                                                                                  | 
| NULL                                                                                  | 
+---------------------------------------------------------------------------------------+
8 rows in set (0.00 sec)

database says this one is processed, and it tried 8 skycells (?). I loaded up the 4 good ones in ds9:

 ds9 -mosaicwcs `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1404.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1405.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1493.fits` `neb-locate --path neb://any/isp/sas.v0/o5345i0213o01.89259/o5345i0213o01.89259.wrp.29.skycell.1494.fits`

Hey, this doesn't look so bad, except for it being out of focus. D'oh...

according to the database, teh fwhm is 17... yuuuuck!

here are some statistics on the camRun:

mysql> select count(*), min(fwhm_major), max(fwhm_major), min(sigma_ra), max(sigma_ra)  from camRun join camProcessedExp using (cam_id) where label = 'SAS.v0' limit 1;
+----------+------------------+-----------------+------------------+-----------------+
| count(*) | min(fwhm_major)  | max(fwhm_major) | min(sigma_ra)    | max(sigma_ra)   |
+----------+------------------+-----------------+------------------+-----------------+
|     3300 | 0.89642798900604 | 28.676900863647 | 0.44685900211334 | 37.357189178467 | 
+----------+------------------+-----------------+------------------+-----------------+

and here's what I think are potentially good: I require sigma_ra < 1 (is that reasonable?), n_astrom > 0 (otherwise, astrometry failed), and fwhm_major < 5 (no ideas if this is reasonable or not...) Note that we do have odd problems including dome occultation, so we do expect some fraction to suck.

mysql> select count(*), min(fwhm_major), max(fwhm_major)  from camRun join camProcessedExp using (cam_id) where label = 'SAS.v0' and sigma_ra < 1 and n_astrom > 0 and fwhm_major < 5;
+----------+-----------------+-----------------+
| count(*) | min(fwhm_major) | max(fwhm_major) |
+----------+-----------------+-----------------+
|     1356 |  1.417240023613 | 4.9119601249695 | 
+----------+-----------------+-----------------+
1 row in set (0.01 sec)

Here's the breakdown by filter:

mysql> select filter, count(*), min(fwhm_major), max(fwhm_major)  from camRun join camProcessedExp using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where camRun.label = 'SAS.v0' and sigma_ra < 1 and n_astrom > 0 and fwhm_major < 5 group by filter;
+--------+----------+-----------------+-----------------+
| filter | count(*) | min(fwhm_major) | max(fwhm_major) |
+--------+----------+-----------------+-----------------+
| g      |      284 |         3.01373 |         3.65707 | 
| i      |      719 |         1.41724 |         4.91196 | 
| r      |      311 |          1.6642 |         4.58159 | 
| z      |       42 |         1.48253 |         1.67822 | 
+--------+----------+-----------------+-----------------+
4 rows in set (0.03 sec)

I'm a little surprised at the lack of y, but y is horrible to process with skyprobe, so I'm not going to worry about it for now... Clearly, there are a lot of images in gri, and a few in z (?). This should be enough to stack. I can sort by skycells:

mysql> select filter, skycell_id, count(*), min(fwhm_major), max(fwhm_major)  from warpRun join warpSkyfile using (warp_id) join fakeRun using (fake_id) join camRun using (cam_id) join camProcessedExp using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where camRun.label = 'SAS.v0' and sigma_ra < 1 and n_astrom > 0 and fwhm_major < 5 and warpSkyfile.quality = 0 group by filter, skycell_id;
+--------+--------------+----------+-----------------+-----------------+
| filter | skycell_id   | count(*) | min(fwhm_major) | max(fwhm_major) |
+--------+--------------+----------+-----------------+-----------------+
| g      | skycell.1314 |        1 |         3.59727 |         3.59727 | 
| g      | skycell.1315 |        2 |          3.6018 |          3.6311 | 
| g      | skycell.1316 |        2 |          3.6018 |          3.6311 | 
| g      | skycell.1404 |        2 |         3.59727 |         3.60911 | 
| g      | skycell.1405 |        4 |         3.59727 |          3.6311 | 
| g      | skycell.1406 |        3 |          3.6018 |          3.6382 | 
| g      | skycell.1407 |        1 |          3.6382 |          3.6382 | 
| g      | skycell.1494 |        1 |         3.60911 |         3.60911 | 
| g      | skycell.1495 |        1 |          3.6382 |          3.6382 | 
| g      | skycell.1496 |        1 |          3.6382 |          3.6382 | 
| i      | skycell.1314 |        2 |         1.63937 |         1.66888 | 
| i      | skycell.1315 |        1 |         1.66888 |         1.66888 | 
| i      | skycell.1404 |        4 |         1.63937 |         2.42179 | 
| i      | skycell.1405 |        4 |         1.63937 |         1.67396 | 
| i      | skycell.1406 |        3 |         1.64807 |         1.67396 | 
| i      | skycell.1407 |        1 |         1.64807 |         1.64807 | 
| i      | skycell.1493 |        4 |         1.76622 |         2.42179 | 
| i      | skycell.1494 |        4 |         1.67151 |         2.42179 | 
| i      | skycell.1495 |        3 |         1.64807 |         1.67396 | 
| i      | skycell.1582 |        1 |         1.81341 |         1.81341 | 
| r      | skycell.1314 |        2 |         1.80198 |         1.94848 | 
| r      | skycell.1315 |        1 |         1.94848 |         1.94848 | 
| r      | skycell.1404 |        4 |         1.80198 |         3.14089 | 
| r      | skycell.1405 |        4 |         1.80198 |         3.14089 | 
| r      | skycell.1406 |        3 |         1.91914 |         1.98396 | 
| r      | skycell.1407 |        1 |         1.91914 |         1.91914 | 
| r      | skycell.1493 |        4 |         1.85289 |         3.14089 | 
| r      | skycell.1494 |        4 |         1.85289 |         3.14089 | 
| r      | skycell.1495 |        3 |         1.91914 |         1.98396 | 
| r      | skycell.1496 |        2 |         1.91914 |         1.98396 | 
| z      | skycell.1493 |        1 |          1.6754 |          1.6754 | 
+--------+--------------+----------+-----------------+-----------------+
31 rows in set (0.00 sec)

of course, it helps if we've processed more than a handful of stuff.... ugh... need to wait now...

Monday

A quick check of the db says that it's processed 920, 1558 left still to go. I wonder why it is so slow for warps? It's still processing according to pantasks_client, with only a minimal number of faults (6 out of 145424). Why are there so many processed skycells?!

This is seriously weird. For example, for warp_id = 955, there are 504 skycells, all but 2 have crappy quality. What gives?

Sometimes it has a reasonable < 10 number of skycells, but often it has hundreds... I'm still investigating, but it seems that the camRuns that have hundreds of skycells are what I would call reasonable (ie, good, sigma_ra, good nastrom, good astrometry, good focus..)