IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 48 and Version 49 of STS.refstack.20120510


Ignore:
Timestamp:
Jun 13, 2012, 10:20:22 PM (14 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • STS.refstack.20120510

    v48 v49  
    352352
    353353--> even for c1f37 has FWHM>4.2 going into it -- gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899919.log
    354 
     354--> however, not as overdriven as feared. --- need to add plot/vector of PSF shape in skycell ---
    355355}}}
    356356
    357357psphot on samples, unconvolved and convolved (compare to ppStack log file):
    358358 * faults because of recent modified psphot config/code to limit peaks and so faults.. need to add -Di PEAKS_NMAX_TOTAL 60000
    359  * unconvolved stacks still faulting.. so do PASS1 only -D BREAK_POINT PASS1. Convolved ones are poor, FWHM_MAJ~9-10??
     359 * unconvolved stacks still faulting.. so do PASS1 only -D BREAK_POINT PASS1. Convolved ones can be poor, FWHM_MAJ~9-10??
    360360{{{
    361361gethead -auhc *fpa.cmf FWHM_MAJ IQ_FW1
    … …  
    505505}}}
    506506
     507== clean-up ==
     508Will need to keep warps around for remaking the refstack if necessary but more importantly for MOPS to get stamps without update delay. ~50x7 or 350 exposures on disk for the skycells.
     509
     510{{{
     511chiptool -updaterun -set_state goto_cleaned -set_label goto_cleaned -data_group STS.refstack.20120510 -dbname gpc1
     512
     513//warptool -updaterun -set_state goto_cleaned -set_label goto_cleaned -data_group STS.refstack.20120510 -dbname gpc1
     514}}}
    507515
    508516----
     517
    509518= Test Diffim samples =
    510519Want to create a sample of diffims with detections for MOPS to look at. Will need to also keep in "full" state so they can get postage stamps quickly. Focus on projection 1.
    511520
    512521Basic setup: (warp - convolved stack) -- need examples and compared to unconv stack (why not using this?)
     522
     523
     524== manual set ==
     525 Pick center, edge, edge+lots overlap for various filling types - 1.44, 1.55, 1.03?
     526gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/STS.V4.skycell.1.44.stk.902946.log
     527gpc1/STS.refstack.20120510/STS.V4/skycell.1.55/STS.V4.skycell.1.55.stk.902957.log
     528gpc1/STS.refstack.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.902905.log
     529--> appear mostly well behaved mix.. need to add one more -- 1.06 is wacky
     530gpc1/STS.refstack.20120510/STS.V4/skycell.1.06/STS.V4.skycell.1.06.stk.902908.log
     531
     532--> need to run target2,1 and dual
     533
     534SELECT exp_id,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) join fakeRun using (cam_id) join warpRun using (fake_id) join stackInputSkyfile using (warp_id) join stackRun using (stack_id) WHERE stackRun.data_group like "STS.refstack.20120510" and stackRun.skycell_id like "%1.44" group by exp_id;
     535
     536--> need night listing with PP1 exp_name -
     537
     538select exp_id,exp_name,exp_time,dateobs,comment from rawExp where obs_mode like "%STS%" and comment like "%PP1%" and dateobs>"2012-04";
     539| exp_id | exp_name    | exp_time | dateobs             | comment                  |
     540+--------+-------------+----------+---------------------+--------------------------+
     541| 475121 | o6029g0441o |       30 | 2012-04-12 13:40:09 | STS.180.ND PP1 center    |
     542| 475128 | o6029g0448o |       30 | 2012-04-12 13:45:11 | STS.180.ND PP1 center    |
     543| 475135 | o6029g0455o |       30 | 2012-04-12 13:50:14 | STS.180.ND PP1 center    |
     544| 475141 | o6029g0462o |       30 | 2012-04-12 13:55:16 | STS.180.ND PP1 center    |
     545| 475149 | o6029g0469o |       30 | 2012-04-12 14:00:20 | STS.180.ND PP1 center    |
     546| 475156 | o6029g0476o |       30 | 2012-04-12 14:05:25 | STS.180.ND PP1 center    |
     547| 475163 | o6029g0483o |       30 | 2012-04-12 14:10:27 | STS.180.ND PP1 center    |
     548| 475170 | o6029g0490o |       30 | 2012-04-12 14:15:31 | STS.180.ND PP1 center    |
     549| 475177 | o6029g0497o |       30 | 2012-04-12 14:20:35 | STS.180.ND PP1 center    |
     550| 475184 | o6029g0504o |       30 | 2012-04-12 14:25:37 | STS.180.ND PP1 center    |
     551| 475191 | o6029g0511o |       30 | 2012-04-12 14:30:41 | STS.180.ND PP1 center    |
     552| 475198 | o6029g0518o |       30 | 2012-04-12 14:35:47 | STS.180.ND PP1 center    |
     553| 476338 | o6031g0408o |       20 | 2012-04-14 13:34:26 | STS.180.GIQ PP1 dither 1 |
     554| 476345 | o6031g0415o |       20 | 2012-04-14 13:38:20 | STS.180.GIQ PP1 dither 2 |
     555| 476352 | o6031g0422o |       20 | 2012-04-14 13:42:15 | STS.180.GIQ PP1 dither 3 |
     556| 476360 | o6031g0429o |       20 | 2012-04-14 13:46:09 | STS.180.GIQ PP1 dither 4 |
     557| 476366 | o6031g0436o |       20 | 2012-04-14 13:50:05 | STS.180.GIQ PP1 dither 1 |
     558| 476373 | o6031g0443o |       20 | 2012-04-14 13:53:56 | STS.180.GIQ PP1 dither 2 |
     559| 476380 | o6031g0450o |       20 | 2012-04-14 13:57:48 | STS.180.GIQ PP1 dither 3 |
     560| 476389 | o6031g0457o |       20 | 2012-04-14 14:01:41 | STS.180.GIQ PP1 dither 4 |
     561| 476397 | o6031g0464o |       20 | 2012-04-14 14:08:15 | STS.180.GIQ PP1 dither 1 |
     562| 476404 | o6031g0471o |       20 | 2012-04-14 14:12:12 | STS.180.GIQ PP1 dither 2 |
     563| 476411 | o6031g0478o |       20 | 2012-04-14 14:16:12 | STS.180.GIQ PP1 dither 3 |
     564| 476418 | o6031g0485o |       20 | 2012-04-14 14:20:07 | STS.180.GIQ PP1 dither 4 |
     565| 476425 | o6031g0492o |       20 | 2012-04-14 14:24:14 | STS.180.GIQ PP1 dither 1 |
     566| 476432 | o6031g0499o |       20 | 2012-04-14 14:28:10 | STS.180.GIQ PP1 dither 2 |
     567| 476437 | o6031g0506o |       20 | 2012-04-14 14:32:10 | STS.180.GIQ PP1 dither 3 |
     568| 476445 | o6031g0513o |       20 | 2012-04-14 14:36:14 | STS.180.GIQ PP1 dither 4 |
     569| 476453 | o6031g0520o |       20 | 2012-04-14 14:40:10 | STS.180.GIQ PP1 dither 1 |
     570| 476460 | o6031g0527o |       20 | 2012-04-14 14:44:06 | STS.180.GIQ PP1 dither 2 |
     571| 477549 | o6033g0430o |       30 | 2012-04-16 13:31:05 | STS.180.ND PP1 center    |
     572| 477556 | o6033g0437o |       30 | 2012-04-16 13:36:06 | STS.180.ND PP1 center    |
     573| 477562 | o6033g0444o |       30 | 2012-04-16 13:41:06 | STS.180.ND PP1 center    |
     574| 477569 | o6033g0451o |       30 | 2012-04-16 13:46:09 | STS.180.ND PP1 center    |
     575| 477576 | o6033g0458o |       30 | 2012-04-16 13:51:15 | STS.180.ND PP1 center    |
     576| 477581 | o6033g0465o |       30 | 2012-04-16 13:56:20 | STS.180.ND PP1 center    |
     577| 477588 | o6033g0472o |       30 | 2012-04-16 14:01:24 | STS.180.ND PP1 center    |
     578| 477595 | o6033g0479o |       30 | 2012-04-16 14:06:24 | STS.180.ND PP1 center    |
     579| 477605 | o6033g0486o |       30 | 2012-04-16 14:11:28 | STS.180.ND PP1 center    |
     580| 477612 | o6033g0493o |       30 | 2012-04-16 14:16:30 | STS.180.ND PP1 center    |
     581| 477619 | o6033g0500o |       30 | 2012-04-16 14:21:31 | STS.180.ND PP1 center    |
     582| 477625 | o6033g0507o |       30 | 2012-04-16 14:26:36 | STS.180.ND PP1 center    |
     583| 478479 | o6037g0401o |       20 | 2012-04-20 13:46:26 | STS.180.GIQ PP1 dither 1 |
     584| 478485 | o6037g0408o |       20 | 2012-04-20 13:50:22 | STS.180.GIQ PP1 dither 2 |
     585| 478492 | o6037g0415o |       20 | 2012-04-20 13:54:14 | STS.180.GIQ PP1 dither 3 |
     586| 478499 | o6037g0422o |       20 | 2012-04-20 13:58:10 | STS.180.GIQ PP1 dither 4 |
     587| 478506 | o6037g0429o |       20 | 2012-04-20 14:02:08 | STS.180.GIQ PP1 dither 1 |
     588| 478514 | o6037g0436o |       20 | 2012-04-20 14:06:05 | STS.180.GIQ PP1 dither 2 |
     589| 478520 | o6037g0443o |       20 | 2012-04-20 14:09:58 | STS.180.GIQ PP1 dither 3 |
     590| 478527 | o6037g0450o |       20 | 2012-04-20 14:13:50 | STS.180.GIQ PP1 dither 4 |
     591| 478533 | o6037g0457o |       20 | 2012-04-20 14:17:47 | STS.180.GIQ PP1 dither 1 |
     592| 478542 | o6037g0464o |       20 | 2012-04-20 14:21:42 | STS.180.GIQ PP1 dither 2 |
     593| 478548 | o6037g0471o |       20 | 2012-04-20 14:25:40 | STS.180.GIQ PP1 dither 3 |
     594| 478554 | o6037g0478o |       20 | 2012-04-20 14:29:40 | STS.180.GIQ PP1 dither 4 |
     595| 478561 | o6037g0485o |       20 | 2012-04-20 14:34:36 | STS.180.GIQ PP1 dither 1 |
     596| 478569 | o6037g0492o |       20 | 2012-04-20 14:38:35 | STS.180.GIQ PP1 dither 2 |
     597| 478576 | o6037g0499o |       20 | 2012-04-20 14:42:30 | STS.180.GIQ PP1 dither 3 |
     598| 478582 | o6037g0506o |       20 | 2012-04-20 14:46:32 | STS.180.GIQ PP1 dither 4 |
     599| 479810 | o6040g0441o |       30 | 2012-04-23 13:31:03 | STS.180.ND PP1 center    |
     600| 479818 | o6040g0448o |       30 | 2012-04-23 13:36:04 | STS.180.ND PP1 center    |
     601| 479822 | o6040g0455o |       30 | 2012-04-23 13:41:07 | STS.180.ND PP1 center    |
     602| 479831 | o6040g0462o |       30 | 2012-04-23 13:46:10 | STS.180.ND PP1 center    |
     603| 479838 | o6040g0469o |       30 | 2012-04-23 13:51:11 | STS.180.ND PP1 center    |
     604| 479845 | o6040g0476o |       30 | 2012-04-23 13:56:11 | STS.180.ND PP1 center    |
     605| 479852 | o6040g0483o |       30 | 2012-04-23 14:01:15 | STS.180.ND PP1 center    |
     606| 479859 | o6040g0490o |       30 | 2012-04-23 14:06:17 | STS.180.ND PP1 center    |
     607| 479867 | o6040g0497o |       30 | 2012-04-23 14:11:20 | STS.180.ND PP1 center    |
     608| 479874 | o6040g0504o |       30 | 2012-04-23 14:16:21 | STS.180.ND PP1 center    |
     609| 479880 | o6040g0511o |       30 | 2012-04-23 14:21:21 | STS.180.ND PP1 center    |
     610| 479888 | o6040g0518o |       30 | 2012-04-23 14:26:25 | STS.180.ND PP1 center    |
     611| 482593 | o6045g0420o |       30 | 2012-04-28 13:37:39 | STS.180.ND PP1 center    |
     612| 482600 | o6045g0427o |       30 | 2012-04-28 13:42:41 | STS.180.ND PP1 center    |
     613| 482609 | o6045g0434o |       30 | 2012-04-28 13:47:43 | STS.180.ND PP1 center    |
     614| 482614 | o6045g0441o |       30 | 2012-04-28 13:52:45 | STS.180.ND PP1 center    |
     615| 482620 | o6045g0448o |       30 | 2012-04-28 13:57:46 | STS.180.ND PP1 center    |
     616| 482628 | o6045g0455o |       30 | 2012-04-28 14:02:53 | STS.180.ND PP1 center    |
     617| 482635 | o6045g0462o |       30 | 2012-04-28 14:07:56 | STS.180.ND PP1 center    |
     618| 482642 | o6045g0469o |       30 | 2012-04-28 14:12:59 | STS.180.ND PP1 center    |
     619| 482647 | o6045g0476o |       30 | 2012-04-28 14:18:01 | STS.180.ND PP1 center    |
     620| 482654 | o6045g0483o |       30 | 2012-04-28 14:23:04 | STS.180.ND PP1 center    |
     621| 482661 | o6045g0490o |       30 | 2012-04-28 14:28:11 | STS.180.ND PP1 center    |
     622| 482667 | o6045g0497o |       30 | 2012-04-28 14:33:16 | STS.180.ND PP1 center    |
     623| 483576 | o6049g0286o |       30 | 2012-05-02 13:09:26 | STS.180.ND PP1 center    |
     624| 483583 | o6049g0293o |       30 | 2012-05-02 13:14:27 | STS.180.ND PP1 center    |
     625| 483590 | o6049g0300o |       30 | 2012-05-02 13:19:26 | STS.180.ND PP1 center    |
     626| 483597 | o6049g0307o |       30 | 2012-05-02 13:24:29 | STS.180.ND PP1 center    |
     627| 483604 | o6049g0314o |       30 | 2012-05-02 13:29:35 | STS.180.ND PP1 center    |
     628| 483611 | o6049g0321o |       30 | 2012-05-02 13:34:37 | STS.180.ND PP1 center    |
     629| 483624 | o6049g0335o |       30 | 2012-05-02 13:44:43 | STS.180.ND PP1 center    |
     630| 483630 | o6049g0342o |       30 | 2012-05-02 13:49:48 | STS.180.ND PP1 center    |
     631| 483638 | o6049g0349o |       30 | 2012-05-02 13:54:49 | STS.180.ND PP1 center    |
     632| 483646 | o6049g0356o |       30 | 2012-05-02 13:59:51 | STS.180.ND PP1 center    |
     633| 483651 | o6049g0328o |       30 | 2012-05-02 13:39:38 | STS.180.ND PP1 center    |
     634| 483652 | o6049g0363o |       30 | 2012-05-02 14:04:52 | STS.180.ND PP1 center    |
     635| 484578 | o6054g0305o |       30 | 2012-05-07 12:48:56 | STS.180.ND PP1 center    |
     636| 484590 | o6054g0312o |       30 | 2012-05-07 12:53:58 | STS.180.ND PP1 center    |
     637| 484599 | o6054g0319o |       30 | 2012-05-07 12:58:58 | STS.180.ND PP1 center    |
     638| 484606 | o6054g0326o |       30 | 2012-05-07 13:04:01 | STS.180.ND PP1 center    |
     639| 484613 | o6054g0333o |       30 | 2012-05-07 13:09:03 | STS.180.ND PP1 center    |
     640| 484620 | o6054g0340o |       30 | 2012-05-07 13:14:04 | STS.180.ND PP1 center    |
     641| 484627 | o6054g0347o |       30 | 2012-05-07 13:19:09 | STS.180.ND PP1 center    |
     642| 484634 | o6054g0354o |       30 | 2012-05-07 13:24:13 | STS.180.ND PP1 center    |
     643| 484641 | o6054g0361o |       30 | 2012-05-07 13:29:18 | STS.180.ND PP1 center    |
     644| 484648 | o6054g0368o |       30 | 2012-05-07 13:34:25 | STS.180.ND PP1 center    |
     645| 484656 | o6054g0375o |       30 | 2012-05-07 13:39:31 | STS.180.ND PP1 center    |
     646| 484662 | o6054g0382o |       30 | 2012-05-07 13:44:36 | STS.180.ND PP1 center    |
     647+--------+-------------+----------+---------------------+--------------------------+
     648
     649--> do recent 2 first - first in list - need script liner to get more of these -
     650
     651neb://@HOST@.0/gpc1/STS.nt/2012/05/02/o6049g0286o -- cleaned..
     652neb://@HOST@.0/gpc1/STS.nt/2012/05/07/o6054g0305o
     653
     654--> will want to know FWHM of input warps too
     655
     656
     657
     658neb-ls gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/%unconv%fits
     659nebds9 gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/STS.V4.skycell.1.44.stk.902946.unconv.fits
     660*** 1.44 @4610,5580 region see stars look elongated/trailed versus ones nearby.. looks like multi-cell mask region -- diffims will show if problem
     661--> maybe has to do with GIQ?
     662
     663
     664./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.44.fits  gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/STS.V4.skycell.1.44.stk.902946.unconv.fits 20120507 > runppsub_sts144_120507_1.bat
     665source runppsub_sts144_120507_1.bat >& runppsub_sts144_120507_1.log &
     666
     667./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0312o.484590/o6054g0312o.484590.wrp.408044.skycell.1.44.fits  gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/STS.V4.skycell.1.44.stk.902946.unconv.fits 20120507 > runppsub_sts144_120507_2.bat
     668source runppsub_sts144_120507_2.bat >& runppsub_sts144_120507_2.log &
     669
     670
     671neb-ls gpc1/STS.refstack.20120510/STS.V4/skycell.1.55/%unconv%fits
     672--> also see odd shape near center but is edge field, but has both PP1 and PP2 inputs -
     673
     674./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.55.fits  gpc1/STS.refstack.20120510/STS.V4/skycell.1.55/STS.V4.skycell.1.55.stk.902957.unconv.fits 20120507 > runppsub_sts155_120507_1.bat
     675source runppsub_sts155_120507_1.bat >& runppsub_sts155_120507_1.log &
     676
     677
     678
     679neb-ls gpc1/STS.refstack.20120510/STS.V4/skycell.1.03/%unconv%fits
     680--> looks ok, good overlap mix pp1/4
     681
     682./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.03.fits  gpc1/STS.refstack.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.902905.unconv.fits  20120507 > runppsub_sts103_120507_1.bat
     683source runppsub_sts103_120507_1.bat >& runppsub_sts103_120507_1.log &
     684
     685
     686
     687neb-ls gpc1/STS.refstack.20120510/STS.V4/skycell.1.06/%unconv%fits
     688--> looks ok, crazy overlap mix pp1,2,4 and maybe odd shape in corner
     689
     690./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.06.fits gpc1/STS.refstack.20120510/STS.V4/skycell.1.06/STS.V4.skycell.1.06.stk.902908.unconv.fits 20120507 > runppsub_sts106_120507_1.bat
     691source runppsub_sts106_120507_1.bat >& runppsub_sts106_120507_1.log &
     692
     693
     694*** diffims look like garbage.. MD04 from 5/8 WS look ok - so look at reftest ones as well..
     695--> update/pss request an STS field from 5/2? what was fwhm?
     696SELECT exp_id,exp_name,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id)  WHERE obs_mode="STS" and dateobs>"2012-01-01" and comment like "%PP1%" group by exp_id;
     697--> next one on 5/7 is better fwhm so try that first - same
     698--> has ND in comment name -- that mean anything?
     699--> try 4/20 o6037g0506o -- will need to update - warp_id=400804, skycell.1.44 bundle -- FWHM~3.8
     700--> better, at least messy resid but the things that look like diffim det look odd, some look like things removed in pixel rejection??
     701
     702gpc1/STS.nt/2012/04/20/o6037g0506o.478582/o6037g0506o.478582.wrp.400804.skycell.1.44.fits
     703
     704./run_ppsub_night_ws.csh gpc1/STS.nt/2012/04/20/o6037g0506o.478582/o6037g0506o.478582.wrp.400804.skycell.1.44.fits  gpc1/STS.refstack.20120510/STS.V4/skycell.1.44/STS.V4.skycell.1.44.stk.902946.unconv.fits 20120420 > runppsub_sts144_120420_x.bat
     705source runppsub_sts144_120420_x.bat >& runppsub_sts144_120420_x.log &
     706
     707--> need to retry diffims above as conv stacks too - seems the same, how is that..
     708
     709--> need to look at the mask images as well -
     710
     711
     712--> look at reftest diffims
     713
     714mysql> select stack_id,data_group,skycell_id from stackRun where data_group like "STS.reftest%c%f%";
     715+----------+----------------------------+--------------+
     716| stack_id | data_group                 | skycell_id   |
     717+----------+----------------------------+--------------+
     718|   899920 | STS.reftest.20120510.c0f45 | skycell.1.03 |
     719|   899919 | STS.reftest.20120510.c1f37 | skycell.1.03 |
     720|   899918 | STS.reftest.20120510.c1f38 | skycell.1.03 |
     721|   899917 | STS.reftest.20120510.c1f39 | skycell.1.03 |
     722|   904603 | STS.reftest.20120510.c2f36 | skycell.1.55 |
     723|   904604 | STS.reftest.20120510.c2f37 | skycell.1.55 |
     724|   904605 | STS.reftest.20120510.c2f37 | skycell.1.40 |
     725|   904606 | STS.reftest.20120510.c2f37 | skycell.1.35 |
     726|   904599 | STS.reftest.20120510.c2f39 | skycell.1.35 |
     727|   904601 | STS.reftest.20120510.c2f39 | skycell.1.40 |
     728|   904600 | STS.reftest.20120510.c2f45 | skycell.1.35 |
     729|   904602 | STS.reftest.20120510.c2f45 | skycell.1.40 |
     730
     731gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899917.unconv.fits
     732gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899918.unconv.fits
     733gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899919.unconv.fits
     734gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899920.unconv.fits
     735
     736gpc1/STS.reftest.20120510/STS.V4/skycell.1.35/STS.V4.skycell.1.35.stk.904599.unconv.fits
     737gpc1/STS.reftest.20120510/STS.V4/skycell.1.35/STS.V4.skycell.1.35.stk.904600.unconv.fits
     738gpc1/STS.reftest.20120510/STS.V4/skycell.1.35/STS.V4.skycell.1.35.stk.904606.unconv.fits
     739
     740gpc1/STS.reftest.20120510/STS.V4/skycell.1.55/STS.V4.skycell.1.55.stk.904603.unconv.fits
     741gpc1/STS.reftest.20120510/STS.V4/skycell.1.55/STS.V4.skycell.1.55.stk.904604.unconv.fits
     742
     743gpc1/STS.reftest.20120510/STS.V4/skycell.1.40/STS.V4.skycell.1.40.stk.904601.unconv.fits
     744gpc1/STS.reftest.20120510/STS.V4/skycell.1.40/STS.V4.skycell.1.40.stk.904602.unconv.fits
     745gpc1/STS.reftest.20120510/STS.V4/skycell.1.40/STS.V4.skycell.1.40.stk.904605.unconv.fits
     746
     747./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.03.fits  gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899918.unconv.fits 20120507_c1f38 > runppsub_sts103_120507_c1f38.bat
     748source runppsub_sts103_120507_c1f38.bat >& runppsub_sts103_120507_c1f38.log &
     749
     750--> ack.. been cleaned and no others since.. will need to update chips too.. so tedious, use pss -
     751//warptool -dbname gpc1 -setskyfiletoupdate -warp_id 408038 -skycell_id skycell.1.03 -set_label
     752--> need to copy over locally -
     753
     754--> for conv, edit the .bat, make WSc and drop the unconv
     755cp runppsub_sts103_120507_c1f38.bat runppsub_sts103_120507_c1f38c.bat
     756source runppsub_sts103_120507_c1f38c.bat >& runppsub_sts103_120507_c1f38c.log &
     757
     758--> same, so guess that is good.. try c1f37 -
     759./run_ppsub_night_ws.csh gpc1/STS.nt/2012/05/07/o6054g0305o.484578/o6054g0305o.484578.wrp.408038.skycell.1.03.fits gpc1/STS.reftest.20120510/STS.V4/skycell.1.03/STS.V4.skycell.1.03.stk.899919.unconv.fits  20120507_c1f37 > runppsub_sts103_120507_c1f37.bat
     760source runppsub_sts103_120507_c1f37.bat >& runppsub_sts103_120507_c1f37.log &
     761
     762--> same.. are the exp_time different? yes 4/20 is 20s.. why would that matter.. IQ is better too, so maybe recent ones are just wacky IQ?
     763
     764
     765-- need to run psphot on the refstack - staticsky? no that would be a mess and only 1 filter anyways..
     766--> then can make
     767
     768
     769
    513770
    514771{{{
    … …  
    529786Rerun of sample diffims for projection 1 and published under IPP-MOPS-TEST-2
    530787
    531 
     788Need to update chip->warp of (re)selected nights for testing
     789 * like before avoid dates used in refstack, try sample similar/better FWHM and worse
    532790{{{
    533791SELECT exp_id,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) join fakeRun using (cam_id) join warpRun using (fake_id) WHERE warpRun.data_group like "STS.refstack.20120510" and comment like "%PP1%" group by exp_id;
    534 }}}
    535 
     792
     793SELECT exp_id,exp_name,exp_time,dateobs,zpt_obs,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id)  WHERE  comment like "STS%PP1%" group by exp_id;
     794
     795| 400812 | o5834g0061o |       30 | 2011-09-30 06:48:31 |  24.643 |     4.9952 | 1.117 | 0.90        | 137.86   | 16.97    | 5.30  | 4.83  | 6509204 | STS.090.ND PP1 center    |
     796| 400819 | o5834g0068o |       30 | 2011-09-30 06:53:41 | 24.6431 |      5.927 | 1.198 | 1.38        | 131.72   | 16.92    | 6.25  | 5.55  | 5929444 | STS.090.ND PP1 center    |
     797| 400826 | o5834g0075o |       30 | 2011-09-30 06:58:46 | 24.6413 |    5.41098 | 1.165 | 1.20        | 123.85   | 16.34    | 5.69  | 5.22  | 6411952 | STS.090.ND PP1 center    |
     798| 400833 | o5834g0082o |       30 | 2011-09-30 07:03:51 | 24.6297 |    6.84521 | 1.120 | 1.16        | 121.33   | 17.11    | 7.07  | 6.75  | 4591816 | STS.090.ND PP1 center    |
     799| 400840 | o5834g0089o |       30 | 2011-09-30 07:08:53 | 24.6256 |    6.46738 | 1.069 | 0.84        | 126.92   | 17.59    | 6.74  | 6.23  | 4562012 | STS.090.ND PP1 center    |
     800| 400847 | o5834g0096o |       30 | 2011-09-30 07:13:55 | 24.6386 |    6.42677 | 1.196 | 1.36        | 125.30   | 16.89    | 6.75  | 6.22  | 5325836 | STS.090.ND PP1 center    |
     801| 400854 | o5834g0103o |       30 | 2011-09-30 07:19:00 | 24.6292 |    6.37093 | 1.103 | 0.88        | 123.32   | 17.25    | 6.67  | 6.13  | 4688216 | STS.090.ND PP1 center    |
     802| 400861 | o5834g0110o |       30 | 2011-09-30 07:24:07 | 24.6316 |    6.62746 | 1.180 | 1.04        | 131.42   | 17.69    | 6.82  | 6.42  | 4652948 | STS.090.ND PP1 center    |
     803| 400868 | o5834g0117o |       30 | 2011-09-30 07:29:12 | 24.6365 |    6.15409 | 1.085 | 1.05        | 128.51   | 17.22    | 6.53  | 5.92  | 5107028 | STS.090.ND PP1 center    |
     804| 400875 | o5834g0124o |       30 | 2011-09-30 07:34:21 |  24.641 |     5.2498 | 1.127 | 0.91        | 123.12   | 16.49    | 5.56  | 5.03  | 6108640 | STS.090.ND PP1 center    |
     805| 400883 | o5834g0131o |       30 | 2011-09-30 07:39:28 | 24.6409 |    5.78907 | 1.154 | 1.19        | 127.42   | 16.82    | 6.00  | 5.65  | 5809232 | STS.090.ND PP1 center    |
     806| 400889 | o5834g0138o |       30 | 2011-09-30 07:44:33 | 24.6389 |    6.12522 | 1.064 | 1.12        | 125.19   | 17.02    | 6.48  | 5.96  | 5185056 | STS.090.ND PP1 center    |
     807
     808| 475121 | o6029g0441o |       30 | 2012-04-12 13:40:09 | 24.5321 |    4.06258 | 1.119 | 1.05        | 315.49   | 21.55    | 4.21  | 3.82  | 8078200 | STS.180.ND PP1 center    |
     809| 475128 | o6029g0448o |       30 | 2012-04-12 13:45:11 | 24.5325 |    3.91594 | 1.132 | 0.95        | 303.82   | 21.37    | 4.41  | 3.67  | 8132400 | STS.180.ND PP1 center    |
     810| 475135 | o6029g0455o |       30 | 2012-04-12 13:50:14 | 24.5375 |    4.06348 | 1.129 | 0.97        | 307.70   | 21.58    | 4.48  | 3.88  | 7907516 | STS.180.ND PP1 center    |
     811| 475141 | o6029g0462o |       30 | 2012-04-12 13:55:16 | 24.5386 |     4.0157 | 1.135 | 0.97        | 301.07   | 21.32    | 4.55  | 3.78  | 8000044 | STS.180.ND PP1 center    |
     812| 475149 | o6029g0469o |       30 | 2012-04-12 14:00:20 | 24.5421 |    4.29222 | 1.196 | 1.03        | 300.51   | 21.44    | 4.66  | 4.07  | 7669568 | STS.180.ND PP1 center    |
     813| 475156 | o6029g0476o |       30 | 2012-04-12 14:05:25 | 24.5459 |    4.32599 | 1.163 | 1.03        | 304.46   | 21.61    | 4.80  | 4.07  | 7576116 | STS.180.ND PP1 center    |
     814| 475163 | o6029g0483o |       30 | 2012-04-12 14:10:27 | 24.5445 |    3.90778 | 1.119 | 0.92        | 308.26   | 21.57    | 4.16  | 3.71  | 8114964 | STS.180.ND PP1 center    |
     815| 475170 | o6029g0490o |       30 | 2012-04-12 14:15:31 | 24.5486 |    4.25772 | 1.135 | 0.96        | 295.86   | 21.39    | 4.70  | 4.02  | 7605516 | STS.180.ND PP1 center    |
     816| 475177 | o6029g0497o |       30 | 2012-04-12 14:20:35 | 24.5477 |    4.11544 | 1.203 | 1.09        | 378.52   | 23.53    | 4.35  | 3.77  | 7926572 | STS.180.ND PP1 center    |
     817| 475184 | o6029g0504o |       30 | 2012-04-12 14:25:37 | 24.5461 |    3.88865 | 1.175 | 1.00        | 293.10   | 21.07    | 4.35  | 3.62  | 8464032 | STS.180.ND PP1 center    |
     818| 475191 | o6029g0511o |       30 | 2012-04-12 14:30:41 |  24.552 |    4.19097 | 1.172 | 1.10        | 277.41   | 20.75    | 4.38  | 3.89  | 8060959 | STS.180.ND PP1 center    |
     819| 475198 | o6029g0518o |       30 | 2012-04-12 14:35:47 | 24.5526 |    4.09804 | 1.174 | 1.05        | 280.17   | 20.83    | 4.36  | 3.79  | 8122168 | STS.180.ND PP1 center    |
     820
     821| 476338 | o6031g0408o |       20 | 2012-04-14 13:34:26 | 24.5537 |    4.27532 | 1.180 | 0.98        | 117.51   | 14.15    | 4.76  | 4.05  | 7485604 | STS.180.GIQ PP1 dither 1 |
     822| 476345 | o6031g0415o |       20 | 2012-04-14 13:38:20 | 24.5533 |    4.17247 | 1.184 | 1.01        | 117.49   | 14.21    | 4.67  | 3.89  | 7636200 | STS.180.GIQ PP1 dither 2 |
     823| 476352 | o6031g0422o |       20 | 2012-04-14 13:42:15 |  24.552 |    4.09387 | 1.144 | 0.94        | 118.19   | 14.27    | 4.45  | 3.90  | 7637616 | STS.180.GIQ PP1 dither 3 |
     824| 476360 | o6031g0429o |       20 | 2012-04-14 13:46:09 |  24.556 |    4.31094 | 1.188 | 1.01        | 118.09   | 14.37    | 4.70  | 4.05  | 7378484 | STS.180.GIQ PP1 dither 4 |
     825| 476366 | o6031g0436o |       20 | 2012-04-14 13:50:05 | 24.5551 |    4.17617 | 1.154 | 0.94        | 119.08   | 14.43    | 4.59  | 3.98  | 7443824 | STS.180.GIQ PP1 dither 1 |
     826| 476373 | o6031g0443o |       20 | 2012-04-14 13:53:56 | 24.5525 |    3.82054 | 1.166 | 0.84        | 121.42   | 14.46    | 4.30  | 3.63  | 7964096 | STS.180.GIQ PP1 dither 2 |
     827| 476380 | o6031g0450o |       20 | 2012-04-14 13:57:48 | 24.5545 |    4.07024 | 1.146 | 0.98        | 123.89   | 14.60    | 4.58  | 3.78  | 7718292 | STS.180.GIQ PP1 dither 3 |
     828| 476389 | o6031g0457o |       20 | 2012-04-14 14:01:41 | 24.5365 |     4.0109 | 1.144 | 0.93        | 133.13   | 15.08    | 4.49  | 3.79  | 7568984 | STS.180.GIQ PP1 dither 4 |
     829| 476397 | o6031g0464o |       20 | 2012-04-14 14:08:15 | 24.5432 |     3.9738 | 1.177 | 0.95        | 131.38   | 14.95    | 4.33  | 3.76  | 7835456 | STS.180.GIQ PP1 dither 1 |
     830| 476404 | o6031g0471o |       20 | 2012-04-14 14:12:12 | 24.5149 |    3.94952 | 1.206 | 0.97        | 138.51   | 15.23    | 4.44  | 3.68  | 7840492 | STS.180.GIQ PP1 dither 2 |
     831| 476411 | o6031g0478o |       20 | 2012-04-14 14:16:12 | 24.5435 |    3.80814 | 1.181 | 0.90        | 126.95   | 14.66    | 4.29  | 3.58  | 8082288 | STS.180.GIQ PP1 dither 3 |
     832| 476418 | o6031g0485o |       20 | 2012-04-14 14:20:07 | 24.5456 |    3.77643 | 1.189 | 0.92        | 128.77   | 14.71    | 4.27  | 3.54  | 8231892 | STS.180.GIQ PP1 dither 4 |
     833| 476425 | o6031g0492o |       20 | 2012-04-14 14:24:14 | 24.5303 |     4.1106 | 1.144 | 0.98        | 139.08   | 15.35    | 4.59  | 3.83  | 7461452 | STS.180.GIQ PP1 dither 1 |
     834| 476432 | o6031g0499o |       20 | 2012-04-14 14:28:10 | 24.5157 |     3.9894 | 1.202 | 0.93        | 143.64   | 15.48    | 4.33  | 3.72  | 7628256 | STS.180.GIQ PP1 dither 2 |
     835| 476437 | o6031g0506o |       20 | 2012-04-14 14:32:10 | 24.4737 |    4.25366 | 1.151 | 0.99        | 157.29   | 15.95    | 4.71  | 3.96  | 7066552 | STS.180.GIQ PP1 dither 3 |
     836| 476445 | o6031g0513o |       20 | 2012-04-14 14:36:14 | 24.5465 |    3.89488 | 1.188 | 0.89        | 139.57   | 15.33    | 4.43  | 3.60  | 7892495 | STS.180.GIQ PP1 dither 4 |
     837| 476453 | o6031g0520o |       20 | 2012-04-14 14:40:10 |  24.555 |    3.64675 | 1.198 | 0.87        | 136.07   | 15.05    | 4.17  | 3.37  | 8450820 | STS.180.GIQ PP1 dither 1 |
     838| 476460 | o6031g0527o |       20 | 2012-04-14 14:44:06 | 24.4858 |    3.62349 | 1.210 | 0.85        | 166.07   | 16.03    | 4.25  | 3.28  | 8069508 | STS.180.GIQ PP1 dither 2 |
     839
     840
     841}}}
     842
     843 * 20110930:
    536844{{{
    537845mysql -hippdb03 -uippuser -pxxxx gpc1 -B -e 'select distinct exp_name,dateobs,chip_id,warp_id from warpRun join fakeRun using (fake_id) join camRun using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where comment like "STS%PP1%" and dateobs like "2011-09-30%" and warpRun.state like "cleaned" and chipRun.state like "cleaned" group by exp_name; ' | grep -v chip_id | awk '{print "chiptool -dbname gpc1 -setimfiletoupdate -set_label STS.mopsdiffv2.mehtest -chip_id "$4"; warptool -dbname gpc1  -setskyfiletoupdate -set_label STS.mopsdiffv2.mehtest -warp_id  "$5}' > ~/updatechipwarp_sts.mopsdiffv2.mehtest_110930.bat
    … …  
    542850
    543851
     852
     853Setup for publishing:
     854 * check but DONT run -- would get sent to 5 as well as 12..
     855{{{
     856pubtool -dbname gpc1 -definerun -label STS.mopsdiffv2.mehtest -simple -pretend
     857
     858--> may need to add the date to the survey task list.. either that or manually load it with pubtool, need to figure out how to only do 12
     859ns.del.date, ns.add.date
     860ns.set.date 2010-09-11 TOWARP -- one for pub?
     861--> seems like a bad idea.. so how about pubtool - basically calls from
     862ipp-localops.lin64/share/pantasks/modules/survey.pro
     863pubtool -definerun -label $label -client_id $client_id
     864
     865pubtool -dbname gpc1 -definerun -label STS.mopsdiffv2.mehtest -client_id 12 -simple -pretend
     866--> seems to work and only put to 12 -- what happened before then??
     867--> and works but just pos detections - thats what MD with diff_mode 2 has for WS so.. OSS now using SWEETSPOT (noconv) and diff_mode 1
     868
     869-- works so setup next 2012 sample and 2011 sample then -- try pick 2011 w/o many in stack -
     870
     871SELECT exp_id,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) join fakeRun using (cam_id) join warpRun using (fake_id) WHERE warpRun.data_group like "STS.refstack.20120510" and comment like "%PP1%" group by exp_id;
     872--> only 1 from 9/19 used so use that date? 9/11 better and none used from it - or 9/30 to have sample of worse input (yes)
     873--> so 4/14 after 4/12 for close in time observ and still pretty good -- would prefer one not as good..
     874
     875mysql -hippdb01 -uippuser -pxxxx gpc1 -B -e 'select distinct exp_name,dateobs,chip_id,warp_id from warpRun join fakeRun using (fake_id) join camRun using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where comment like "STS%PP1%" and dateobs like "2011-09-30%" and warpRun.state like "cleaned" and chipRun.state like "cleaned" group by exp_name; ' | grep -v chip_id | awk '{print "chiptool -dbname gpc1 -setimfiletoupdate -set_label STS.mopsdiffv2.mehtest -chip_id "$4"; warptool -dbname gpc1  -setskyfiletoupdate -set_label STS.mopsdiffv2.mehtest -warp_id  "$5}' > ~/updatechipwarp_sts.mopsdiffv2.mehtest_110930.bat
     876
     877mysql -hippdb01 -uippuser -pxxxx gpc1 -B -e 'select distinct exp_name,dateobs,chip_id,warp_id from warpRun join fakeRun using (fake_id) join camRun using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where comment like "STS%PP1%" and dateobs like "2012-04-14%" and warpRun.state like "cleaned" and chipRun.state like "cleaned" group by exp_name; ' | grep -v chip_id | awk '{print "chiptool -dbname gpc1 -setimfiletoupdate -set_label STS.mopsdiffv2.mehtest -chip_id "$4"; warptool -dbname gpc1  -setskyfiletoupdate -set_label STS.mopsdiffv2.mehtest -warp_id  "$5}' > ~/updatechipwarp_sts.mopsdiffv2.mehtest_120414.bat
     878
     879difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -pretend
     880}}}
     881
     882
     883
     884
     885* 9/30 only 1 is going to go through diffs for some reason.  because diffs make last year.. but were WW diffims
    544886{{{
    545887select * from diffRun where data_group like "STS.20110930";
    546888
     889--> try with -rerun -- yes seems to be there, runs then all -- seems rerun works to run new type of diffim but not rerun old one?
     890
    547891difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -rerun -pretend
    548892
    549893pubtool -dbname gpc1 -definerun -label STS.mopsdiffv2.mehtest -client_id 12 -simple -pretend
    550894
    551 }}}
     895--> seems to have worked, out onto datastore now for MOPS to look at. 
     896}}}
     897
     898
    552899
    553900|| ||
    … …  
    558905
    559906
    560