| Version 4 (modified by , 8 years ago) ( diff ) |
|---|
The recent time SNAFU of silently changing a telescope header keyword for SHUTOUTC because system /etc/localtime was incorrect (PS1_IPP_Czarlog_20150810, PS1_timing_problems) has highlighted the need for a regular/daily test sample of IPP processing... Ken has agreed this is necessary ASAP.
MOPS is willing to do the extra work to select and analyze the test set, the IPP should be willing to also process and provide it. In the near future a set of WS diffs for MOPS+QUB should also be added as it could help to highlight other problems.
Issues:
- original case was to catch odd node issue, needs to touch on all nightly science nodes (if >60 then some might not be checked) -- need to check hostname for any missed
- ...
Initial/current test quad -- 162.OSS.O.Q.w (2013-03-19) exposures -- o6370g0351o, o6370g0373o, o6370g0395o, o6370g0417o
+-------------+---------------------+----------+---------+-----------------------------------+------------+-----------+----------+ | exp_name | dateobs | exp_time | filter | comment | posang | radeg | decdeg | +-------------+---------------------+----------+---------+-----------------------------------+------------+-----------+----------+ | o6370g0351o | 2013-03-19 11:37:26 | 45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 1 | 337.868713 | 186.44580 | -2.16131 | | o6370g0373o | 2013-03-19 11:59:29 | 45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 2 | 337.876892 | 186.44597 | -2.16143 | | o6370g0395o | 2013-03-19 12:21:39 | 45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 3 | 337.877411 | 186.44591 | -2.16137 | | o6370g0417o | 2013-03-19 12:43:52 | 45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 4 | 337.877594 | 186.44574 | -2.16125 | +--------+-------------+-------+------------+---------+---------+ | exp_id | exp_name | fault | fwhm_major | quality | deteff | +--------+-------------+-------+------------+---------+---------+ | 590994 | o6370g0351o | 0 | 5.26383 | 0 | 22.7106 | | 591016 | o6370g0373o | 0 | 5.99344 | 0 | 22.4768 | | 591038 | o6370g0395o | 0 | 6.14446 | 0 | 22.4466 | | 591060 | o6370g0417o | 0 | 6.48741 | 0 | 22.3831 |
Manually to run for code change test (never use a script except for generating the base commands) -- normal chip/cam/warp, WWdiff v12,v34 and WSdiff, publishing
- should be run in account and tag to reflect what is being tested
- chip-cam-warp -- include date and revision being tested
chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.DATErN -set_data_group MOPS.dailytestset.DATErN -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0351o -pretend
- WWdiffs to queue -- o6370g0351o-o6370g0373o, o6370g0395o-o6370g0417o (warp_id - template_warp_id)
difftool -dbname gpc1 -definewarpwarp -warp_id VISIT1/VISIT3 -template_warp_id VISIT2/VISIT4 -backwards -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.DATErN -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.DATErN -set_reduction SWEETSPOT -simple -rerun -pretend
- WSdiffs to queue -- testing new w-band reference stack, currently only run for testing that
difftool -dbname gpc1 -definewarpstack -good_frac 0.05 -warp_id WARPID -stack_label PSNSC.wref.20170707 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.WS.dailytestset.DATErN -set_workdir neb://@HOST@.0/gpc1/MOPS.WS.dailytestset.DATErN -set_reduction SWEETSPOT_WS -simple -rerun -pretend
- checks before publishing -- skycell count, good skycell count, number of detections per skycell...
- publishing -- using IPP-MOPS-TEST-2, client_id 12 until time to restore the IPP-MOPS-dailytestset location...
pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.dailytestset.DATErN -client_id 12 -simple -pretend ////pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.WS.dailytestset.DATErN -client_id 12 -simple -pretend
- publishing list email to Serge/MOPS -- should be 4 files, 2 .pos. and 2 .neg.
mysql -hippdb08 -uippuser -p gpc1 -B -e ' select path_base from publishRun join publishDone using (pub_id) join diffRun where publishRun.label like "MOPS.dailytestset" and stage_id=diff_id and diffRun.data_group like "MOPS.dailytestset.DATErN"; ' | grep -v path_base | awk '{print "neb-ls -p "$1"%.mops"}' | tcsh
- cleanup -- should never do for a normal test set unless setup for testing updates etc
Initial run description (until script finished) for daily test to run in the current nightly processing account and ops tag -- normal chip/cam/warp, WWdiff v12,v34 and publishing
- chip-cam-warp commands --
chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_data_group MOPS.dailytestset.20150815 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0351o -pretend chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_data_group MOPS.dailytestset.20150815 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0373o -pretend chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_data_group MOPS.dailytestset.20150815 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0395o -pretend chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_data_group MOPS.dailytestset.20150815 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0417o -pretend
- WWdiffs to queue -- o6370g0351o-o6370g0373o, o6370g0395o-o6370g0417o (warp_id - template_warp_id)
difftool -dbname gpc1 -definewarpwarp -warp_id 1615271 -template_warp_id 1615273 -backwards -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.20150815 -set_reduction SWEETSPOT -simple -pretend difftool -dbname gpc1 -definewarpwarp -warp_id 1615268 -template_warp_id 1615270 -backwards -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.20150815 -set_reduction SWEETSPOT -simple -pretend
- WSdiffs -- not ready, still in testing with new w-band refstack
difftool -dbname gpc1 -definewarpstack -good_frac 0.05 -warp_id WARPID -stack_label PSNSC.wref.20170707 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.WS.dailytestset.DATE -set_workdir neb://@HOST@.0/gpc1/MOPS.WS.dailytestset.DATE -set_reduction SWEETSPOT_WS -simple -rerun -pretend
- publication location on datastore -- IPP-MOPS-dailytestset, client_id 16 -- was lost in the move to ITC and needs to be restored, likely needs to just point to ipp113 in db
pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.dailytestset.20150815 -client_id 16 -simple -pretend
- cleanup -- no cleanup until setup worked out, then likely <12 hours is acceptable
Summary of past runs -- ...
Note:
See TracWiki
for help on using the wiki.
