Changes between Version 3 and Version 4 of PS1_IPP_Czarlog_20190426
- Timestamp:
- Apr 29, 2019, 9:32:43 AM (7 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
PS1_IPP_Czarlog_20190426
v3 v4 19 19 Reporting missing .mdc file, so it fails in the burntool stage, so I reprocessed it (really should have tried updating, this was in the morning) 20 20 Will reprocess: 21 {{{ chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190428 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190428 -set_reduction LAP_SCIENCE -exp_name o5444g0110o}}} 21 {{{ 22 chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190428 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190428 -set_reduction LAP_SCIENCE -exp_name o5444g0110o 23 }}} 22 24 Check they're finished: 23 {{{SELECT rawExp.exp_name, rawExp.exp_id, chipRun.chip_id, chipRun.label chip_label, chipRun.state chip_state FROM rawExp JOIN chipRun USING (exp_id) WHERE exp_name IN ("o5444g0110o");}}} 25 {{{ 26 SELECT rawExp.exp_name, rawExp.exp_id, chipRun.chip_id, chipRun.label chip_label, chipRun.state chip_state FROM rawExp JOIN chipRun USING (exp_id) WHERE exp_name IN ("o5444g0110o"); 27 }}} 24 28 25 29 Now the mopsfix job is stuck as new in the chip stage, complaining about the burntool: 26 {{{o5444g0110o XY10 219612 2142808 new mopsfix mops.fixbrokenLAP.20190428 NULL 2}}} 30 {{{ 31 o5444g0110o XY10 219612 2142808 new mopsfix mops.fixbrokenLAP.20190428 NULL 2 32 }}} 27 33 28 34 Perhaps try: 29 {{{ipp_apply_burntool_fix.pl --exp_name o5444g0110o --class_id XY10 --verbose --dbname gpc1}}} 35 {{{ 36 ipp_apply_burntool_fix.pl --exp_name o5444g0110o --class_id XY10 --verbose --dbname gpc1 37 }}} 30 38 31 39 Tell it to update? 32 {{{chiptool -dbname gpc1 -setimfiletoupdate -set_label mopsfix -chip_id 2142808}}} 40 {{{ 41 chiptool -dbname gpc1 -setimfiletoupdate -set_label mopsfix -chip_id 2142808 42 }}} 33 43 34 44 … … 37 47 38 48 Try to update them first: 39 {{{chiptool -dbname gpc1 -setimfiletoupdate -set_label ps_ud_WEB -chip_id XXXXXX 40 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id XXXXXX}}} 49 {{{ 50 chiptool -dbname gpc1 -setimfiletoupdate -set_label ps_ud_WEB -chip_id XXXXXX 51 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id XXXXXX 52 }}} 41 53 42 54 One of the ones sent to update came back complaining about a missing file, but it is there, so try again: 43 {{{o6224g0254o 537186 1281637 1295352 1270256 1276893 skycell.1060.063 update ps_ud_WEB 55 {{{ 56 o6224g0254o 537186 1281637 1295352 1270256 1276893 skycell.1060.063 update ps_ud_WEB 44 57 chiptool -dbname gpc1 -setimfiletoupdate -set_label ps_ud_WEB -chip_id 1281637 45 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1276893}}} 58 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1276893 59 }}} 46 60 47 61 Failing that, reprocess them: 48 {{{chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190428 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190428 -set_reduction LAP_SCIENCE -exp_name}}} 62 {{{ 63 chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190428 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190428 -set_reduction LAP_SCIENCE -exp_name 64 }}} 49 65 have sent them to reprocess, check status soon! 50 66 … … 59 75 It still complains about being unable to find a file, but nebulous says it is there, and I have checked that the file is there and it does exists (and looks a reasonable size). 60 76 Cheeky tip -> set to clean, then set it back to update to clean up the stuck ones. 61 {{{chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 1281637 62 warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 1276893}}} 77 {{{ 78 chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 1281637 79 warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 1276893 80 }}} 63 81 Then, set to update again: 64 {{{chiptool -dbname gpc1 -setimfiletoupdate -set_label ps_ud_WEB -chip_id 1281637 65 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1276893}}} 82 {{{ 83 chiptool -dbname gpc1 -setimfiletoupdate -set_label ps_ud_WEB -chip_id 1281637 84 warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1276893 85 }}} 66 86 67 87 … … 86 106 87 107 88 89 90 91 108 '''missing 2 darks''' by JRF 92 109 … … 94 111 95 112 Now, this is where I made a mistake. '''DO NOT DO THIS''' 96 {{{check_server.sh start.server summitcopy 113 {{{ 114 check_server.sh start.server summitcopy 97 115 }}} 98 116 Turns out it does not accept an argument and actually just starts all the servers...and I already had all the servers running... … … 101 119 So now I'm going through all the machines that have the pantasks_servers on them and clearing them out. In general just log into pantasks for each service and: 102 120 {{{stop 103 shutdown now}}} 121 shutdown now 122 }}} 104 123 make sure the jobs have actually cleared before shutting down. Do this until all the servers are gone. Then go to the correct machine, and start it up: 105 124 {{{ … … 107 126 server input input 108 127 setup 109 run}}} 128 run 129 }}} 110 130 Note that you need to be in the correct directory, and logged in as the correct user. 111 131
