| | 1 | == 2014-02-07 == |
| | 2 | |
| | 3 | I think that the stask interface is working now, although I still would like to try and sort out the retry code (I have some untested changes for my next test). However, extending the current scripts from the single exposure to the large N format is not really possible without the database changes, so I'm making the final changes to the database schema and tool. I'd like to fold this change into the working database on Monday, which will allow me to finish the scripts early next week. |
| | 4 | |
| | 5 | |
| | 6 | * remoteRun |
| | 7 | * remote_id |
| | 8 | * state |
| | 9 | * stage |
| | 10 | * path_base |
| | 11 | * job_id |
| | 12 | * last_poll |
| | 13 | * fault |
| | 14 | * remoteComponent |
| | 15 | * remote_id |
| | 16 | * stage_id |
| | 17 | |
| | 18 | With this schema, a remote job contains only one stage, a simplification largely done to be able to keep the different stages in different scripts. There's no absolute limit on the number of components that are included in a given job, so this can be tuned to whatever size is most efficient to work with. I still expect to do a SAS run once smaller tests are complete, and given the current discussions, we may simply just put the 620 exposures in this set into a single remoteRun entry (for each stage), and allocate enough processing to do it in one pass. The processing structure is now: |
| | 19 | |
| | 20 | * Define chip jobs manually as we do now. |
| | 21 | * Define remoteRun/Component entries: remotetool -definebyquery -label asdf -stage chip -limit N |
| | 22 | * Prepare the run: sc_prepare_chip.pl --remote_id X |
| | 23 | * Execute the run: sc_remote_exec.pl --remote_id X |
| | 24 | * Poll for completion: sc_remote_exec.pl --poll --remote_id X |
| | 25 | * Retrieve the completion status for the chipRun level updates. |
| | 26 | * Do any file transfers needed. This may need to simply be scheduled. |
| | 27 | * Mark remoteRun as full |
| | 28 | * Complete chipRuns auto-queue replacement camRun entries, which are subsequently polled by remotetool -definebyquery -stage cam |
| | 29 | |
