| | 1 | == 2014-02-04 == |
| | 2 | |
| | 3 | Due to the limitations with srun, it's likely going to be easier in the future to use stask instead. This isn't a major change in the scripting, as it's simply an issue of switching the syntax around. |
| | 4 | |
| | 5 | The more major shift is going to be in making sure we use the system efficiently. The scheduling system is designed to run jobs that execute for 8-12 hours, which the current single stage-single exposure system doesn't do. In addition, processors are allocated as full nodes (24 cores), so it's most efficient to use all of them. A quick calculation suggests that an 8 hour, 5 node block could process ~525 exposures through chip, ~2750 through camera, or ~1000 through warp (based on some quick averages from the database dtime values). Launching jobs of this scale requires some rethinking of how to stage things. |
| | 6 | |
| | 7 | I still think holding the remoteRun fixed to a single mrun command is best. However, instead of the remoteRun defining everything, it needs to have child rows in a different table (remoteExp?) that points to the stage, stage_id, file lists, etc that are contained in the remoteRun. This will need some work to redevelop. |
| | 8 | |
