Changes between Version 8 and Version 9 of LAP_Science

Timestamp:

May 3, 2011, 5:45:41 PM (15 years ago)

Author:

watersc1

Comment:

--

LAP_Science

@@ -41 +41 @@
 == Sequence definition ==
 
-A `lapSequence` is defined by calling `laptool -definesequence -name "unique name" -description "long description explaining what this sequence is"`.  `laptool -listsequence` prints the information about sequences in the metadata format. 
+A `lapSequence` is defined by calling 
+{{{
+laptool -definesequence -name "unique name" -description "long description explaining what this sequence is"
+}}}
+`laptool -listsequence` prints the information about sequences in the metadata format. 
 
 == Run definition ==
 
-A new `lapRun` is defined by calling `laptool -definerun -seq_id S -projection_cell PC_NAME -ra RA_CENTER -decl DEC_CENTER -radius R -filter F -label L`.  This adds a row to `lapRun`, and then creates a row in `lapExp` for each exposure that falls within the area on the sky defined by the -ra/-decl/-radius options.  If these exposures have been previously processed (and therefore have a row in `lapExp` associated with a different `lapRun`), the `chip_id` for the new lapExp is copied from that previous entry, provided that the `private` flag is not true.  
+A new `lapRun` is defined by calling 
+{{{
+laptool -definerun -seq_id S -projection_cell PC_NAME -ra RA_CENTER -decl DEC_CENTER -radius R -filter F -label L
+}}}
+This adds a row to `lapRun`, and then creates a row in `lapExp` for each exposure that falls within the area on the sky defined by the -ra/-decl/-radius options.  If these exposures have been previously processed (and therefore have a row in `lapExp` associated with a different `lapRun`), the `chip_id` for the new lapExp is copied from that previous entry, provided that the `private` flag is not true.  
 
 == Chip-Warp processing ==
 
-Defining the run does not queue chipRuns for those exposures that have not been processed previously.  This is done by the `lap_science.pl --chip_mode --lap_id LAP` command.  This script calls `laptool -pendingexp -lap_id LAP` which returns a list of all the exposures included in this `lapRun`.  Each exposure is checked, and a `chipRun` queued if the `chip_id` entry is blank.
+Defining the run does not queue chipRuns for those exposures that have not been processed previously.  This is done by 
+{{{
+lap_science.pl --chip_mode --lap_id LAP
+}}}
+This script calls `laptool -pendingexp -lap_id LAP` which returns a list of all the exposures included in this `lapRun`.  Each exposure is checked, and a `chipRun` queued if the `chip_id` entry is blank.
 
 This stage also defines the pairs for differencing, using the object and comment fields to organize the pairs as is done in nightly science.  If a pair is found, the associated `pair_id` fields point at the `chip_id` of the companion exposure, and the `pairwise` flag is set to true.  If a pair is not found, `pair_id` is left as NULL, `pairwise` is set to false, and `private` is set to true.  This ensures that the difference against the quickstack will not be used by other `lapRun`s.
 
-Once all the chips have been queued, the state of the `lapRun` will be polled using the `lap_science.pl --status --lap_id LAP`.  This calls `laptool -exposures -lap_id LAP` which returns the status of each exposure at the chip, camera, fake, warp, diff, and magic stages.  In addition to gauging completion, exposure pairs are checked against each other to confirm that no faults have occurred.  If one member of a pair has an unrecoverable fault, that entry is marked with `data_state` = 'drop', and is no longer used.  The companion of that unrecoverable exposure is set to be private so it will be diffed against the quickstack.  
+Once all the chips have been queued, the state of the `lapRun` will be polled using the 
+{{{
+lap_science.pl --monitor_mode --lap_id LAP
+}}}
+This calls `laptool -exposures -lap_id LAP` which returns the status of each exposure at the chip, camera, fake, warp, diff, and magic stages.  In addition to gauging completion, exposure pairs are checked against each other to confirm that no faults have occurred.  If one member of a pair has an unrecoverable fault, that entry is marked with `data_state` = 'drop', and is no longer used.  The companion of that unrecoverable exposure is set to be private so it will be diffed against the quickstack.  
 
 Setting the `lapExp` to private "steals" this processing run away from any other `lapRun` that may want to use it. If `lap_science.pl` detects that another `lapExp` with this `exp_id`/`chip_id` set has `private` = true, then it will requeue a `chipRun` for that `exp_id`, and record the new `chip_id` for itself, and set `private` = true.
 
-The script will also check the status of any stacks via the `laptool -stacks -lap_id LAP` which returns the status of all associated stacks for this `lapRun`.  This information is used to set a pair of flags for both the quick and final stacks: the `defined_`flag notes that stacks have been queued, and the `have_` flag notes that the stacks have completed and may be used.
+The script will also check the status of any stacks via the `laptool -stacks -lap_id LAP` which returns the status of all associated stacks for this `lapRun`.  This information is used to set a pair of flags for both the quick and final stacks: the `defined_`flag notes that stacks have been queued, and the `have_` flag notes that the stacks have completed (or attempted and failed with a fault) and may be used.
 
 Using the information about the exposures and stacks for this `lapRun`, `lap_science.pl` will decide what the next step of the processing will be.  After all exposures have a `warpRun.state` of full (or have been dropped due to quality), `lap_science.pl` checks which step is most appropriate:
@@ -67 +83 @@
 === Diffs ===
 
-If no `lapExp` has `private` = true, and they all have valid `pair_id` values, then they will all be pairwise diffed.  If any `lapExp` has `private` = true, and `have_quick` is true (marking that the quickstack has finished), then the quickstack diff for that exposure will be queued as well.
+If all `lapExp` have completed the warp stage (or been dropped along the way), diffs may be queued.  If no `lapExp` has `private = true`, then the pairwise diffs are performed.  If a `lapExp` is set to be private, and the `have_quick = true` (marking that the quickstack has finished), then the private `lapExp`s are diffed against the quickstack, and the others diffed as a pair.
 
 === Final stacks ===
@@ -75 +91 @@
 === Cleanup ===
 
-Once the final stacks have completed, the `lapRun` is also complete.  If have_final is true, then `lapRun.state` is set to 'full', and all `lapExp.active` is set to false.  Periodically, `lap_science.pl --cleanup` will poll all `lapExp`, and if all `lapExp` with a given `chip_id` have `active` = false, then the products of all stages generated from that `chip_id` will be marked for cleanup (`STAGE.label = 'goto_cleaned'`, `STAGE.state = 'goto_cleaned'`).
+Once the final stacks have completed, the `lapRun` is also complete.  If have_final is true, then `lapRun.state` is set to 'full', and all `lapExp.active` is set to false.  Periodically, `lap_science.pl --cleanup_mode` will poll all `lapExp`, and if all `lapExp` with a given `chip_id` have `active` = false, then the products of all stages generated from that `chip_id` will be marked for cleanup (`STAGE.label = 'goto_cleaned'`, `STAGE.state = 'goto_cleaned'`).
 
 == Proposed Changes ==