== Nightly Science: pantasks utilities ==

||{{{nightly.stacks.on}}}||Turn on the entire automation code.||
||{{{nightly.stacks.off}}}|| Turn off the entire automation code.||
||{{{ns.add.date YYYY-MM-DD}}}||Add a date to be processed.||
||{{{ns.show.dates}}}||List the dates currently in the book, along with their associated states.||
||{{{ns.del.date}}}||Remove a date from processing (dates that make it to stack are automatically removed).||
||{{{ns.set.date YYYY-MM-DD STATE}}}||Set the state for a given date.||

== State values: ==

||State:       || Criterion ||
||NEW          || Registration has not completed yet (or registration for this date has not been checked). ||
||REGISTERED   || Registration complete. ||
||NEEDSBURNING || No burntool tables found for some portion of the data. || 
||QUEUEBURNING || Burntool jobs are being sent to hosts for processing. ||
||BURNING      || The first pass of sending out jobs has completed (signals ns.chips.load to check for completion). ||
||QUEUECHIPS   || Burntool has finished, and chips can be queued. ||
||TOWARP       || Chips have been queued, and ns.stacks.load is waiting for all stackable warps to finish. ||
||QUEUESTACKS  || Stackable warps have finished, and stacks will be queued. ||
||STACKING     || stacktool commands have run to queue all stackable data.||
||DROP         || No exposures were found for the date, or the date string was malformed.||

== automate_stacks.pl script ==

Useful modes for manually checking and queueing data:

{{{automate_stacks.pl --date YYYY-MM-DD --check_registration}}}

Confirm that registration is complete, and all needed exposures are available.  If exposures are missing (due to an incomplete download or registration error), they are listed along with a note stating whether they are important to the automation or not.

{{{automate_stacks.pl --date YYYY-MM-DD --check_chips}}}

{{{automate_stacks.pl --date YYYY-MM-DD --queue_chips}}}

The first checks the chips to see if they're ready to be queued (confirms that burntool has finished for all exposures), and the second issues the appropriate chiptool commands.  

{{{automate_stacks.pl --date YYYY-MM-DD --check_stacks}}}

{{{automate_stacks.pl --date YYYY-MM-DD --queue_stacks}}}

{{{automate_stacks.pl --date YYYY-MM-DD --queue_stacks --force_stack_count}}}

Similar to the chip mode options, check the number of warps against the input exposures to determine if all the warps for a stack are finished, and queue them if so.  The --force_stack_count allows this counting to be skipped if an exposure is known to have a uncorrectable error.

Both chip and stack mode options support the --this_target_only and --this_filter_only which allow a specific target or filter to be queued without checking the rest.  This is safe because previously queued runs will not be duplicated.

== automate_stacks.pl config file == 

Each target in ippconfig/recipes/nightly_science.config has the following parameters:

|| NAME  || STR || Target name. Used as the base of the data_group variable. ||
|| TESS  || STR || Tessellation ID. ||
|| OBSMODE || STR || obs_mode value from rawExp. This is a LIKE search, so % wildcards are supported. ||
|| OBJECT || STR || object from rawExp.  This was not populated, so although it is supported, none of the current target uses it.||
|| COMMENT || STR || comment from rawExp. Also a LIKE search. ||
|| STACKABLE || BOOL || This flag determines whether or not nightly stacks can be made of this target. ||

The known filters are also stored in this file, although this is only used while queueing stacks.

The RETENTION_TIME value stores the number of days data should be stored before being marked for cleanup.  This is commented out (it is part of ns.initday.load), but is fully implemented.