
**** burntool / archive / video ****

** outline

* btChipNights : table (book) of all nights
  fields: YYYYMMDD.XYnn STATE ENTRY OUTTABLE

* CHIPNIGHT : books with names like YYYYMMDD.XYnn with sequence of exp/chip


* btChipHosts : table (book) of all chiphosts
  fields : HOSTNAME NRUN 




** background thoughts:

* burntool issues

A single chip is processed with a command like:

> burntool INPUT out=OUTPUT tableonly=t persist=t

To limit the impact on the storage nodes, we need to target the
machines which host the chips.

OR limit the number of jobs running which want a chip on a specific
machine. 

Either of these solutions requires that we know the location of each
chip before processing.

Burntool chips must be processed in sequence, so we must block chip N
until chip N-1 is processed.

***

* start with a night, select all exposures for the night
* generate a list of all exposures in sequence for each chip
  * what if a chip is bad / missing / should be skipped?
    * don't generate an entry in the list
    * use the rawImfile table to identify the available & good chips
      for eac


---------------

How to track the sequence for a given night / chip?

Option 1: I have a book with a bunch of chip entries:

o54321g0001.XY01  DONE
o54321g0002.XY01  DONE
o54321g0003.XY01  NEW
o54321g0004.XY01  INIT
o54321g0005.XY01  INIT
o54321g0006.XY01  INIT
o54321g0007.XY01  INIT
o54321g0008.XY01  INIT
o54321g0009.XY01  INIT

* grab next chip which is ready to go:
book getpage BOOK 0 -var pageName -key pantaskState NEW

* mark it as in-flight:
book setword BOOK $pageName pantaskState RUN 

o54321g0001.XY01  DONE
o54321g0002.XY01  DONE
o54321g0003.XY01  RUN
o54321g0004.XY01  INIT
o54321g0005.XY01  INIT
o54321g0006.XY01  INIT
o54321g0007.XY01  INIT
o54321g0008.XY01  INIT
o54321g0009.XY01  INIT

* when done, update states:
book setword BOOK $pageName pantaskState DONE
book getword BOOK $pageName sequence -var Nchip
$Nchip ++
book getpage BOOK $sequence -var pageName
book setword BOOK $pageName pantaskState NEW

o54321g0001.XY01  DONE
o54321g0002.XY01  DONE
o54321g0003.XY01  DONE
o54321g0004.XY01  NEW
o54321g0005.XY01  INIT
o54321g0006.XY01  INIT
o54321g0007.XY01  INIT
o54321g0008.XY01  INIT
o54321g0009.XY01  INIT

* on failure, set the current to fail

** still need to track the sequence and info on the current state is
   missing.

-------

Option 2: the book of night/chips includes a counter for the current
active with a state.

20201010.XY01 INIT 0 -- do I need a different starting state from DONE?
20201010.XY02 RUN  5
20201010.XY03 DONE 2
20201010.XY04 FAIL 10

* for the current night/chip, grab the current state and sequence:

book getpage btChipNight $Nseq -var pageName
book getword btChipNight $pageName sequence -var chipSeq
book getword btChipNight $pageName state    -var chipState

if ($chipState == DONE)
  $chipSeq ++

  book getword BOOK $pageName rawImfileReal -var rawImfileReal
  ...

  [push to the CHIPHOST book]

  book setword btChipNight $pageName sequence $chipSeq
  book setword btChipNight $pageName state    RUN
end
