Example: How to build a reference stack

As of 2011-03-22 we do not have a reference stack for MD06 in y band. We were waiting until we build MD06.V2 stacks for all filters but now we've decided to build a "V0" stack just for y.

Since the IPP always cleans up data as we go along we will need to reprocess the exposures. We want to select those with the best image quality while making sure that we have a deep enough stack. After several iterations doing this manually Bill built a tool called refstacktool. It is misnamed since it doesn't make the stack, it selects exposures for them.

It lots of options. Luckily it has pod documentation: See perldoc ipp/tools/refstacktool in your local source tree. Basically there are two sets of options "selection options" and "run options". Let's see what we've got for MD06 y filter using the default options.

perl ipp/tools/refstacktool --field MD06 --filter y
Found 98 exposures for MD06 y.
Range of fhwm_major: 3.28499 - 4.17658

That's a pretty good set of inputs even with the default options. What if we cut down to fwhm <= 4 pixels (~1 arc second)?

(ippc18:~) bills% perl ipp/tools/refstacktool --field MD06 --filter y --fwhm_max 4
Found 87 exposures for MD06 y.
Range of fhwm_major: 3.28499 - 3.99525

We only lose 9. Lets go with that.

We've got the parameters to select our input images, now let's set up our "run options". We need to select a label. The default will be MD06.refstack.20110322. That's fine. Next we need to select a tess_id. Since we want to use the "V0" tess_id "MD06" while the default is "MD06.V2" we will need to set this explicitly. I think that's all we need. Let's do a trial run

 perl ipp/tools/refstacktool --field MD06 --filter y --fwhm_max 4 --tess_id MD06 --pretend --verbose

****** lots of output like the following omitted

chiptool -dbname gpc1 -definebyquery -set_label MD06.refstack.20110323 \
-set_workdir neb://@HOST@.0/gpc1/MD06.refstack.20110323 -set_dist_group MD06 -set_tess_id MD06 \
 -set_end_stage warp -pretend -simple -exp_id 174036
174036 o5344g0130o GPC1 PS1 2010-05-28T08:18:05.000000 o5344g0130o.174036 OBJECT CHIP neb://@HOST@.0/gpc1/20100528 \
full  reg y.00000 MD06_y_N5344_MD06_dither_1 MD  1.224000 3.228834 0.822521 240.000000 0.000745 8702.118164 74.691444 \
 915.238647 54.750154 -34.775806 -84.713303 53.752789 0.000000 1000.000000 0.026800 -18.999800 -73.997101 471.622986\
 -506.848999 -1044.849976 74.055397 204.945007 6.600000 28.000000 3.900000 166.000000 8.550000 8.200000 6.550000 \
5.550000 7.300000 6.100000 302469.000000 nan nan nan nan nan MD06_dither_1 92.733803 -38.059399 93.250504 \
32.030800 0.514249 ipp015 0 2010-05-27T22:26:13.000000 0

This shows the chiptool -pretend commands for each exposure and the results. We get the expected 87 exposures and the arguments to the definebyquery command look good.

Let's get the party started

 perl ipp/tools/refstacktool --field MD06 --filter y --fwhm_max 4 --tess_id MD06 --queue
Found 87 exposures for MD06 y.
Range of fhwm_major: 3.28499 - 3.99525

mysql> select count(*) from chipRun where label = 'MD06.refstack.20110323';
+----------+
| count(*) |
+----------+
|       87 | 
+----------+
1 row in set (0.00 sec)

Next we need to add the label to stdscience. Once the warps are done we'll come back and queue the stack.

Later that night all of the warps finished. To queue the stacks I used the following command

stacktool -definebyquery -dbname gpc1 -set_dist_group MD06 -set_data_group MD06.refstack.20110323 -set_label MD06.refstack.20110323 -set_workdir neb://@HOST@.0/gpc1/refstack/MD06.refstack.20110323 -select_label MD06.refstack.20110323 -select_good_frac_min 0.1 -min_num 4 -simple

92 skycells were queued for processing. As of 9:28 UTC 44 had already finished interestingly only 39 with good quality.