Sebastian's notes on reducing LBCred data with IPP. Following SimtestEAM.

Preliminaries: setup

First I need to create the admin databases:

dbadmin project aida77 ipp jester_LBCred pxadmin -create -dbname jester_LBCred

and define some paths in site.config:

DATAPATH METADATA

LBCin STR path://LBC/sci/ LBCrun STR path://LBC/ippreduce/ LBCflat STR path://LBC/calib/flats/ LBCbias STR path://LBC/calib/bias/ LBCdark STR path://LBC/calib/dark/

END

# List of tessellations, and their DVO CATDIR TESSELLATIONS METADATA

BOOTESCELLS STR path://LBC/skycells/

END

# dvo databases used for output DVO.CATDIRS METADATA

BOOTESCAT STR path://LBC/catdir

END

Then I set up some skycells

skycells 6 -scale 0.2 -fix-ns -D CATDIR /disk1/jester/data/LBT/z7qso/sci/skycells

-fix-ns aligns the skycells with declination - otherwise we have a discontinuity (accidentally we're in a special part of the tessellation).

Somehow, I can't plot up the images in dvo - this happens:

dvo: imbox -c red ../lbcr.20080212.105410.fits error getting all elements for coordinate mode NONE

Injection

For detrending

Inject the data, giving skycell and dvodb names (and maybe also a reduction class?)

ipp_serial_inject.pl --telescope LBT-DX --camera LBC-RED \

--workdir path://LBCrun --dbname jester_LBCred \ --reduction PPIMAGE_OBDFR --tess_id BOOTESCELLS --dvodb BOOTESCAT lbcr.*.fits

Actually, telescope and camera (or instrument) are worked out by the database from format.config, so just given here for the user's reference.

Calibration images are injected in the same way, the injection again works out each file's type from format.config:

ipp_serial_inject.pl --telescope LBT --camera LBCRED \

--workdir path://LBCrun --dbname jester_LBCred \ ../calib/*/lbcr.*.fits

Start up pantasks and register the data:

module pantasks.pro module.tasks controller host add aida77 # up to 3 times, but check the load on the machine first! add.database jester_LBCred # or fan or kschmidt_lbc_stack run

To see what's going on, there are several commands:

status # Show status of various queue tasks status -taskstats # show how long the various tasks took on average controller status # Show how many jobs are queued and how many CPUs are working or not

If something doesn't work, repeat the register_imfile.pl call that's failing in an xterm but with extra diagnostic output, e.g.:

register_imfile.pl --exp_id 89 --tmp_class_id fpa --tmp_exp_name lbcr.20080211.103621 --uri path://LBC/sci/lbcr.20080211.103621.fits --dbname jester_LBCred --verbose --no-update

Then you can 'revert' the file that's stuck:

regtool -revertprocessedimfile -dbname fan

For mosaicing

Setup

Directories in set up in /IPP/site.config

# in DATAPATH METADATA LBTPROC STR /disk2/jester/data/LBT/z7qso/IPPmosaic/ # in TESSELLATIONS METADATA LBTCELLS STR path://LBTPROC/skycells # in DVO.CATDIRS METADATA LBTPROCCATDIR STR path://LBTPROC/catdir

Skycell creation

skycells 6 -scale 0.2 -fix-ns -D CATDIR ipp_datapath.pl path://LBTCELLS/skycells

Running

In an xterm:

ipp_serial_inject.pl --telescope LBT-DX --camera LBCREDPROC --reduction PROCESSED \

--workdir path://LBTPROC --dbname kschmidt_lbc_stack --tess_id LBTCELLS \ --dvodb LBTPROCCATDIR ~fan/data/bootes12.fits

Then start pantasks as above, but you only want the stages starting with register, none of the detrend ones. I typically say

all.off register.on chip.on camera.on

and if that works,

fake.on warp.on stack.on diff.on # if you're into difference images

Start a detrend run

ppImage.config has a section with constraints that are applied to figure out which detrend files to use for which science image. SimtestEAM mentions a pantasks module mkdetrend.pro which is in ipp-2.5 but only in the source tree at ipp-2.5/ippTasks/mkdetrend.pro - it doesn't get copied into the share/pantasks/modules/ directory for some reason.

mysql detRun table defines date ranges for individual detrend runs