This is a quick kind to the basic processing of the IPP steps that an end user might want to redo at his or her own institution.  This was written with the Jan 2009 GPC1 commissioning data release in mind.  There's probably some good integration that could be done with previous such pages by Jester et al.


=== Executive Summary ===



=== Detailed Step-by-Step guide ===

I have written these with the Odyssey cluster installation at Harvard as the example.  Please contact your local DRAVG representative to learn where images are kept at your institution and how to set up the IPP environment.

At Harvard we use the 'module load' system to configure things on the cluster

 module load hpc/ipp
 psconfig default

and here's where we keep the data

 cd /n/data1/panstarrs/GPC1/MD01_20081002_1

In this directory are each of the different observations.  The directories are named by the observation number.  These span the course of several weeks.

 [wwoodvas@heroint2 MD01_20081002_1]$ ls
 36475/	36476/	36477/	36478/	36479/	36480/	36481/	36625/	stack/

I created the 'stack' directory as a place where we can put our coadd stacked images.  The coadded images were not distributed with this data release.  It wouldn't make sense to put the coadds under any particular directory because they don't belong to any particular observation so I've just created a directory for them here at the base level.

Each of the observation number directories contains directories for the raw, chip, warp, and diff images

<code>
 [wwoodvas@heroint2 MD01_20081002_1]$ ls 36475
 chip/  diff/  raw/  warp/
</code>

There are .log files in these directories detailing the steps and output that were used to create them.  You may find referring to them a useful exercise if you want to understand all of the details of what was done to derive the images.  

Since we are not interested in recreating the basic detector calibration at this point we will start with the 'chip' images.

=== Photometry and DVO ===

 * Photometry.  Basic photometry with 'psphot'

<code>
cd MD01_20081002_1/36475
psphot -D CAMERA gpc1 -file chip/o4741g0234o.36475.ch.11948.XY55.ch.fits -mask chip/o4741g0234o.36475.ch.11948.XY55.ch.mk.fits -weight chip/o4741g0234o.36475.ch.11948.XY55.ch.wt.fits testout
</code>
will create testout.cmf.  If you want to see what psphot is doing as it runs, you may be interested in the '-visual' option

psphot -file chip/o4741g0234o.36475.ch.11948.XY55.ch.fits -mask chip//o4741g0234o.36475.ch.11948.XY55.ch.mk.fits -weight chip/o4741g0234o.36475.ch.11948.XY55.ch.wt.fits testout -visual

The main output from the psphot command will be the .cmf file, which is a binary FITS table that contains information about the objects as gleaned from the image.  One of the fields that generates the most questions is the FLAGS value for objects.  This is a bitmask.  See [wiki:IPP_Detection_Bitmasks IPP Detection Bitmasks] for more details.

 * DVO.  Let's say we have a set of cmf files and we want to use DVO to combine the detections from these files.  A series of commands such as 

 cd MD01_20081002_1
 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36475/chip/o4741g0234o.36475.ch.11948.XY55.cmf
 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36476/chip/o4741g0235o.36476.ch.11949.XY55.cmf
 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36477/chip/o4741g0236o.36477.ch.11950.XY55.cmf
 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36478/chip/o4741g0237o.36478.ch.11951.XY55.cmf

will create a DVO database and fill it with the detections in the given CMF files.  Since these are the same pointings we should get multiple detections (matches) when we put together the catalog.  Currently -accept-astrom is still necessary until the WCS solutions finalizes and we know our true tolerances for good WCS.  Need other page reference or more notes to explore DVO further.

=== Warping, Stacking, and Subtraction ===

==== Warping ====

 * To create warped images that conform to a given position on the sky, we need to define how we want to break up th sky.  This tessellation is what IPP will use to map the chip images to what we call skycells.

 skycells -D CAMERA gpc1 8 -scale 0.2 -D CATDIR default

This will create a directory ('default') that will contain FITS files that define the tessellation of the sky.  Now we can recreate the warping of chip images to warp images:

First let's create a sandbox for ourselves to put our new files into

 mkdir -p workspace/warp workspace/stack workspace/diff 

Now we'll fill a skycell.  You may wonder how to know what skycell a particular chip will warp to.  That is a good question that currently has an answer that's too complicated to be practical 

'''Easy RA,Dec->skycell is a prime opportunity for a contribution'''.  [On Feb 4 Mark Huber said he was going to work on this].

For now just take the following example:

pswarp -file 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.fits -mask 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.mk.fits -weight 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.wt.fits workspace/warp/o4741g0234o.36475.wrp.11948.skycell.02879.51

==== Stacking ====

If we have a set of images that we have warped to the same skycell and we want to stack (coadd) these images together we first specify a file that defines the images, masks, weights, PSF models, etc. for the input images to the stack:

<code><pre>
INPUT0  METADATA
    IMAGE      STR   36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.fits
    MASK       STR   36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.mask.fits
    WEIGHT     STR   36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.wt.fits
    PSF        STR   36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.psf
    SOURCES    STR   36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT1  METADATA
    IMAGE      STR   36476/warp/o4741g0235o.36476.wrp.1279.skycell.02879.51.fits
    MASK       STR   36476/warp/o4741g0235o.36476.wrp.1279.skycell.02879.51.mask.fits
    WEIGHT     STR   36476/warp/o4741g0235o.36476.wrp.1279.skycell.02879.51.wt.fits
    PSF        STR   36476/warp/o4741g0235o.36476.wrp.1279.skycell.02879.51.psf
    SOURCES    STR   36476/warp/o4741g0235o.36476.wrp.1279.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT2  METADATA
    IMAGE      STR   36477/warp/o4741g0236o.36477.wrp.1280.skycell.02879.51.fits
    MASK       STR   36477/warp/o4741g0236o.36477.wrp.1280.skycell.02879.51.mask.fits
    WEIGHT     STR   36477/warp/o4741g0236o.36477.wrp.1280.skycell.02879.51.wt.fits
    PSF        STR   36477/warp/o4741g0236o.36477.wrp.1280.skycell.02879.51.psf
    SOURCES    STR   36477/warp/o4741g0236o.36477.wrp.1280.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT3  METADATA
    IMAGE      STR   36478/warp/o4741g0237o.36478.wrp.1284.skycell.02879.51.fits
    MASK       STR   36478/warp/o4741g0237o.36478.wrp.1284.skycell.02879.51.mask.fits
    WEIGHT     STR   36478/warp/o4741g0237o.36478.wrp.1284.skycell.02879.51.wt.fits
    PSF        STR   36478/warp/o4741g0237o.36478.wrp.1284.skycell.02879.51.psf
    SOURCES    STR   36478/warp/o4741g0237o.36478.wrp.1284.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT4  METADATA
    IMAGE      STR   36479/warp/o4741g0238o.36479.wrp.1281.skycell.02879.51.fits
    MASK       STR   36479/warp/o4741g0238o.36479.wrp.1281.skycell.02879.51.mask.fits
    WEIGHT     STR   36479/warp/o4741g0238o.36479.wrp.1281.skycell.02879.51.wt.fits
    PSF        STR   36479/warp/o4741g0238o.36479.wrp.1281.skycell.02879.51.psf
    SOURCES    STR   36479/warp/o4741g0238o.36479.wrp.1281.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT5  METADATA
    IMAGE      STR   36480/warp/o4741g0240o.36480.wrp.1285.skycell.02879.51.fits
    MASK       STR   36480/warp/o4741g0240o.36480.wrp.1285.skycell.02879.51.mask.fits
    WEIGHT     STR   36480/warp/o4741g0240o.36480.wrp.1285.skycell.02879.51.wt.fits
    PSF        STR   36480/warp/o4741g0240o.36480.wrp.1285.skycell.02879.51.psf
    SOURCES    STR   36480/warp/o4741g0240o.36480.wrp.1285.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT6  METADATA
    IMAGE      STR   36481/warp/o4741g0241o.36481.wrp.1282.skycell.02879.51.fits
    MASK       STR   36481/warp/o4741g0241o.36481.wrp.1282.skycell.02879.51.mask.fits
    WEIGHT     STR   36481/warp/o4741g0241o.36481.wrp.1282.skycell.02879.51.wt.fits
    PSF        STR   36481/warp/o4741g0241o.36481.wrp.1282.skycell.02879.51.psf
    SOURCES    STR   36481/warp/o4741g0241o.36481.wrp.1282.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END

INPUT7  METADATA
    IMAGE      STR   36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51.fits
    MASK       STR   36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51.mask.fits
    WEIGHT     STR   36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51.wt.fits
    PSF        STR   36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51.psf
    SOURCES    STR   36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51.cmf
    WEIGHTING  F32   1.0
    SCALE      F32   1.0
END
</pre></code>

If we called this file 'skycell.02879.51.mwv.002.mdc', then we might run the following to get the output stack

 ppStack skycell.02879.51.mwv.002.mdc skycell.02879.51.mwv.002 -sources 36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.cmf

(The '-sources' might be redundant since we're specify source lists in the .mdc (meta-data configuration) file).

==== Subtraction ====

Once we have a coaddition that hopeful has enough dithers to fill in the area we will want to run subtractions of individual images against that coadd template.  In this case we built the coadd from the images themselves, but that can't be helped for now as long as we have limited data.

 set ref=skycell.02879.51.mwv.002
 set new=36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51
 ppSub $new.fits $ref.fits sub_isis \
    -inmask  $new.mk.fits -inweight  $new.wt.fits \
    -refmask $ref.mk.fits -refweight $ref.wt.fits \
    -sources $new.cmf -type ISIS -threads 2 -photometry

would run a subtraction of $new minus $ref using the ISIS kernels.  ppSub --help will generate a somewhat detailed list of options but they probably won't all make sense on a first reading.

Since we asked 'ppSub' to do photometry it will create a difference .cmf file along with the subtraction.  If we had a collection of these difference .cmf files we might use DVO to combine them and if we were looking for stationary transients, we would look for multiple detections at the same position.