[[PageOutline]]

This page outlines the procedures and responsibilities for the person currently acting as 'IPP Processing Czar'. In a nutshell, these include:

 * monitoring the various pantasks servers running on the production cluster
 * alerting the IPP group to any notable errors or failures
 * keeping an eye on production cluster load using 'ganglia'
 * adding and removing labels based on the current set of processing priorities, outlined here


= Setup =

You will need to have ipp user access on the production cluster. For convenience, have someone with access (anyone on the IPP team) add your ssh public key to ~ipp/.ssh/authorized_keys.

= Resources =

Mostly, you be logged into a production cluster machine using and using the {{{pantasks_client}}} program to monitor operations, however there are other useful resources.

 * [http://ganglia.pan-starrs.ifa.hawaii.edu/?r=hour&s=descending&c=IPP%2520Production Ganglia] - For monitoring load on production cluster machines
 * [http://ipp004.ifa.hawaii.edu/ippMonitor/Login.php ippmonitor] - a window onto the gpc1 database, particularly the [http://ipp004.ifa.hawaii.edu/ippMonitor/nightSummary.php NightSummary] page

= Rebuilding the IPP code =

The IPP in use presently is in


{{{
~ipp/ipp-20100211
}}}

If the code needs an update and rebuild, then:

 * stop pantasks (as above)
 * {{{cd ~ipp/ipp-20100211}}}
 * {{{svn update}}}
 * {{{psbuild -optimize}}}
 * restart pantasks (as above)