General debugging tips

  • There are logs written for each stage of processing (<code>*.log</code> and <code>*.trace</code>) that can be useful for figuring out what went wrong.
  • pantasks will usually display the command that failed. You can copy and paste this into your shell to investigate.
    • If the command involves a SQL query, which most pantasks commands do, you can add "-trace psLib.db 10" to see the SQL query that was involved and investigate directly in the database what's going on.
  • When debugging Perl scripts (identifiable from the <code>*.pl</code> filename), add <code>--verbose --no-update</code> to the end of the command-line. <code>--verbose</code> makes it spew more stuff to the screen (useful for debugging), and <code>--no-update</code> means "don't push results into the database" (also useful while debugging, so you don't alter the state while you track down the problem). You probably also want to remove the <code>--log /path/to/log</code> if it's present.
  • When copying a command from a Perl script to the command line to run, you probably want to remove the <code>-tracedest /path/to/trace</code> and <code>-log /path/to/log</code> arguments, if present. These redirect output to files instead of the terminal.
  • Some Perl scripts generate temporary files (e.g., list of input filenames) for the binaries to process, which are usually automatically deleted when the Perl script completes (even when you stop it with CTRL-C). To preserve them (so you can run the binary without the Perl script), add <code>--save-temps</code> to the command-line.
  • The IPP binaries (anything that doesn't have the <code>*.pl</code> filename, with the exception of the DVO commands) take trace arguments that allow you to get more information out (like <code>--verbose</code> for the Perl scripts, but more configurable, and more useful). For example, if you're debugging an astrometry problem, add <code>-trace psastro 5</code> to the command line. You specify a component and a level ranging from 0 to 10. The idea is that there are statements buried in the code that will produce trace output at different levels. 0 is the lowest-verbosity (off) level; 10 is the highest-verbosity (all the gory details) level. The exact calibration of these levels is somewhat arbitrary and very dependent upon what component you're looking at. If you're not sure, use a level of 3 and adjust down or up according to whether you get too much or not enough.

Debugging binaries with gdb

Here are some useful definitions to help when debugging Pan-STARRS binaries with gdb.

Some particularly useful functions are:

  • pserr and pswarn set break points for whenever an error or warning is encountered. Note that pswarn will only catch calls to <tt>psWarning</tt>, and not calls to <tt>psLogMsg</tt> with a level of <tt>PS_LOG_WARN</tt>; this could be easily fixed.
  • psmemid sets break points for when a memblock with the provided id is allocated or freed. Note that this is kind of finicky (RHL says it's because of bugs in gdb) --- make your id negative when you first call it. The breaks so set may or may not result in useful breaks when you rerun your program. Also, you generally can't call it before the program has started running (because the functions aren't defined) --- set a break at <tt>main()</tt>, and then use it. You can set the alloc and free break points separately using psallocid and psfreeid
  • psmemprint prints some information about the memblock associated with a pointer.
  • psimagewrite will write a <tt>psImage</tt> as a FITS file. Be careful to enclose the name of the FITS file in double quotes. This uses the <tt>psFits</tt> functions, so is only available when running a program (as opposed to examining a core dump).
  • psmetadataitem sets a break point on the allocation of a particular <tt>psMetadataItem</tt>, identified by its id. This can be useful when debugging multiple frees of an item on a <tt>psMetadata</tt>.
  • psmetadatalist gives a list of the names of items on a metadata. You can subsequently print a particular item using psmetadataget. These are intended for use when you're debugging a core dump. When you're running a program, use calls to <tt>psMetadataPrint</tt> (or <tt>psMetadataConfigFormat</tt>).
  • Calls to <tt>pmFPAPrint</tt> are useful for seeing what's in a particular <tt>pmFPA</tt> --- can only be used when you're running a program.
  • rr re-runs the program, with no confirmation required.

Debugging configuration errors

  • If you suspect you have a config error, start by determining which config files are active.
  • Look in your .ipprc for the PATH variable. Any config file location specified as a relative path (e.g. <code>gpc1/camera.config</code> rather than <code>/absolute/path/to/gpc1/camera.config</code>) gets the value of $PATH prepended to it. So if your .ipprc file sets the PATH and SYSTEM variables as: PATH STR /home/panstarrs/ipp/ippconfig SYSTEM STR system.config

then the active system.config file will be <code>/home/panstarrs/ipp/ippconfig/system.config</code>

  • check your .tcshrc file (or equivalent) for the psconfig command. If you aren't entering the psconfig command by hand every time you start any IPP work, then it is probably being set by the shell like this:

alias psconfig "source ~ipp/bin/shell/psconfig.csh" psconfig ipp-2.6

Verify that the .tcshrc psconfig version and .ipprc PATH are in sync.

  • Check your system.config and site.config files. Note that the system.config specifies which camera.config file is active for each available camera.
  • Inspect the camera.config file, which will point to other camera-level config files like psphot.conifg and ppImage.config.

Some specific config errors and their causes:

ERROR: can't find configuration file (null)

You're missing the .ptolemyrc file. Copy from your site-level config directory into your home directory and rename it as .ptolemyrc.