Spectroscopic Sky Probe (ssp)

This is John/Chris's toy on PS1, see https://svn.ifa.hawaii.edu/gpc/wiki/AtmosphericMonitor for more information.

The goal is to do the following things:

  • free up JT's disk space
  • auto download and register into the ssp database these files
    • download into nebulous
    • make 2 copies
    • make a backup on one of the ippatrc machines
  • run JT's processing on it, save in nebulous (2 copies)

Tasks/blog of what Heather's doing

  • download JT's stuff
  • make ssp database
    • ippadmin project ippdb01 ipp ssp
    • pxadmin -dbname ssp -create
  • ingest jt's stuff into ssp database (this will likely require some tests/ modifications of ippconfig)
    • insert the first date of stuff using:
      • ipp_serial_inject.pl --telescope simple --camera simple --workdir neb://@HOST@.0/ssp/20110702 --end_stage reg --dbname ssp *fits
      • note to self, use the full path /data/ipp004.0/ssp/20110702 in the future, or else you have to run pantasks in that same dir
    • mod #1 -- regtool -pendingimfile is broken for manually inserted stuff because it joins on summitExp which does not exist for stuff manually inserted using ipp_serial_inject.pl. To get around this:
      • insert into summitExp (summit_id, dateobs) values (0,'2012-01-31 20:30:26');
      • update summitExp set summit_id = 0;
    • This adds a fakey entry in summitExp that allows it to run registration in pantasks:
      • module pantasks.pro
      • module register.pro
      • add.database ssp
      • controller host add ipp006
      • run
    • pantasks does not work - it does not recognize simple as the telescope... I need to make the proper ssp things now
    • ssp is now in ippconfig, and it's all hacked together and it works.
    • new ingest script: ipp_serial_inject_into_nebulous.pl which works the same as the other script, but copies into nebulous (2 copies).
    • working on the first night , all seems to be going well...
    • there are some duplicates (due to problems with injest stopping suddenly when my X crashed)
    • there are some files that don't register (they are 0 byte fits files)
  • until datastore on uip/ssp machine is working
    • transfer from atrc to somewhere (/data/ipp004.0/ssp/ps1atrc/
    • injest (I need to make it smart - don't injest stuff that already exists in db - done!)
      • some injests failed when nebulous was cranky (about 100) - the entry exists in newImfile but the nebulous entry is a 0 byte file -how best to fix this?
  • install monsta
  • make tasks for JT's monsta scripts
  • have it process the data and put the data products somewhere sensible