IPP Progress Report for the week 2011.05.02 - 2011.05.06

(Up to IPP Progress Reports)

Eugene Magnier

This week, we upgraded the operations IPP installation to a new tag: ipp-20110505.

There are not a huge number of algorithmic changes in this new tag. Some of the salient algorithmic changes include improvements in the photometry analysis to make aperture and psf magnitudes internally consistent and consistent between psphot and psphotForced. There are also a number of improvements to the Kron flux analysis in response to issues identified by Nigel et al -- in particular, the reported Kron mags are now measured with all other object models subtracted off.

Note that the previous release (ipp-20110406) included all of our upgrades to the detrend analysis (non-linearity correction at the faint end, improved dark model with better temperature trends, improved masking resulting from camera tuning).

This tag includes management elements needed to organize large area processing (LAP) which allows us to process exposures up to 3pi stacks.

We are now starting to run large area processing of 3pi data, using the photometry and astrometry reference database as it currently exists. Note that this reference database is good for astrometry more-or-less everywhere, but I'm still unsatisfied with the photometry results. Some areas of the sky are good, but others do not yet have good results. We are doing 3pi stack analysis for sample areas where there are 4 visits in all 5 filters, and we will limit for now to areas with good (or at least OK) photometric solutions.

We will process a modest chunk of sky (~1000 - 2000 square degrees) before the start of the Boston meeting. The outputs from the large area processing will be placed in a new product on our datastore. Information on the data distribution labels will be forthcoming.

Serge Chastel

  • Helped Peter for MOPS
  • MySQL replication. ippdb01 databases are replicated on a second server running on ippdb02 (port 3307). Unfortunately the host cannot hold the load.

Heather Flewelling

  • power outage at manoa tuesday afternoon
  • linked new dvo db to ifaps1
  • helped Roy/answered Roy questions.
  • JT stacks (pushed a few more along, answer questions)
  • sdss 3pi plots - I got started on this, and got distracted by isp...
  • sorting out isp astrometry problems:
    • r and i 3 pi chunk has lovely astrometry - initially they did not:
      • ISP was using gpc1 CRLIMITS, found limits for isp
      • needed 3rd order fitting
      • needed multiple iterations
    • y needs a lot of work (problems related to few stars? too bright? wrong selection of refstars?) I'm still sorting it out
  • sick half a day

Roy Henderson

  • Detection batches from last week loaded successfully. A relief.
    • able to test new, fast DetectionCalib table with PSVO
    • work to confirm that we have the same calibrated magnitude in PSPS as in the original IPP smf files
  • Stack loading:
    • issue with NULL xPos and yPos values in StackDetection: was failing load
    • discovered that deep stacks can have duplicate objIDs and NULL flux values in psf table, thus explaining some previous issues
    • power-cut on monday. Hassles getting datastore back up, loading all the stuff processed the night before
    • discovered hundreds of nightly stacks with 0.0 EXPTIME, thus screwing up flux calculations
      • gpc1 query kindly supplied by Bill, but highlighted that a proportion of the previous stacks had non-zero, but incorrect, EXPTIME
      • changed code to use Bill's query, and reloaded everything
    • dupe objID confusions in deep stacks - ok in detection table, NOT in object table. Reloaded all deep stacks AGAIN
    • merge failed on Friday: ID we are using for stackDetectID is not unique. Need to add stack ID to primary key as a temporary fix.
  • PSVO
    • discussions and testing of PSVO with query builder
    • made MOPSSchema class for query builder to use
    • slow queries now passed straight to TOPCAT, only first 1000 rows shown in table
    • many other minor changes

Bill Sweeney

  • started working on adding authentication to the data store products and the tools used that access them.
  • spent one day analyzing the apache access logs for our data store, postage stamp server, and other services provided by po02.
  • changed the web proxy configuration to restrict access to ps1sc institutions.
  • spent Friday finally installing a new operating system on my desktop. (I was about two years behind.)
  • set up the database to enable distribution of the LAP outputs.

Chris Waters

  • Finished Large Area Processing code, and ran initial tests. Began reprocessing for large area (15 < RA < 40, -5 < DEC < 25).
  • Converted synthetic reference catalog photcode names from x_SYNTH to x, to match the convention in the new internal reference catalog. Confirmed that new reference catalog provides more consistent results than the synthetic catalog.
  • Debugging LAP code.