IPP Progress Report for the week 2011.04.11 - 2011.04.15

(Up to IPP Progress Reports)

Eugene Magnier

The all-sky relphot analysis of 3pi database ran for most of the week. Despite my modifications to conserve memory used by the analysis, memory usage still made the process slow. The data load and analysis of the zero points ran smoothly in about 7.5 hours, using a modest amount of memory. However, at the 'update the database' stage, the memory usage jumped more than expected. I believe this resulted in a lot of swapping, slowing down the update processing. This stage ended up taking 5 days, but it did finally complete. I have made an initial examination of the results (comparing the stellar locus from the MD field), and it seems to have generally done a good job of finding the calibrations (though areas with clouds and insufficient connections to photometric data can be seen). For large areas of the sky, this database is probably good enough to start GR.

While relphot ran, I also worked on psphot issues raised by Nigel M. and Michael W.V. I found that the Kron radius was allowed to go to excessively small or large values for faint objects, so I have put in limits. This is the cause of some of the discrepant Kron magnitudes. I have also found that the aperture magnitudes were affected by inconsistencies in model addition/subtraction during the processing. I also found places were some of the fit status flags were not correctly set. I fixed this by moving the code to set the flags closer to the action to avoid interpretation errors. I also found that the position errors were not being reported for extended non-linear fits because of a logic error, which I fixed.

Serge Chastel

  • MOPS:
    • czar
    • Hemocrat dev
  • IPP:
    • System survey (partial)
    • Backup
    • ICD update/synchronization

Heather Flewelling

  • merging of the ThreePi? minidvodbs are done (except for 2 that need some work).
  • czar for 1 day
  • created databases for Jim: dvodb of the cam stage MD04s, and dvodb of the static sky MD04 stacks (Bill set up the staticsky).
  • updated addRun database and checked in changes to addstar to handle different stages (cam and staticsky are tested to be working).
  • booked flights, hotels, and registrations for PS1SC and AAS conference
  • worked on videophotometry stuff for pavlos - still working on it.

Roy Henderson

* ippToPsps

  • wrote a C program (based on old code) for interfacing with DVO. Is called from Jython code to populate database tables for quick detection matching
  • talked to STILTS developer in UK and got a fix for issues caused by infinity values that sneak into IPP FITS files
  • made numerous PSPS schema changes with Jim
  • code changes within ippToPsps to deal with schema changes
  • wrote a more generic FITS header parser for image meta extensions for detections (STILTS doesn't read headers)
  • tested Heather's new DVO Db containing stacks: success. Was able to pull out object IDs
  • more work on detection batches with Jython.


  • finished changes necessary to implement Daniel's query builder:
    • wrote PSPSSchema as subclass of abstract Schema class. Figures out tables, columns, constraints and populates Schema fields
    • added the GUI stuff necessary to pop up builder, passing it relevant Schema and Listener objects

* vacation on Wednesday

Bill Sweeney

  • Worked with Johannes K. at MPG to refine the new STS tessellation. Started processing the 2010 data with this skycell definition.
  • Distributed the MD04.deeptest data.
  • Finished the script to perform the staticsky photometry on nightly stacks.
  • Queued and executed staticsky runs for all MD04 nightly stacks
  • Wrote a long email to Robert S. of MPG about the operation of a data store. He is considering using our code to send photo-z data to PSPS. (At some point I need to refine this into a wiki page)
  • Spent an afternoon running psastro on some sts exposures affected by the astrometry problem. The goal is to find a set of parameters for the analysis that improve the results. This will continue next week.
  • Insured that the test exposures for new STS fields got processed.
  • Investigated the persistence effect caused by a 3rd magnitude star in one of the possible new fields.
  • transferred chip outputs from 4 exposures to the rsync server for the outreach program.

Chris Waters

  • Reprocessing Tests: Created z-band comparison, which has a less complete set of exposures. This shows large gaps, most likely due to insufficient numbers of input exposures for some regions. Checked that quick stacks have the expected noise distribution (S/N is a Gaussian with sigma = 1). Checked dual-convolution of quickstack-warp diffs compare to single convolution diffs. The dual convolution results in far more magic streaks. Implemented a no-convolution mode for ppSub, and tested that. This seems to be an improvement over any convolutions in the diff, but is still worse than pair-wise diffs (based on stack good_frac and magic Nstreaks).
  • Astrometry: processed first microtest data, and organized wiki page with results: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/GPC_BrightStarAstrometry_Microtest
  • Registration: identified bug that can stall registration at the beginning of a night if an exposure has summitExp.dateobs and rawImfile.dateobs that fall on opposite sides of the burntool cutoff (currently UTC 03:30).
  • Detectability: fixed first batch of bugs, which should allow data that has not been cleaned to process correctly. I have not yet tested data that needs to be updated.