IPP Progress Report for the week 2012.07.23 - 2012.07.27

(Up to IPP Progress Reports)

Eugene Magnier

  • I worked with Serge to define the cmf translation code for the TRAIL fits. I was getting some rare core dumps when running trail fitting tests. I put it on the shelf for a bit and Bill found the same problem independently. He tracked it down to a thread-unsafe piece of code being used in a threaded context and fixed the bug. Now we can go ahead with TRAIL tests on gpc1 data and finalize the analysis.
  • I implemented Tamas' exact projection cell centers in our skycell generation code (using the C algorithm he kindly sent me). I then explored the differences between RINGS.V3 (our current version) and Tamas' projection cells. I'm writing a more detailed report, but the summary is that the differences are mostly minor.
  • I created a code concept to define the projection cell boundaries (Boundary Tree table) and generated a boundary tree for the RINGS.V3 tessellation. The boundary tree translates between a specified RA & DEC and the primary projection cell based on the lines of constant RA & DEC at the mid points of the projection cell centers. Because RINGS.V3 has uniform spacing of the zones in DEC and uniform spacing of the projection cell centers in a zone, the translation is a two step evaluation of a linear equation, and thus very fast. Evaluation of 108 points only takes about 0.6 seconds on my desktop machine. (Tamas' projections cells also have linear spacing of RA along a zone, but the zone centers are not uniformly spaced. I have not (yet?) implemented a boundary tree sufficiently generic for that tessellation as it requires an initial search by bisection and a bit more information to define each zone.)
  • I added code to DVO to set the secfilt mean flux (psf or kron) based on the primary projection cell using the boundary tree code above. This is a two-step selection:
    • determine the primary projection cell for the object's RA & DEC
    • if multiple stack detections come from the primary projection, use the one closest to its skycell center.
    • if no stack detections come from the primary projection, use the non-primary detection which is closest to its skycell center.
  • I also worked on the text and figures for Ken's NSF proposal response.

Serge Chastel

  • Czar on Monday
  • Recovered 10 otas that were "lost"
  • Tidied up ipp001 backup for June backup files
  • Started installation of ippc62 and ippc63 as nebulous replicants.
  • Played with swappiness on ippc63, ippc11, ipp009, ipp005, ipp006.

Heather Flewelling

  • sciency stuff for jt and for kc
    • crash course in pstamp from bill
  • helped roy with ipp/psphot issues
  • testing of gene dvo changes
    • check 1: skytable.fits is not overlapping (could be a clue for why it is broken?)
    • building a dvo with new features (testing this still)
      • note to self, make sure .ptolemyrc AND ptolemy.rc both have PS1_V4 or else it doesn't work.
  • building Q2 (again)
    • the first two iterations failed at the same minidvodb, so it is now excluded (pending investigations)
      • nothing obvious in number of stars in camRun
      • nothing obvious in size of dvodb
      • running dvoverify - no problems spotted
      • checking num of stuff in Images.dat - nothing suspicous.
      • checked czarlogs - suspicious - that day of repeated nebulous problems happened the same day the minidvodb was built - maybe there is something funny...
    • the third one is building.. and.. it crashed on the minidvodb after (so, it crashes on 1696 and 1698)
    • making various test merge dvodbs consisting of 1691.. though 1702 containing 1696 and 1698 - do those still crash? or others? no, these work fine.
    • attempting to google this error/try other merges. I think once the big dvodb gets to a certain size, it won't merge any more, and I'm not sure why. I think the minis are ok
  • ipptopsps
    • few more tweaks to ipptopsps wiki on what Mags/Fluxes we want in objects
    • investigating Peder's questions about stackDetectId and skycells -- so it appears that:
      • skycell is not properly ingested into psps (and this is a known problem)
      • stackDetectId is not unique (again a known problem, and we should fix it)
    • added/modded column names and tested -it works! init batch and object are now on psps_test for psps to test (SAS.20120709)
    • cleanup of datastores - problems due to roy owning files... bad roy!
  • skycal
    • first pass at adding skycal to addstar, needs testing
  • mac ipp problems
    • cfitsio is broken - need to hack the make file to have -install_name (final_path_to/libcfitsio.dylib) after which it works properly (otool -L agrees)

Mark Huber

  • MD.GR0: talk outline for Durham and finishing up problems (updating wiki for semi-final summary (MD.GR0), fixing/dumping MD09 problem chips to push out remaining SMFs and nightly stacks, fixing plots).
  • MD10.refstack.20120705 finally out and diffims made (accidentally triggered rerun of all StackStack? diffims) after a test sample run. Additional verification plots against previous MD10.GR0 2011 refstack and one of the best nightly stacks made. Parsing of ppStack logs to make histograms of input FWHM per skycell with note of the target PSF. Info/plots put to wiki when time MD10.refstack.20120705.
  • MD.deeptest stacks setup for MD04 again and now adding MD09 with most recent processed exposures to flush out more large scale processing bugs and provide extra-deep comparison samples.
  • Looked into some diffim problem examples from Peter where the w magnitudes were off by a magnitude, maybe case of poor measurement with FLAGS2 burntool/poor convolution set.

Bill Sweeney

  • Completed final testing and integration of the changes to the psphot footprint analysis code that we worked on last week.
  • Modified psphotStack to split the final Linear fit into two stages. In the first detected sources are fit, in the second the matched sources (sources detected in one or more of the other bands) are fit with the others held fixed.
  • Debugged a memory corruption problem in psphotStack. Over the weekend we found a race condition that may be the root cause of the problem.
  • Changed psphotStack to reduce the number of radial apertures when a large number of sources are detected. (For example near the Galactic plane). This will reduce the memory requirement for the program to a more reasonable level in dense areas.
  • Did some analysis of postage stamp jobs. Discovered a bug in the queuing of warp jobs that multiple jobs from the same exposure to be queued. Fixed that.
  • Implemented code to identify a list of preferred data groups which will be used to select data for postage stamp jobs. This should reduce the number of useless stamps returned to users.

Chris Waters

  • Unsuccessfully tried to find a FFT solution to the row-by-row bias issue.
  • Added code to nightly_science.pl to make MOPS stacks using the nightly data and the reprocessed ecliptic.rp data.
  • PSPS tests, with some work on showing that the PSPS results are reasonably clean for N > 1 and psfQf > 0.85.
  • Writing.