Changes between Version 3 and Version 4 of IPP_Progress_Report_20090927

Timestamp:

Oct 4, 2009, 4:19:42 PM (17 years ago)

Author:

eugene

Comment:

--

IPP_Progress_Report_20090927

@@ -6 +6 @@
 === Eugene Magnier ===
 
+This week, I've spent much of my time making sure my recent work on psphot could be merged into the main SVN trunk and not break other things.  I ran a good-sized test of the photometry using one of the r-band flat-field correction dithers.  I needed to address a number of small bugs caught in that processs.  Eventually, the full analysis completed without errors, and the results look very encouraging.  The flat-field correction determined from that analysis is quite similar to the pattern observed by Eddie Schlafly in the difference between PS1 and SDSS photometry from 3pi data (reported at Harvard).  Since the merge was complete, I have been running the complete set of filters of the flat-field correction data taken in late June 2009.  A little work is needed to update the flat-field correction system since we changed the was 'addstar' is run in the pipeline.  I expect that to be done by Monday, so that we can start processing with the improvements by Tuesday.  
+
 === Heather Flewelling === 
+
+ * finished processing MD09.20090925 and M31.20090925
+ * still processing MD10.20090925
+ * burntooled jun, jul, oct, nov, dec. of 2008. We have burntooled Jun 01, 2008 through now.
+ * burntooled the previous nights data each morning.  Working on a script to automate that.
+ * continued md5sum checks on rawimage files (to verify that all copies of a raw file have the same md5sum, save the md5sums, and to fix the ones that do not).
+  * found 63 corrupted raw image files (not this week, but since I've started)
+  * all but 4 were fixed. Of those 4, 1 is lost, and the other 3 are missing only 1 or 2 cells.
+  * wrote a script to help automate the repair of these files. Only works for a subset of the corrupt files.
+ * found 80,000 files that had both copies on the same host. This has been fixed.
 
 === Bill Giebink ===
@@ -30 +42 @@
 === Bill Sweeney ===
 
+* The major piece of this was implementing the 'byskycell' lookup method for the Postage Stamp Server.
+This feature enables users to select the images of interest for postage stamps based on a skycell identifier.
+This is a very efficient method for the system and is pretty easy for users to use. Especially those interested
+in the medium deep fields.
+
+* To support this effort he modified the ippTools to make it easy to select processed images at
+each stage of the pipeline based on the observation date, filter, and label.
+* Updated the his program for creating requests and started including it directly in the ipp build.
+The next step is finding an efficient way to find images based only on the coordinates.
+
+* Unfortunately part of last weekend and Monday was wasted (Note from EAM: the work was needed, if frustrating) due to a problem where the DataStore
+returned corrupted data to the distribution client. After some difficult debugging it was
+determined to be a problem in one of the open source perl modules that we use as part of
+the ipp. A new version seems to avoid the problem.
+
 === Chris Waters ===
 
+ * Cleanup: More chip/warp cleanups.  Tests of detrend cleanup on
+  non-simtest data.  Identified even more data to be cleaned.
+
+ * Burntool: Identified that burntool is now not expiring trails as
+  expected.  This can lead to oversubtractions on cells that are too
+  noisy, as burntool believes it has found a reasonable fit.  Updated
+  ppImage.config BURNTOOL.TRAILS value to 0x07, which adds all pixels
+  that burntool has altered to the mask as SUSPECT.
+
+ * Nebulous: Added table "deleted" to active Nebulous database to
+  allow the deletion of storage objects that have instances on
+  inactive hosts.  This now functions properly for so_ids that only
+  have a single missing instance, but not for more (unclear why this
+  is).  Working on neb-replicate bug which is not properly
+  randomizing hosts.  Modified "dsget" to work around this bug in the
+  meantime, preventing every backup copy from being put on ipp049.
+
+ * y-band Fringe: Created a new fringe.  Identified that the fringe
+  isn't being scaled correctly for most chips, likely due to a
+  quantization bug (fixed with Paul Price's help) and a sky
+  subtraction bug (solution in progress).
+
+ * Flat verification: Ran flat verify runs, looking into possible
+  causes of residual structure.