== IPP Progress Report for the week 2010.11.29 - 2010.12.03 == [[PageOutline]] (Up to [wiki:IPP_Progress_Reports IPP Progress Reports]) === Eugene Magnier === === Serge Chastel === * OC133.OSS data preparation for copy to MOPS data. Copy started on Friday; * Tried to understand how to change version of ppMops in IPP; * Tessellation: trying to find time to finish it; * Various meetings (Ken, IPP, MOPS). === Heather Flewelling === === Roy Henderson === Vacation all week. === Bill Sweeney === * Moved postage stamp server off of ippdb02 to its new (and hopefully permanent home) ippc17. First did a trial transfer Found many hundreds of GB old crufty data. Doing the transition and cleaning that stuff up took the better part of Monday. On Tuesday the actual transition appeared to go smoothly. Unfortunately one piece of the configuration was not updated so distribution posted file sets to the old database. Publishing and postage stamp server are configured differently so they worked fine. Since we had bad weather nobody noticed that nothing was getting distributed. Finally on Monday Dec 6 Mark Huber reported the problem. Fixed this by updating the incorrect table, setting faults on the incorrectly posted file sets and letting revert do the registration over again. All fixed within 15 minutes. * Investigated why some user's postage stamps at the position of a supernova were masked. It turned out that the coordinates were just outside the field of view of the exposures. Since they requested warp stamps they got masked postage stamps. * Made some refinements to the code to "replicate outputs". We will turn this on next time we update the production tag. * Investigated why the fraction of pixels masked by magic was higher than expected. It turns out something changed in the ipp since the last time tests were run that made things worse. Further investigation is needed to track down the cause. * Made changes to the database and scripts to store the resutls of the psphot detection efficiency analysis in the chip and camera database tables. * Tested detection efficiency changes using gpc1 data and simtest. Fixed some latent bugs revealed by simtest. * Selected exposures for MD03 reference stack. Started the processing. Something has changed in the software or the cluster. ppStack required lots of memory for many skycells. So much so that we had some systems crash. Worked around this by continuing the stack processing in a special pantasks instance which only uses 32GB nodes. This is slow going though. * Spent Friday and some time on the weekend as processing czar. === Chris Waters === * Detrends: Reviewed the new darks, and checked that linearity was working correctly. This seems to be the case, so linearity development is now finished. The new darks look much better, although there are some residual dark signal in a few OTAs (ota15, ota45, ota43). * Burntool: Coding on registration for the realtime burntool is finished, and still needs to be tested. * Throughput: Looked at the apache/mysql logs for nebulous to see if we are overloading the database server. We should be able to easily handle 5e5 requests per hour, which should be more than enough to handle processing (translates to ~1400 exposures a day).