| 7 | | Most of the IPP team were at the meeting at Harvard last week, so not |
| 8 | | a lot of progress was accomplished. We had a major incident on the |
| 9 | | MHPCC cluster, losing the RAID that hosted the home directories. The |
| 10 | | RAID was recovered eventually, thanks to Brad, Cindy, & Gavin, and in |
| 11 | | the processed we learned that the drive bays are mis-labeled. In the |
| 12 | | mean while, we took the opportunity to re-assign the home directory |
| 13 | | machine temporarily to ipp053, with the goal of moving to a pair of |
| 14 | | mirrored 1U compute nodes. We'll need to buy some raid hardware to |
| 15 | | make that a easy-to-recover setup, but it will mean a more reliable |
| 16 | | home directory in the long run. This also gives us the chance to fix |
| 17 | | or address the drive bay in ipp004 that is inactive (we've been |
| 18 | | running at raid 5 as a result for > 2 years). There is still some |
| 19 | | re-organization work needed, which was made difficult to arrange with |
| 20 | | nearly the whole team gone to Harvard, but that should be finished by |
| 21 | | early the coming week. |
| 23 | | For my part, in addition to recovery from too much travel, I've been |
| 24 | | working on the extended source analysis. I'm now fairly happy with |
| 25 | | the radial profile extraction + petrosian analysis, and I'm |
| 26 | | incorporating it back into the psphot code base. I'm also making some |
| 27 | | tests and fixes at a low level of issues that have been reported |
| 28 | | lately, including some trouble converging on a good 2D psf model. the |
| 29 | | coming week, I will continue to focus on psphot. |
| | 9 | |
| | 10 | === Heather Flewelling === |
| | 11 | |
| | 12 | * copied my home files from 004backup to either 053home or to a good place for data |
| | 13 | * restarted replication pantasks on ipp user |
| | 14 | * investigated MD09.20090904 processing from last week |
| | 15 | * burntooled MD09 and MD10 dates |
| | 16 | * found instances where burntool burns files out of order |
| | 17 | * while burntooling (and with lots of NFS glitches) pushed the load on 041 to 130. YIKES. |
| | 18 | * found corrupted raw image file |
| | 19 | * copied both copies (the corrupted and the uncorrupted) to a safe place, neb-culled the corrupted file, neb-replicated the good file |
| | 20 | * processed MD09 through to warp stage |
| | 21 | * started MD09 stacking |
| | 22 | * added processing status to wiki |
| | 23 | |
| | 24 | === Bill Giebink === |
| | 25 | |
| | 26 | This week I finished locating and making notes on code causing seg |
| | 27 | faults and aborts from the psLib tap tests. I started fixing tests and |
| | 28 | hopefully my work will still be there when ipp000 comes back up. |
| | 29 | |
| | 30 | I also spent some time looking for some cards for Jim Heasley at |
| | 31 | MHPCC, but didn't find what he wanted. And I started working with |
| | 32 | Heather to script the characterization of the reduction process. |
| 33 | | * Cluster woes |
| 34 | | + Checked operations pantasks scripts into repository |
| 35 | | * MOPS support: branches/pap_mops/ |
| 36 | | + Revised publish_file.pl to support revised ppMops, following "ICD lite" |
| 37 | | + System working again |
| 38 | | + Republished !ThreePi_SouthernRegion.090818 and !ThreePi_NorthernRegion.090818 to MOPS |
| 39 | | + Merged into trunk |
| 40 | | * Destreaking: fixed doCopyExtraExtensions in streaksremove --- wasn't actually doing the write, wasn't setting compression appropriately |
| 41 | | * Detection efficiency: branches/pap/ |
| 42 | | + Got first cut of code done |
| 43 | | + Discovered psphotSignificanceImage doesn't seem to be using correct smoothing size: decreased sensitivity, but not too bad? |
| 44 | | + Finally worked out how to calculate rough limiting magnitude |
| 45 | | + Working now, next concern is formatting for output |
| 46 | | {{{ |
| 47 | | @FAKE.EFF F32 0.884 0.838 0.34 0.046 0.026 # Efficiency fractions |
| 48 | | @FAKE.MAG F32 -1 -0.5 0 0.5 1 # Efficiency magnitudes |
| 49 | | FAKE.REF F32 -7.570797 # Efficiency reference magnitude |
| 50 | | }}} |
| | 36 | * Detection efficiency: branches/pap |
| | 37 | * Measure magnitude bias from fake sources |
| | 38 | * Discovered bug in statistics on fake source magnitudes: |
| | 39 | psphotFitLinear and psphotMagnitudes sort the source list) |
| | 40 | * Fixed radius of fake sources (pmReadoutFake) |
| | 41 | * I/O through FITS table in CMF file (chipname.deteff) |
| | 42 | * Optimised through convolving only the pixels of interest |
| | 43 | * Contributes 1.4 sec of 66 sec psphot run time (on 6k square warp, |
| | 44 | unthreaded, optimised on ipp022) |
| | 45 | * Ready to merge into trunk |
| | 46 | * MOPS support |
| | 47 | * Compiled Magic for Jan Kleyna to test comets |
| | 48 | * Magic does not compile with recent gcc (4.3.2 doesn't work; 4.1.2 |
| | 49 | does); mentioned this to Paul Sydney previously, but no response yet |
| | 50 | * Processing |
| | 51 | * Discovered that MD09.20090904 was done without PATTERN turned on |
| | 52 | * Stack/Diff convolutions |
| | 53 | * Assisting Draper/Metcalfe (stack) and MWV/Huber (diff) |
| | 54 | investigations |
| | 55 | * Wrote up stamp selection algorithm for ps-ipp-users |
| | 56 | * Fixed ppStack so that extended sources can't be used as stamps for |
| | 57 | PSF matching |
| 61 | | In the upcoming week he will be 'releasing' the IPP postage stamp |
| 62 | | server to the consortium. |
| 63 | | |
| 64 | | === Heather Flewelling === |
| 65 | | |
| 66 | | 3 weeks ago |
| 67 | | * culled a few million raw science files. I hope we never have to do |
| 68 | | that again. Learned a few more things about nebulous in the process. |
| 69 | | |
| 70 | | 2 weeks ago |
| 71 | | |
| 72 | | * contacted Natalia. Gave her instructions on compiling ipp and some |
| 73 | | links to read on dvo for when she gets an account on the manoa |
| 74 | | cluster. |
| 75 | | * worked on dvo stats for smf/cmf. Started writing macros to make dvo |
| 76 | | more idl like, but also useful for the dvo stats I'm working on. |
| 77 | | Should be useful for other ipp-users. |
| 78 | | * investigated other raw images files with >2 copies |
| 79 | | * investigated raw image files with <2 copies that are not replicated |
| 80 | | (caused by ipp008 being pulled out of nebulous and the raw image files |
| 81 | | not having the proper xattr bit set) |
| 82 | | * flew to boston for conference |
| 83 | | |
| 84 | | last week |
| 85 | | |
| 86 | | * Attended panstarrs conference in boston |
| 87 | | * ate lobster for the first time |
| 88 | | * met with harvard |
| 89 | | * installed ipp on my mac laptop |
| 90 | | * forced cfitsio, libtool, and libjpeg to install, coped glibtool to |
| 91 | | psLib and psModules directory (renamed to libtool, thanks to a helpful |
| 92 | | hint from our consortium members from NCU), and hacked a few of the |
| 93 | | header files to make it work. Probably forgetting a few other things. |
| 94 | | Not pretty, but it works. |
| 95 | | * spent thursday and friday in michigan (go blue!!) |
| 96 | | |
| 97 | | === Bill Giebink === |
| 98 | | |
| 99 | | * Continued working on psLib tap tests throughout week |
| 100 | | * Restart and maintenance of DIMM on Tuesday |
| 101 | | * Observed with Natalia Wednesday night |
| | 63 | * created a web page to allow users to upload a postage stamp request table for testing |
| | 64 | * modified the request form page to use a session to make it more user friendly |
| | 65 | * added user authentication to the pstamp site |
| | 66 | * fixed various bugs uncovered in testing |
| | 67 | * wrote some wiki pages for describing how to create requests files |
| | 68 | * lots of testing |
| | 69 | * started updating instructions on installing the ipp data store software |
| 105 | | * Changes to ppImage to allow optional masking of unfit burntool |
| 106 | | trails as well as pixels repaired by burntool. |
| 107 | | |
| 108 | | * Harvard meeting. |
| 109 | | |
| 110 | | * Began investigation of spurious transient sources noted by Armin |
| 111 | | Rest to determine if they are really caused by burntool, and whether |
| 112 | | ppImage masking could fix this. |
| | 73 | * tracked down spurious supernova transients for Armin Rest, and determined that they were not burntool related. |
| | 74 | * merged my svn branch to the trunk, adding the cleanup, addstar stage, and burntool tools. |
| | 75 | * fixed all known bugs arising from that merge. |
| | 76 | * worked with John Tonry on implementing different noise thresholds for burntool. Have not determined that the current value of 20 isn't already good enough. |
| | 77 | * attempted to implement a burntool_state variable in the register stage. |
| | 78 | * gave a talk. |
