| 24 | | |
| | 24 | * LAP: Finished reworking of LAP code to use a one exposure = one processing system. This should cut down on some work by eliminating the need to reprocess an exposure multiple times. This also uses calls to the update system to regenerate the warps for exposures that have been processed as part of that LAP and then cleaned up. This will also help reduce the work load. |
| | 25 | * Processing speed: Rewrote the host allocation code used in the pantasks input files for all servers, allowing us to use a single central host list. This also should ensure that each host is used more efficiently than in the past. This is especially important given the refurbishment of the wave 1 nodes, which now have more available processing power. Preliminary indications show that we are completing more jobs/day than before. |
| | 26 | * Targeting: Shuffle code to retarget the raw OTA data is running. It has a large error rate, but this is due to the paranoid checking it performs before attempting to move the files. Re-examining the files that are causing the errors suggests that the majority are not real, but an artifact of checking data at the ATRC over the slower network. The remainder of error-causing files (usually a file with a bad copy due to the shuffle issues from before June) are being repaired by a script written to do so. |
| | 27 | * Identified possible slow-down in chip stage processing: we are making an unindexed database request for the average ccd temperature across the exposure for each chip processed. This is slow, and as far as I can tell, unnecessary to any part of processing. This query seems to be putting extra load on the gpc1 database, and if we can eliminate it, we may be able to gain in the total throughput. |
