Changeset 35352 for branches/eam_branches/ipp-20130307/ippToPsps/jython/fits.py

Timestamp:

Apr 3, 2013, 12:52:33 PM (13 years ago)

Author:

eugene

Message:

changed dvoDetectionFull mysql engine to 'MEMORY' to improve ingest speed

handle errrors in calls to dvopsps (only in detectionbatch.py)

removed a lot of excess verbosity used for debugging / testing

determine the mysql major version number via mysql query

added test option -realgpc to use test ipptopsps databases with a real gpc1 dataset

add version-dependent call to 'set session old_alter_table' to allow
the ALTER IGNORE statements to work on mysql v5.5 machines (like our
desktops).

raise an exception in case the load infile operations fail

remove unneeded 'killLastConnectionID'

extend logging time to milliseconds

allow an exposure to be skipped without aborting loader

check that a P2 image exists in the dvoImage table (otherwise skip the chip)

change the mysql floating point types for the dvoDetection table to use FLOAT/DOUBLE instead of REAL/FLOAT

remove the old test for valid astrometry of a chip (NASTRO >= 50)

this test was crude and not really the right test. it was also used
to verify that an image should / should not be included. now we check
against dvoImages to see if the chip should be loaded or not (DVO
rejects input images for bad astrometry)

skip an exposure if too many detections have been rejected. this is a sign of a problem and should require more attention

File:

• branches/eam_branches/ipp-20130307/ippToPsps/jython/fits.py (modified) (2 diffs)

branches/eam_branches/ipp-20130307/ippToPsps/jython/fits.py

@@ -87 +87 @@
 
     2880 is the magic number that the FITS format uses to break up data: extensions always begin on multiples of 2880 bytes
+
+    Note that we may not be able to distinguish the case where a header is expected from one where the absence of a header
+    just tells us that the header is not available.  In some cases, we may not want to emit the error message ("not found")
+
+    XXX WOW! this block is reading the entire file to get the header.
+    major inefficiency...  I thought the FITS interactions all went
+    through STILTS
     '''
-    def findAndReadHeader(self, name):
+    def findAndReadHeader(self, name, VERBOSE):
 
         found = False
@@ -114 +121 @@
 
             self.file.seek(origIndex, 0)
-            self.logger.errorPair("Could not read header", name)
+            if VERBOSE: self.logger.errorPair("Could not read header", name)
             return
         else: