Changeset 36630 for trunk/Ohana/src/relphot/doc/parallel.txt

Timestamp:

Apr 1, 2014, 12:03:17 PM (12 years ago)

Author:

eugene

Message:

merge changes from EAM dev branches 20140206 (relastro, relphot extensions to parallel processing)

Location:

trunk

Files:

• . (modified) (1 prop)
• Ohana/src/relphot/doc/parallel.txt (modified) (1 diff)

trunk/Ohana/src/relphot/doc/parallel.txt

@@ +1 @@
+
+2014.02.15
+
+Nearly done with the relphot mods.  some outstanding questions;
+
+ * image vs mosaic?
+ * how do decide which images need to be shared?
+
+2014.02.14
+
+ more relphot notes:
+
+ * I load the catalogs, with a  
+
+2014.02.12
+
+I am making progess on the relphot -parallel-regions implementation.  some things I need to deal with:
+
+ * assign_images: select mosaics for all images saved for a given host (need to supply them as well)
+   ** I need to decide on the Image / Mosaic split.  If I am calibrating by image, then the image center defines ownership
+   ** if I am calibrating by exposure, then the mosaic center must define ownership
+ o determine SkyList covering the images for region host
+ o tag detections and objects which I own or do not own
+   o ImageOps.c / matchImage matches detections to images and needs to handle mine / not mine cases
+   o ditto for matchMosaics
+ o load_catalogs : I need to make my requested catalogs unique (just add uniquer to load_catalogs.c:130
+ o figure out what fields I'm saving in meanmags and how to construct it
+   -- mag, objID, catID, photcode, (dmag or other stats?)
+ * from which hosts do I slurp mean mags?
+ o function to merge a new meanmag array into the existing one
+ o how to go from (objID,catID) to a given catalog[i].average[j]
+
+2014.02.06
+
+** a single region-level host owns images for which the center lands
+   in its region.
+
+** it owns detections which come from images which it owns
+   
+** it owns objects which land in its region
+
+Extending parallel relphot processing to split the sky (and images)
+into regions, each of which runs in parallel at the same time:
+
+Top Level (relphot -parallel-images):
+ * define regions of the sky -> hosts
+ * load images, assign to hosts
+ * launch region-level jobs on remote hosts
+
+ Region Level (relphot -parallel-images-region)
+  * load my image subset table
+  * request objects and detections for my skyregion
+  * match images & objects, etc
+
+  * update my image parameters
+  * update my detections
+  * write out detections 
+  * load detections from my border hosts
+  * match to my objects
+  * update my objects
+  * write out objects
+  * load objects from my border hosts
+  * update images
+    (iterate N times)
+  * write out image parameters
+
+Top Level
+  * read image parameters, update
+  * update objects
 
 2012.02.13