Context Navigation

Changes between Version 21 and Version 22 of External_Supercomputing_Notes

-              v21
+              v22
 A search reveals that this file is larger than the 60000 byte limit (at 69854).  I suspect this means rewriting things to do multiple parallel runs in series.
+I think the solution to this is to simply break up the srun config into smaller chunks.  Since all chip stage things are likely to be roughly the same length as this one, breaking it in half should work.  The camera stage is one command, and splitting the warp commands (~70ish) in half should be under the limit as well.
+Another issue I've thought about is the reference catalog.  For the chip stage detrends, I'm assuming that constant use will keep them active on the scratch partition.  In addition, we're already precalculating which are needed for the ppImage calls, so keeping these as an entry in the set of files to check existance of is fine.  However, the reference catalog files aren't necessarily used regularly, and I don't know if we know ahead of time which entries will be used (I suspect not).  Therefore, I think instead of treating this as a detrend, it will need to be put into the permanent storage area.  The same is true of the tessellation directories.  A quick check indicates this is ~137G (refcat) and 530M (tess dir) in space.
 == 2014-01-31 ==