IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 37 and Version 38 of External_Supercomputing_Notes


Ignore:
Timestamp:
Apr 8, 2014, 5:41:44 PM (12 years ago)
Author:
watersc1
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • External_Supercomputing_Notes

    v37 v38  
     1== 2014-04-08 ==
     2
     3The final test to do was the parallelized test, using differing numbers of front end nodes.  I manually launched these jobs, as I haven't organized a proper automatic parallelization over the front end nodes.  I also increased the bundle size to 100MB, which does squeeze a bit more speed relative to the 10MB bundles (54.5 MB/s vs 30.5 MB/s at 32 jobs per node).  The linear fits are simple eye-ball lines, illustrating that we do not scale perfectly between 1 and 4 nodes.  Based on this, we can get gigabit/s speeds using 3 nodes without seriously overtaxing the system with connections.
     4
     5[[Image(rate_by_parallel.png,400px)]]
     6
     7The only block now is to write something that can take input file lists, construct the necessary bundles, and then farm those jobs out across the front end nodes to do the work.
     8
    19== 2014-04-04 ==
    210