Changes between Version 30 and Version 31 of External_Supercomputing_Notes

Timestamp:

Feb 24, 2014, 4:42:05 PM (12 years ago)

Author:

watersc1

Comment:

--

External_Supercomputing_Notes

@@ -5 +5 @@
 The next constraint that I'm running into is that attempting to process the SAS illustrates the need to sort out a better way of transferring data.  The SAS detrend set and the test detrend set I have previously transferred are effectively disjoint.  This means transferring ~9x60 files, bumping "how to transfer data" from a thing to think about once the SAS test has been started to a more active issue.
 
+=== Allocation calculation ===
+
+For chip, I'm assuming the median ppImage command takes time_cost ~150 seconds with 2x threading (based on some manipulation of the database values).  Then, for a bundle containing N ppImage commands, the allocation is calulated as:
+
+{{{
+ proc_per_core = 24
+ fill_factor   = 0.8
+ max_nodes     = 1000
+
+ Core_need = N * Nthreads
+ Node_need = Core_need / proc_per_core
+ Time_need = N * time_cost
+ 
+ time_requested = int((node_need * time_cost) / (fill_factor * max_nodes)) + 1
+ node_requested = int((node_need * time_cost) / (fill_factor * time_requested)) + 1
+
+ N         = 37200
+ Nthreads  = 2
+ time_cost = 150 / 60 / 60
+
+ Core_need = 74400
+ Node_need = 3100
+ Time_need = 1550
+
+ time_requested = int(3100 * 150 / 60 / 60 / 0.8 / 1000) + 1 = 1
+ node_requested = int(3100 * 150 / 60 / 60 / 0.8 / 1) + 1 = 162
+ 
+ unused side calculation:
+ Time_need / (node_request * proc_per_core / Nthreads) = 0.79733 hours
+
+}}}
 == 2014-02-19 ==