Changes between Version 5 and Version 6 of External_Supercomputing_Notes

Timestamp:

Dec 18, 2013, 12:40:12 PM (13 years ago)

Author:

watersc1

Comment:

--

External_Supercomputing_Notes

@@ -54 +54 @@
 }}}
 in my .cshrc.
+
+== IPP Running ==
+
+There is no database running on SEC_HOST, which impacts how the IPP can be run.  The ippScripts assume a database exists that can be probed to determine what should be executed.  In addition, the detrend information is stored in the database, which means processing raw images will need to have detrends specified manually.
+
+Despite this issue, I was able to test that the IPP build does work.  Running ppImage
+{{{
+> ppImage -file o5303g0240o.ota67.fits test.out -recipe PPIMAGE CHIP -Db PHOTOM F -Db DARK F -Db FLAT F -Db NOISEMAP F -threads 1 -trace config 89 -Db MASK F -Db NONLIN F -log test.log -tracedest test.trace
+Number of leaks to display: 500
+}}}
+produced all expected outputs from the recipe (taking into account the command line option changes).  For this test, I disabled the detrending manually to avoid having to specify a series of -mask/-dark/-flat options.
+
+== Parallelization ==
+
+The supercomputing resources are managed by Moab, for which I've been using [https://computing.llnl.gov/tutorials/moab/ this page] as documentation.  Briefly, a job script is constructed that outlines the resources required for completion, and this is submitted to the moab scheduler.  The resources are allocated, the job is run, and the job terminates.  This is significantly different than the standard IPP scheduling system, which assumes full access to the computing hardware.
+
+=== stask ===
+
+The example stask scripts seem to provide a path around this limitation.  From my reading of the scripts:
+
+1. A call to the {{{stask.py}}} python script is defined in the {{{stask}}} shell script.  This script also defines a task list.  {{{stask}}} is then submitted to moab as a job.
+1. The moab job constructs a list of nodes allocated to the job.
+1. The moab job passes this node list and the task list to the python script.
+1. The python script (running under moab) connects to the individual node via ssh, changes to the appropriate workdir, and runs the shell script {{{mult.sh}}} with the task parameters.
+1. The shell script (under python as a moab job, on the individual node) runs the requested job under the [http://www.gnu.org/software/parallel/parallel_tutorial.html GNU parallel] framework.
+1. Python completes when the shell scripts are finished on all nodes, clearing the moab job to complete and release the resources requested.
+
+As of 2013-12-18, I have not run an IPP command through this stask scripts.