IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 33 and Version 34 of External_Supercomputing_Notes


Ignore:
Timestamp:
Mar 21, 2014, 5:28:03 PM (12 years ago)
Author:
watersc1
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • External_Supercomputing_Notes

    v33 v34  
     1== 2014-03-21 ==
     2
     3To fix the time problems from the previous update, I reworked the code to serialize the mkdir/ppImage/dbinfo steps into a single job that's farmed out.  This completed successfully (~49 minutes), and I reran this test today with the previous run moved out of the way.  This prevents the mkdir from succeeding faster for directories that already existed.  This also completed in ~50 minutes, suggesting that this has resolved the timeout issue.  This also works out to ~83% of the request time, close to the expected target.
     4
     5I'm still trying to come up with a useful data-return system, but at least the stage level scripts can largely finished now that a working chip stage is done.
     6
     7=== returning data ===
     8
     9I think this has to be done such that the data is requested by the local side, and then pushed from the remote side (due to the connectivity issues with fetching data with the password system).  My current model is:
     10 * Local script polls for job completion.
     11 * Upon receiving the completion signal, launch the data return script remotely that will send the data back.
     12 * Parse the results of that script, install files into the correct local location, execute database commands.
     13
    114== 2014-03-19 ==
    215