Changes between Initial Version and Version 1 of Pantasks_server_mode

Timestamp:

Feb 24, 2009, 4:23:57 PM (17 years ago)

Author:

trac

Comment:

--

Pantasks_server_mode

@@ +1 @@
+The pantasks "server" mode allows to pantasks to run using a server/client interface.  This is especially handy if you want to be able to control pantasks from multiple locations (e.g., check processing from home) or accounts (e.g., multiple people controlling or eavesdropping on the processing).
+
+There is a small convenience issue when running pantasks in the "server" mode: some commands which you would usually run in a standalone pantasks are only available when wrapped inside a macro; see below.
+
+
+=== Configuration ===
+
+Ensure the following is in your <code>~/.ptolemyrc</code> file:
+<pre>
+PANTASKS_SERVER         XXXXX.ifa.hawaii.edu
+PASSWORD                XXXXX
+#PANTASKS_SERVER_STDOUT  pantasks.stdout.log
+#PANTASKS_SERVER_STDERR  pantasks.stderr.log
+</pre>
+
+ * <code>PANTASKS_SERVER</code> defines the name of the server for the client to connect to.  There's no real need to update this, since we'll use a script to set this on the fly, allowing multiple pantasks servers to be used simultaneously.
+ * <code>PASSWORD</code> provides minimal access control for the pantasks server.  Set this to a unique word.
+ * <code>PANTASKS_SERVER_STDOUT</code> and <code>PANTASKS_SERVER_STDERR</code> define where the output goes that is usually displayed in the pantasks session.  Since setting these also affects the standalone version of pantasks, I recommend leaving them commented out; we'll use a script to set them on the fly.
+
+
+If you are using Nebulous, you will also need to set up an ipphosts.config file.  This config file has METADATA cards that assign a particular host to each chip on the camera, so that nebulous can distribute every  exposure consistently, without piling too many chips onto the same machine.  There is also a metadata card for indicating which hosts are to be used for skycell processing.  Here is an example of how this config file should look:
+
+<pre>
+ipphosts MULTI
+
+ipphosts METADATA
+  camera STR skycell
+  count S32  8
+  sky00 STR  ipp006
+  sky01 STR  ipp007
+  sky02 STR  ipp008 
+  sky03 STR  ipp009
+  sky04 STR  ipp010
+  sky05 STR  ipp011
+  sky06 STR  ipp020
+  sky07 STR  ipp021
+END
+
+ipphosts METADATA
+  camera STR GPC1
+
+  XY01  STR  ipp014
+  XY02  STR  ipp014
+  XY03  STR  ipp014
+  XY04  STR  ipp014
+
+  XY05  STR  ipp023
+  XY06  STR  ipp023
+  XY10  STR  ipp023
+  XY11  STR  ipp023
+
+  XY12  STR  ipp024
+  XY13  STR  ipp024
+
+  < etc. ...  >
+
+  XY75  STR  ipp015
+  XY76  STR  ipp015
+END
+</pre>
+
+
+
+=== Helpful scripts ===
+
+Here are some convenient scripts for starting the pantasks server and client.
+
+ * Save the following script as '''start_pantasks_server''' in some directory on your path
+<pre>
+#!/bin/bash
+# Start up a pantasks server
+# Output is redirected to files
+                                                                                
+server_redirect="-D PANTASKS_SERVER_STDOUT pantasks.stdout.log -D PANTASKS_SERVER_STDERR pantasks.stderr.log"
+                                                                                
+echo pantasks_server $server_redirect
+pantasks_server $server_redirect
+</pre>
+
+ * Save this one as '''start_pantasks_client''' on your path
+
+<pre>
+#!/bin/bash
+# start up pantasks_client
+# 1st argument is the name of the host to connect to
+                                                                                
+server_arg=""
+server_host=$1
+                                                                                
+if [ $server_host ]; then
+   server_arg="-D PANTASKS_SERVER $server_host"
+   echo pantasks_client $server_arg
+   pantasks_client $server_arg
+else 
+   echo "Please provide the hostname of the pantasks server."
+fi
+</pre>
+
+
+=== Starting pantasks ===
+
+Starting the server is as easy as:
+<pre>
+% start_pantasks_server
+bound to port: 2000
+</pre>
+You can start it in the background if you want: it's not going to print anything more to the terminal.
+
+In a different window (or machine or user), start the client:
+<pre>
+% start_pantasks_client mymachine
+</pre>
+where <code>mymachine</code> is the name of the machine on which the server is running.
+
+You should be presented with the usual pantasks environment.  However, some things are subtly different: the environment you see before you is particular to the client, not the server, and it's the server that runs everything.  This is important to keep in mind, and it means that you should bury within macros some commands that you would ordinarily just type on the pantasks command-line.
+
+=== Setting up ===
+
+Here are some useful definitions for controlling pantasks using the server mode:
+ * Save this as server.pro in your pantasks modules directory
+
+<pre>
+macro setup
+    $LABEL = "my_label"
+    $DBNAME = "mydatabase"
+    module pantasks.pro
+    module.tasks
+    all.off
+         # NEXT LINES ARE ONLY NEEDED IF USING NEBULOUS
+    $default_host = machine1
+    queueload tmp -x "cat ipphosts.config"
+    ipptool2book tmp ipphosts -key camera
+         # END NEBULOUS ONLY
+    add.database $DBNAME
+    echo Using database $DBNAME
+    echo Current active label is $LABEL
+end
+ 
+macro change_label
+    $LABEL = $1
+    echo Current active label is $LABEL
+end
+ 
+macro print_label
+    echo Current active label is $LABEL
+end
+
+macro machines
+    controller host add machine1 -threads $1
+    controller host add machine2 -threads $1
+    controller host add machine3 -threads $1
+    controller host add machine4 -threads $1
+end
+</pre>
+
+ * If you're not using Nebulous, you can remove the lines that are specific to Nebulous use (these send jobs to particular machines).
+ * Ensure the <code>my_label</code> is changed to the label of the data you're processing (or <code>"NONE"</code> including quotes if none).
+ * Ensure the <code>my_database</code> is changed to the database you're using.
+ * Ensure the <code>machineN</code> are changed according to your system.
+
+So, now in the pantasks client, you can say:
+<pre>
+pantasks: server input server.pro
+</pre>
+to load the macro file.  Note that the server has to do this command, hence the prefix <code>server</code>.
+
+Here is how you would set up for processing, and add hosts with a maximum of 4 threads each:
+<pre>
+pantasks: setup
+pantasks: machines 4
+</pre>
+
+Now you should be able to simply say:
+<pre>
+pantasks: run
+</pre>
+and pantasks will run as usual.  The <code>status</code>, <code>stop</code> and <code>halt</code> commands are also available as usual (no need to prepend <code>server</code> or anything like that).  But if you want to change the label, then you should do:
+<pre>
+pantasks: change_label "new_label"
+</pre>
+whereas in the standalone pantasks, you could have done:
+<pre>
+pantasks: $LABEL = "new_label"
+</pre>
+Such commands now have to be buried within macros.
+
+=== Shutdown ===
+
+You can quit out of the client at any time by saying:
+<pre>
+pantasks: quit
+</pre>
+
+This does not stop the server from continuing to process.  You can reconnect to the server as you please.
+
+When you're all done, you can force the server to quit by issuing the following command from the client:
+<pre>
+pantasks: shutdown now
+</pre>