
stdout_cntl:
  GetJobOutout
    CheckControllerJob
      CheckController
  StartController
  ControllerCommand
    CheckController
    controller_check
    controller_host
    controller_jobstack
    controller_run
    controller_status
    controller_verbose
    DeleteControllerJob
    CheckControllerJobStatus
    SubmitControllerJob
    PrintControllerBusyJobs
    KillControllerJob
    QuitController
    RestartController    
  CheckControllerOutput
    controller_output
    controller_threads







20090322

  We've been surviving with a slightly broken threading / locking
  model.  I would like to clean it up.  Previously, the threads where
  blocking at a very coarse level, and there were certain interactions
  that were not thread-safe.  Here are my notes on re-working this:

  * we have 6 threads:

    * main (top-level parent) : after it spawns the 5 child threads,
      this thread waits for new clients and adds them to the list of
      active clients with 'AddNewClient'

    * clientsThread (ListenClients): this thread is listening for
      commands from the clients; when it receives a complete command,
      it executes the command using 'multicommand'.

    * tasksThread
      CheckTasks
	NextTask



    * jobsThread

    * controllerThread
      CheckControllerStatus : race condition is irrelevant

    * inputsThread
      AddNewInput called by clientThread
      (not sure this is really being used: 
      	   server input (filename) calls 'input'
	   server module (filename) calls 'module'

  * functions which need to be thread-safe:
    * AddNewClient
    * gprint and supporting functions must be thread safe (extensively
      used...)


* race condition is irrelevant for this variable
static int ControllerStatus = FALSE;

static int stdin_cntl, stdout_cntl, stderr_cntl;
GetJobOutput -> CheckControllerJob -> CheckController -> controllerThread
ControllerCommand -> controllerThread / clientThread
CheckControllerOutput -> controllerThread / clientThread
StartController -> controllerThread / clientThread
StopController



static IOBuffer stdout_buffer;
static IOBuffer stderr_buffer;
static int ControllerPID = 0;

/* local static variables to track the controller host properties */
static Host *hosts = NULL;
static int Nhosts = 0;
static int NHOSTS = 0;
























PanTasks Client / Server design

Client

client uses readline to parse the incoming commands commands are only
validated by the client before being sent to the server:

client loop:

   line = readline ()
   validate argv[0]
   send to server
   wait for response?

Server

Server does not use readline at all.  server has two threads.  first
thread listens for commands from the clients.  how many clients?  use
poll to wait for data?  can we manage enough clients with a single
thread?

the second thread runs the scheduler loop: this is the same background
check that is being run by the current model on the timeouts.

server loop:

  check socket for new incoming client connection
  check known clients for new messages
    execute new command
    send response to the client
  run the scheduler for a single loop

client / server communication is modelled after addstar client /
server?

