ProcessModule 2.0

ProcessModule: The backbone of ParaView

ProcessModule is arguably the backbone of the ParaView framework. As the name vaguely suggests, ProcessModule is supposed to manage the "process". That includes following primary functions:

1. Initialize the processes
2. Manage/Enable interprocess connectivity
3. Manage/Enable interprocess communication
4. Provide core functionality such as GatherInformation(), SendMessage() (or PushState() etc. in the ServerManager 2.0 design).

ProcessModule however ended up being the global-singleton accessible by everyone and hence became a kitchen sink. Currently it has any and every thing starting with undo-redo (are you kidding me!), plugins, directory listing -- you name it.

The goals of this proposal are:

1. Make process initialization simple
2. Stick with original functions for ProcessModule -- clean up crud accumulated over the years
3. Add new functionality for bi-directional interprocess communication necessitated by collaboration.

Process Initialization

Currently, initialization of process is split up between vtkPVMain and vtkProcessModule. There's no real reason for this split. Instead we provide 2 static methods on vtkProcessModule (modeled after vtkPVMain that deal with the process and process module intialization.

class vtkProcessModule : public vtkObject
{
  ...
public:
  enum ProcessType
    {
    CLIENT, /* Capable of connecting to remote server or self.
               Cannot run in Parallel */
    SERVER, /* data-server+render-server */
    DATA_SERVER, /* data-server */
    RENDER_SERVER, /* render-server */
    BATCH, /* Capable of running in parallel with root node acting as client.
              Cannot connect to remote server */
    SYMMETRIC_BATCH /* Same as BATCH except, every node acts as a client */
    };

  // Description:
  // Returns the process type.
  static ProcessType GetProcessType();

  // Description:
  // Initializes the process. The function is to initialize MPI if applicable
  // for the process and setup some environment e.g. DISPLAY.
  // This will also create the ProcessModule singleton and initialize it.
  static bool Initialize(ProcessType type, int argc, char** argv);


  // Description:
  // Finalizes and cleans up the process.
  static bool Finalize();
  ...
};

So now a typical main() would look as follows:

int main(int argc, char** argv)
{
  vtkProcessModule::Initialize(vtkProcessModule::SERVER, argc, argv);

  ....


  vtkProcessModule::Finalize();
  return 1;
}

• Unlike existing code, none of this methods block on satellite or server. All these methods return and then the vtkProcessModule will provide API to start waiting for activity either on the MPI channels or the client-server socket-channel.

• Also ProcessType is now a globally defined attribute for that process. No more hidden access to process information using come hidden API on vtkPVOptions.

• Note the absence of ProcessModuleGUIHelper. That class disappears. All events are notified using (surprise, surprise) vtk events and even-loop callback is not needed since no call during the process module initialization blocks as before.

Connecting to other processes

Currently, this API is provided by ProcessModule itself i.e. one can create socket connections to other servers using ProcessModule. However, this makes it difficult to add support for different network protocols such as SSL/HTTP etc. To make that easier, we move the create a new abstract class vtkNetworkAccessManager. This class has an API as follows:

class vtkNetworkAccessManager : public vtkObject
{
public:
  ...

  // Description:
  // Creates a new connection given the uri.
  // This call may block until the connection can be established. To keep
  // user-interfaces responsive, one can listen to the vtkCommand::ProgressEvent
  // fired periodically by this class while waiting.
  //
  // vtkNetworkAccessManager can  be waiting for atmost one connection at a
  // time. Calling NewConnection() while another connection is pending will
  // raise an error.
  //
  // To abort the connection and cancel the waiting, simply call
  // AbortPendingConnection() in the vtkCommand::ProgressEvent callback.
  //
  // Returns the new connection instance on success, otherwise NULL.
  //
  // Example uris for client processes:
  //   * builtin://  -- no remote connection
  //   * cs://<hostname>:<port>
  //   * csrc://:<port> -- reverse connect
  //   * cdsrs://<dshost>:<dsport>//<rshost>:<rshost>
  //   * cdsrsrc://:<dsport>//:<rsport>
  // Example uris for server processes:
  //   * csrc://<hostname>:<port> -- reverse connect.
  //   * cs://:<port> -- wait for 1 client to connect
  //   * csm://:<port> -- wait for multiple clients.
  virtual vtkNetworkConnection* NewConnection(const char* uri);

  // Description:
  // Used to abort pending connection creation, if any. Refer to
  // NewConnection() for details.
  virtual void AbortPendingConnection();
  
  // Description:
  // Process any network activity.
  virtual int ProcessEvents(unsigned long timeout_msecs);

};

• vtkNetworkConnection is basically an encapsulation of the vtkMultiProcessController(s) involved. It handles the initial handshake among the processes as well. API may be something as follows:

class vtkNetworkConnection : public vtkObject
{
public:
  ...

  // Description:
  // Provides access to the controller of a given type, in case this connection
  // has multiple controllers eg. DataServer-RenderServer connection.
  virtual vtkMultiProcessController* GetController(int type);

  ...
};

• Subclasses will handle the actual handshake between processes and setting up the controllers for communication.
• vtkNetworkConnection API merely defines the communication mechanism, it does not define the application-level protocol for communication between the processes. That is defined by a vtkSession. vtkSession uses a vtkNetworkConnection to send messages, but what kinds of messages are sent and what to do when such a message is received is defined in a vtkSession subclass.

vtkSession : Defining communication protocol

• The vtkNetworkConnection mentioned in the previous section merely defines the transport layer connectivity between processes. It does not define the protocol for communicating over the tcp/ip connection(s) established. That's done by vtkSession.
• vtkSession is an encapsulation of the transport channels to add communication protocol over them e.g. we will have a vtkProcessModuleSession (I know, the name sucks, any suggestions?) which defines the API to SendState(), GatherInformation() etc. (the API that was previously on ProcessModule).
• Now, if we want to develop a custom application that opens up a new connection between the client and server to send arbitrary data, say for streaming, in parallel with the regular communication channel, then that's easily possible by creating a new vtkSession subclass. Now I am free to add my own RMI handlers, define my on RMI tags in this vtkConnection subclass and handle them as I please.
• Now we have two options, we can still leave the API to GatherInformation(), SendState() on ProcessModule() which then forwards it to the vtkProcessModuleConnection, or expect the vtkSMRemoteObject to locate the vtkProcessModuleSession for the given id and then use API on it to communicate. We'd probably go for the latter, keeping the ProcessModule API easy to read.

Managing multiple vtkSession instances

• ProcessModule will act as session tracker (a session was formerly known as a connection, however it lead to confusion when talking about render-server-data-server connections together and the individual connections between the 3 processes. The term Session makes it possible to visualize multiple connections underneath without confusion). It will provide API to register and access sessions using ids, something as follows:

class vtkProcessModule : public vtkObject
{
public:
  ...

  // Description:
  // Registers a new session. A new ID is assigned for the session and
  // that ID is returned. The ID can be used in future to access this
  // session.
  vtkIdType RegisterSession(vtkSession*);

  // Description:
  // Unregister a session given its ID. This is the same ID that is returned
  // when the session was registered. Returns true is the session was
  // unregistered. Unregistering a session implies that the ProcessModule
  // will no longer monitor communication on the sockets, if any, in the
  // session.
  bool UnRegisterSession(vtkIdType sessionID);

  // Description:
  // Returns the session associated with a given ID.
  vtkSession* GetSession(vtkIdType);

  // Description:
  // Returns a new session iterator that can be used to iterate over the
  // registered sessions.
  vtkSessionIterator* NewSessionIterator();

  ...
};

Monitoring activity

If a processes expect messages from another processes without having triggered them explicitly, then that process must periodically monitor the network for any activity. That can be done by using vtkNetworkAccessManager::ProcessEvents() method. Since all network connections are initialized through the vtkNetworkAccessManager, it has information about all of them.

Creation of a concrete vtkSession

• vtkSession defines the interprocess communication protocol. Processes will simply instantiate the session of the type they need, and the register it with the ProcessModule. vtkSession may go to the vtkNetworkAccessManager to create a vtkNetworkConnection if needed.
• In case of server processes that accept multiple client connections, the first call to vtkNetworkAccessManager::NewConnection() will block till a client connects. It will then leave the server-socket open, and when a new client-connects to that port, it will fire an event with the "uri" on which the connection was received. The server will have to have an observer for this event that will instantiated a new vtkSession which would call NewConnection() with the same uri and the client connection will be established a vtkNetworkConnection instance returned.

vtkClientServerInterpreter

• Currently ProcessModule also houses a global instance of vtkClientServerInterpreter. This makes things complicated especially with multiple sessions.
• To make things more flexible, ProcessModule no longer houses an interpretor. In fact it doesn't say anything about who holds an interpretor? how many? etc.
• To simplify initialization of an interpretor and updating of it when a plugin is loaded, we provide a single vtkClientServerInterpreterInitializer which is aware of all interpreters and can initialize them and keep them updated. This class has nothing to do with ProcessModule and it totally separate.
• vtkProcessModuleSession can house an interpreter for itself. On the server side, to simplify synchronization between multiple clients, all sessions share the interpreter, while on client, each session has it's own interpreter (this is non-issue for most part since we don't support multiple server connections from clients yet).

// vtkClientServerInterpreter instances need to initialized before they can be used.
// Also as and when new plugins are loaded, we need to update the already
// existing instances of the vtkClientServerInterpreter with the new
// information. To streamline this process, we have singleton
// vtkClientServerInterpreterInitializer. Once simply registers an interpretor
// with this class or use NewInterpreter() to create a new interpreter and
// that's it. It will be initialized and updated automatically.

class vtkClientServerInterpreterInitializer : public vtkObject
{
public:
  ...

  // Description
  // Provides access to the singleton. This will instantiate
  // vtkClientServerInterpreterInitializer the first time it is called.
  // This class does not have a public static New() function.
  static vtkClientServerInterpreterInitializer* GetInitializer();

  // Description:
  // Creates (and registers) a new interpreter.
  vtkClientServerInterpreter* NewInterpreter();

  // Description:
  // Registers an interpreter. This DOES NOT affect the reference count of the
  // interpreter (hence there's no UnRegister).
  void Register(vtkClientServerInterpreter*);

//BTX
  typedef void (*InterpreterInitializationCallback)(vtkClientServerInterpreter*);

  // Description:
  // Use this method register an interpreter initializer function. Registering
  // such a callback makes it possible to initialize interpreters created in the
  // lifetime of the application, including those that have already been
  // created (but not destroyed).
  void RegisterCallback(InterpreterInitializationCallback callback);

  ...
};

Logging and Debugging

• This is critical, we need to ensure that it's possible to save detailed logs for any communication between processes. The burden probably falls on the vtkConnection, but that will become obvious once we start implementing.
• Whenever possible vtkNetworkConnection should have a mode for synchronized execution to facilitate debugging.