Non-intrusive and incremental evolution of Grid programming systems

About

The goal of this project is to investigate if an existing Grid programming system can be enabled with new features in a non-intrusive and incremental way. Non-intrusiveness means that the original system does not change and the new features are provided by a supplementary software component working on the top of the system. Correspondingly, all applications not requiring those new features will only use the basic original software and be developed and executed in the same way both in the original and modified systems. Increment means that the supplementary software component does not have to be installed on all computers to enable applications with the new features. It can be done incrementally, step by step, and the new features will be enabled in part, with the completeness dependent on how many nodes participating in the execution of the application have been upgraded with the supplementary software component.

Design and Implementation

We use NetSolve as a target Grid programming system an+d developed a supplementary software component for it. The particular feature added to NetSolve is direct communication between remote tasks. The main features of our non-intrusive and incremental approach to enabling direct communications in NetSolve:

1. The component is built on top of NetSolve rather than built into the system.
2. To enable direct communications, the software component does NOT need recompilation or reinstallation of the NetSolve programming system.
3. It uses the existing programming infrastructure and is functioning non-intrusively on the top of the existing functionality.
4. The software component does not have to be installed on all nodes simultaneously. It can be used to enable direct communications between remote tasks incrementally. It allows for remote calls both to tasks enabled for direct communications and to tasks not enabled, within the framework of the same application. Naturally, direct communications are only possible between enabled tasks.

The proposed software component consists of three parts: Client API & Argument Parser, Server Connector and Job Name Service (JNS). Client API provides a uniform interface for the client to make remote procedure calls. Server Connector is on the server side, which is a proxy program responsible for interacting with clients and other Server Connectors to enable direct communications. Job Name Service (JNS) is responsible for registration of procedure upon its invocation during RPC call.

To use our software component:

• Client side: The only thing for client programmers to do is to install the wrapper API and Job Name Service on the client side, then compile the client program with the wrapper library. The wrapper API allows the programmers to explicitly specify the dataflow between remote tasks. So they only need to slightly modify their client code. The principle is quite easy: the programmer just replaces the input/output arguments with handlers as the input/output data.
• Procedure developers: The procedure programmers should do nothing to enable direct communications. They develop their own procedures as usual. The supplementary software component has no effect on both existing procedures and newly added procedures.
• Server administrator: To enable direct communication control on server side, the server administrator needs to register the software component as a new problem file to NetSolve. No re-installation and re-compilation of NetSolve itself are needed.

Downloads

Version	Date	Size	Links	Status
1.0.0	2010-Jan-13	14.93 KB	Download Release notes	Recommended for NI-Connect