Heterogeneous MPI (HeteroMPI)

Programming tools

Downloaded 6578 times

Authors:

Alexey Lastovetsky

Heterogeneous MPI (HeteroMPI) is an extension of MPI for programming high-performance computations on heterogeneous networks of computers (HNOCs). It allows the application programmer to describe the performance model of the implemented algorithm in a generic form. This model allows for all the main features of the underlying parallel algorithm, which have an impact on its execution performance, such as the total number of parallel processes, the total volume of computations to be performed by each process, the total volume of data to be transferred between each pair of the processes, and how exactly the processes interact during the execution of the algorithm. Given the description of the performance model, HeteroMPI tries to create a group of processes that executes the algorithm faster than any other group of processes.

HeteroMPI provides all the features to the user to write portable and efficient parallel applications on HNOCs. These features automate all the essential steps involved in application development on HNOCs:

1) Determination of the characterization parameters relevant to the computational requirements of the applications and the machine capabilities of the heterogeneous system. The machine capabilities are determined before the application execution and supplied to the model of executing network of computers. The model of the executing network of computers is implementation-dependent.
We use a static structure automatically obtained by HeteroMPI environment and saved in the form of an ASCII file. However, the parameters of the model can be updated at runtime taking into account the changing network loads.

2) Decomposition of the whole problem into a set of sub-problems that can be solved in parallel by interacting processes. This step of heterogeneous decomposition is parameterized by the number and speeds of processors and the latencies and bandwidths of the communication links between them, and the user-available memory capacity of the machine. The Heterogeneous Data Partitioning Interface (HDPI) is developed to automate this step of heterogeneous decomposition. HDPI provides API that allows the application programmers to specify simple and basic partitioning criteria in the form of parameters and functions to partition the mathematical objects used in their parallel applications.

3) Selection of the optimal set of processes running on different computers of the heterogeneous network by taking into account the speeds of the processors, and the speeds and the bandwidths of the communications links between them. During the creation of this set of optimal processes, HeteroMPI runtime system solves the problem of selection of the optimal set of processes running on different computers of the heterogeneous network using an advanced mapping algorithm. The mapping algorithm is based on the performance model of the parallel algorithm in the form of the set of functions generated by the compiler from the description of the performance model, and the performance model of the executing network of computers, which reflects the state of this network just before the execution of the parallel algorithm.

4) Application program execution on the HNOCs. The command line user interface of HeteroMPI consists of a number of shell commands supporting the creation of a virtual parallel machine and the execution of the HeteroMPI application programs on the virtual parallel machine. The notion of virtual parallel machine enables a collection of heterogeneous computers to be used as single large parallel computer.

Old versions of HeteroMPI (prior to 1.3) use mpC-3.0, which is to be installed separately. Starting from HeteroMPI-1.3, mpC-4.0 is included into the HeteroMPI packages.