Configuration of MPI process

Classes
struct	fupermod_process_conf
Defines
#define	FUPERMOD_DEVICE_MAX_STRING   12
#define	FUPERMOD_BIND_MAX_STRING   256
#define	FUPERMOD_SUBOPTION_MAX_STRING   256
#define	FUPERMOD_DEVICE_DEFAULT   cpu
Functions
void	fupermod_conf_free (fupermod_process_conf conf)
fupermod_process_conf	fupermod_get_conf (MPI_Comm comm, char *filename)
fupermod_process_conf *	fupermod_get_conf_all (char *filename, int *size)
fupermod_process_conf *	fupermod_get_conf_all_sorted (MPI_Comm comm, char *filename, int *size)
void	fupermod_print_conf (MPI_Comm comm, int root, FILE *file, char *default_device_type, char *subopts)
int	fupermod_comm_intra (MPI_Comm comm, MPI_Comm *comm_intra)
int	fupermod_comm_inter (MPI_Comm comm, MPI_Comm *comm_inter)
char **	fupermod_gather_hostnames (int root, MPI_Comm comm)
char **	fupermod_allgather_hostnames (MPI_Comm comm)

---

Detailed Description

Provides configuration settings specific to each MPI process.

• Hardware to run on.
• File names to read and write to.
• Distribution to use.

Allows the distribution outputted by the partitioner to be associated with a specific device on a host. Devices are addressed by their hostname and rank_intra (MPI rank within a node). The device_type parameter can be used to cause the execution of the kernel or routine to be done on a specific device, eg. cpu or gpu, however since it is device_type char* more specific device types such as gpu1, gpu2, fpga etc. may be used.

This data is stored in fupermod_device_info structure and input and output to conf_file. It is also used in file names of FPMs and in the partition.dist file.

Example conf_file

#hostname                      rank_intra  bind  device_type  suboptions
adonis-1.grenoble.grid5000.fr  0           na     gpu           <some optional params for gpu on adonis-1: device_id_start number_devices memory_alloc_mode algorithm is_overlap>
adonis-1.grenoble.grid5000.fr  1           na     cpu           <some optional params for first cpu on adonis-1>
adonis-1.grenoble.grid5000.fr  2           na     cpu           
edel-1.grenoble.grid5000.fr    0           na     cpu           OMP_NUM_THREADS=8 <If implemented, this MPI process could use all cores on host>

Example FPM filename:

adonis-1.grenoble.grid5000.fr.0.gpu.fpm

Example partition.dist file:

#size 4
#D 1400
#host                          rank_intra  d    time       i  x       complexity  speed
adonis-1.grenoble.grid5000.fr  0           800  1.000e-02  0  0       800         80000
adonis-1.grenoble.grid5000.fr  1           100  1.000e-02  1  800     100         10000
adonis-1.grenoble.grid5000.fr  2           100  1.000e-02  2  900     100         10000
edel-1.grenoble.grid5000.fr    0           400  1.000e-02  3  1000    400         40000

---

Define Documentation

#define FUPERMOD_DEVICE_MAX_STRING   12

Maximum lenght of the device string

#define FUPERMOD_BIND_MAX_STRING   256

Maximum lenght of the string for process/thread binding

#define FUPERMOD_SUBOPTION_MAX_STRING   256

Maximum lenght of the string for suboptions

#define FUPERMOD_DEVICE_DEFAULT   cpu

Default device

---

Function Documentation

void fupermod_conf_free

(

fupermod_process_conf

conf

)

Frees memory allocated in fupermod_process_conf

fupermod_process_conf fupermod_get_conf	(	MPI_Comm	comm,
		char *	filename
	)

Reads a file and returns the configuration struct for that MPI process.

Parameters:

	comm	MPI communicator (should be MPI_COMM_WORLD).
	filename	filename string (default: ./conf_file).

fupermod_process_conf* fupermod_get_conf_all	(	char *	filename,
		int *	size
	)

Reads a machine file and returns an array of configuration structs. This function does not use MPI communicator and hence is suitable for serial programmes such as Partitioner. conf's are ordered as per machine file.

Parameters:

	filename	(default: conf_file).
	size	on return will be the size of that array and equal to the size of comm

fupermod_process_conf* fupermod_get_conf_all_sorted	(	MPI_Comm	comm,
		char *	filename,
		int *	size
	)

Reads a file and returns an array of configuration structs in the same order as processes MPI rank.

Parameters:

	comm	MPI communicator (should be MPI_COMM_WORLD).
	filename	(default: conf_file).
	size	on return will be the size of that array and equal to the size of comm

void fupermod_print_conf	(	MPI_Comm	comm,
		int	root,
		FILE *	file,
		char *	default_device_type,
		char *	subopts
	)

Prints a default usable conf_file for all of the processes in the current mpirun. This file can then be customised further by the user.

Parameters:

	comm	MPI communicator (should be MPI_COMM_WORLD).
	root	root processor to print
	file	File pointer to output file (default: conf_file).
	default_device_type	prints all device_type's to be this (eg "cpu").
	subopts	Suboptions passed to kernal

int fupermod_comm_intra	(	MPI_Comm	comm,
		MPI_Comm *	comm_intra
	)

Creates multiple communicators, each consisting of the processes running on the same processor.

Note:

Do not forget to free the result communicators.

Parameters:

	comm	MPI communicator
	comm_intra	intra communicator (non-MPI_COMM_NULL at all processes)

Returns:

error status

int fupermod_comm_inter	(	MPI_Comm	comm,
		MPI_Comm *	comm_inter
	)

Creates a single communicator consisting of the processes running on different processors

Note:

Do not forget to free the result communicator.

Parameters:

	comm	MPI communicator
	comm_inter	inter communicator (non-MPI_COMM_NULL only on the first process running on the processor)

Returns:

error status

char** fupermod_gather_hostnames	(	int	root,
		MPI_Comm	comm
	)

Gathers hostnames from all processors in comm to root process. Returns to root process a pointer to array of size: MPI_MAX_PROCESSOR_NAME * size.

Parameters:

	root	root processor to hold result
	comm	MPI communicator

char** fupermod_allgather_hostnames

(

MPI_Comm

comm

)

Same as fupermod_gather_hostnames but all processors will have result.

Parameters:

comm

MPI communicator