Splitting MPI Groups and Communicators

In our previous examples, we defined a variable flag for C++ and Fortran. This variable labeled the process as even or odd as it was divided into the appropriate group. We can use a built-in MPI routine that uses such a labeling to bypass the need to create the new group or groups explicitly in order to set up our communicators. This routine, MPI_COMM_SPLIT, splits the original communicator into subcommunicators based on a labeling of the processes called color and key. The color sorts the processes into communicators, while the key specifies the rank orderint relative to the new communicator.

We can split a communicator into non-overlapping subcommunicators.

The MPI_Comm_split routine splits the entire original group. Each process will belong to one unique subcommunicator, even though the name (“new_comm”) will be the same for each. If we do not want all processes to be a member of some subcommunicator, we can set the “color” to be MPI_UNDEFINED. The process “new_comm” will then have the value MPI_COMM_NULL.

The key specifies the order, not necessarily the value, of the rank relative to the new communicator. Hence if we use the rank in the original communicator, the ordering of the new ranks will be the same as their order to one another in the parent coummunicator. If we do not care about speciying rank order in the new communicator at all, we can pass a value of keys equal to zero and MPI will assign the ranks in the same order as in the parent.

int MPI_Comm_split(MPI_Comm oldcomm, int color, int key, MPI_Comm *newcomm)

MPI_Comm_split(old_comm, color, key, new_comm, ierror)
     TYPE(MPI_Comm), INTENT(IN) :: old_comm
     INTEGER, INTENT(IN) :: color, key
     TYPE(MPI_Comm), INTENT(OUT) :: new_comm
     INTEGER, OPTIONAL, INTENT(OUT) :: ierror

new_comm=comm.Create()
new_comm.Split(color=0, key=0)

Exercise

Rewrite the group-creation exercise to use MPI_Comm_split.

C++

#include <iostream>
#include <cstring>
#include "mpi.h"

using namespace std;

int main(int argc, char *argv[])  {
    int  rank, new_comm_rank, nprocs;
    MPI_Comm   new_comm; 
    int color, key;

    MPI_Init(&argc,&argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &nprocs);

    if (rank%2==0) {
        color=0;
    }
    else {
        color=1;
    }

    key=0;
    MPI_Comm_split(MPI_COMM_WORLD, color, key, &new_comm);
    MPI_Comm_rank (new_comm, &new_comm_rank);

    char message[16];
    strcpy(message,"               ");
    if (new_comm_rank==0) {
        if (rank%2==0) {
            strcpy(message,"The secret is 1");
         }
         else {
            strcpy(message,"The secret is 2");
         }
    }

    MPI_Bcast(message, strlen(message), MPI_CHAR, 0, new_comm);

    cout<<"rank "<<rank<<" new_rank "<<new_comm_rank<<" "<<message<<endl;

    MPI_Finalize();

return 0;
}

Fortran

program mpi_split
use mpi_f08

  integer :: nprocs, rank, new_comm_rank
  integer ::  color, key
  character(len=15) :: message
  type(MPI_Comm)  :: new_comm

  call MPI_Init(ierr)
  call MPI_Comm_rank(MPI_COMM_WORLD, rank)
  call MPI_Comm_size(MPI_COMM_WORLD, nprocs)

  if (mod(rank,2)==0) then
     color=0
  else
     color=1
  endif

  key=0
  call MPI_Comm_split(MPI_COMM_WORLD, color, key, new_comm)

  call MPI_Comm_rank(new_comm, new_comm_rank)

  message="               "
  if (new_comm_rank==0) then
      if (mod(rank,2)==0) then
          message="The secret is 1"
      else
          message="The secret is 2"
      endif
  endif

  call MPI_Bcast(message,len(message),MPI_CHARACTER,0,new_comm)

  print *, 'rank= ',rank,' new_rank= ',new_comm_rank, message

  call MPI_Finalize(ierr)

end program

Python

import sys
import numpy as np
from mpi4py import MPI

comm=MPI.COMM_WORLD
nprocs=comm.Get_size()
rank=comm.Get_rank()

#If not even, then odd
if rank%2==0:
    color=0
else:
    color=1

new_comm=comm.Split(color)

new_comm_size=new_comm.Get_size()
new_comm_rank=new_comm.Get_rank()

message=np.array(['               '],dtype='str')
if new_comm_rank==0:
    if rank%2==0:
        message=np.array(["The secret is 1"],dtype='str')
    else:
        message=np.array(["The secret is 2"],dtype='str')

new_comm.Bcast([message,MPI.CHAR])
print(rank,new_comm_rank,message[0])

Last updated on Nov 17, 2020