Problem

Example: Process 0 reads the initial data and distributes it to
other processes, according to the domain decomposition.

 

Problem:What if the number of points can't be divided evenly?

A typical use of MPI_Scatterv might arise when you distribute initial data in a data-parallel code. In other words, we're going to assume you've parallelized your code via domain decomposition. Obviously, if the number of points is not divisible by the number of processes, not all the processes will receive the same number of data points. In this simplified illustration, process 0 has all the data initially, while the colors indicate a possible target distribution of data points across processes including process 0.

---

Solution

Solution: Use MPI_Scatterv with arrays as follows...

(Assume NPTS is not evenly divisible by NPROCS)

NMIN = NPTS/NPROCS

NEXTRA = MOD(NPTS,NPROCS)

K = 0

DO I = 0, NPROCS-1

IF (I .LT. NEXTRA) THEN
SENDCOUNTS(I) = NMIN + 1
ELSE
SENDCOUNTS(I) = NMIN
END IF
DISPLS(I) = K
K = K + SENDCOUNTS(I)

END DO

CALL MPI_SCATTERV(SENDBUF, SENDCOUNTS, DISPLS ...)

or

nmin=npts/nprocs;

nextra=npts%nprocs;

k=0;

for(i=0;i< nprocs-1;i++)

{

if(i< nextra) sendcounts[i]=nmin+1;
else sendcount[i]=nmin;
displa[i]=k;
k=k+sendcounts[i];

}

MPI_Scatterv(sendbuf,sendcounts,displa ...);

The code here divides up the points among processors as evenly as possible. First, it figures out how many extra points there are by using the modulo function. Then it loops over the processes. Via the SENDCOUNTS array, it assigns one extra point to each process until all NEXTRA extra points have been accounted for. And while it's doing this, it's using the DISPLS array to keep track of the relative starting locations of the chunks. Finally, MPI_SCATTERV is called with the calculated SENDCOUNTS and DISPLS arrays in order to perform the desired data distribution.

Thanks for your attention! Here are some references that may be helpful, and please do try the exercises so you can get better acquainted with some of the more advanced features of MPI collective communication.