MPI_ALLGATHER can be thought of as MPI_GATHER where all processes, not just the root, receive the result. The jth block of the receive buffer is the block of data sent from the jth process. A similar relationship holds for MPI_ALLGATHERV and MPI_GATHERV. The syntax of MPI_ALLGATHER and MPI_ALLGATHERV are similar to MPI_GATHER and MPI_GATHERV, respectively. However, the argument root is dropped from MPI_ALLGATHER and MPI_ALLGATHERV.

C

int MPI_Allgather	(void* sbuf, int scount, MPI_Datatype stype, void* rbuf, int rcount, MPI_Datatype rtype, MPI_Comm comm )
int MPI_Allgatherv	(void* sbuf, int scount, MPI_Datatype stype, void* rbuf, int* rcount, int* displs, MPI_Datatype rtype, MPI_Comm comm)

FORTRAN

MPI_ALLGATHER	(sbuf, scount, stype, rbuf, rcount, rtype, comm, ierr)
MPI_ALLGATHERV	(sbuf, scount, stype, rbuf, rcount, displs, rtype, comm, ierr)

The variables for Allgather are:

sbuf	starting address of send buffer,
scount	number of elements in send buffer,
stype	data type of send buffer elements,
rbuf	address of receive buffer,
rcount	number of elements received from any process,
rtype	data type of receive buffer elements,
comm	group communicator.

Note: the arguments are the same as MPI_GATHER or MPI_GATHERV except for no root argument.

Example:

• Matrix-vector multiplication of matrix distributed by rows
• Output vector needed in entirety by all processes

Sample Code

      DIMENSION A(25,100), b(100), cpart(25), ctotal(100)

      DO I=1,25
            cpart(I) = 0.
            DO K=1,100
                  cpart(I) = cpart(I) + A(I,K)*b(K)
            END DO
      END DO
      call MPI_ALLGATHER(cpart,25,MPI_REAL,ctotal,25,
            MPI_REAL, MPI_COMM_WORLD, ierr)

or

 
      double a[100,25], b[100], cpart[25],ctotal[100];
      for(i=0;i<25;i++)
      {
         cpart[i]=0;
         for(k=0;k<100;k++)
         {
             cpart[i]=cpart[i]+a[k,i]*b[k];
         }
      }
      MPI_Allgather(cpart,25,MPI_REAL,ctotal,25,MPI_REAL,MPI_COMM_WORLD);