Collective Communication II

1. Advanced Features

Hello, this module is MPI Collective Communication II. Our purpose in this brief module is to introduce you to some features of MPI that allow for greater flexibility when you are using MPI collective operations like gather or scatter.

MPI_Scatterv, MPI_Gatherv, MPI_Allgatherv, MPI_Alltoallv

What does the "v" stand for?
varying -- size, relative location of messages

Examples for discussion: MPI_Gatherv and MPI_Scatterv

I guess you could say that this module is brought to you by the letter V! The MPI routines we'll be talking about all end with V--for example, MPI_Scatterv. You can pretend that the V stands for something like "varying" or "variable". This is because these routines allow you to vary both the (1) size and the (2) memory locations of the messages that you are using for the communication. We will take MPI_Scatterv and MPI_Gatherv as examples, but you can easily extend these ideas to MPI_Allgatherv and MPI_Alltoallv.

Advantages
  • more flexibility in writing code
  • less need to copy data into temporary buffers
  • more compact final code
  • ndor's implementation may be optimal
    (if not, may be trading performance for convenience)
  • As you'll see, there are several advantages to using these more general MPI calls. The main one is that there is less need to rearrange the data within a processor before doing an MPI collective operation. Furthermore, when you use these calls you give the vendor an opportunity to optimize the data movement for your particular platform. But even if the vendor hasn't bothered to optimize an operation like MPI_Gatherv for you, it will always be more convenient and compact for you to issue just a single call to MPI_Gatherv if your data access pattern is irregular.