MPI objects are the internal representations of important entities such as groups, communicators, and datatypes. To increase program safety, programmers cannot directly create, write to, or destroy objects. They are manipulated via handles that are returned from or passed to MPI routines. An example of a handle is MPI_COMM_WORLD, which accesses a communicator object. You have also encounted the request handle returned by the non-blocking communication calls.

The request object accessed by this handle is the internal representation of a send or receive call. It archives all the information contained in the arguments to the message passing call (but not the message data itself), plus the communication mode and the status of the message.

When a program calls a non-blocking message-passing routine such as MPI_Isend, a request object is created, and then the communication is started. These steps are equivalent to two other MPI calls, MPI_Send_init and MPI_Start. When the program calls MPI_Wait, it waits until all necessary local operations have completed, and then frees the memory used to store the request object. This second step equals a call to MPI_Request_free.

When you call a non-blocking message-passing routine many times with the same arguments, you are repeatedly creating the same request object. Similarly, when you wait for completion of these communications, you repeatedly free the request object.

The idea behind persistent communication is to allow the request object to persist, and be reused, after the MPI_Wait call. You create the request object once (using MPI_Send_init), start and complete the communication as many times as needed (using MPI_Start and MPI_Wait), and then free the request object once (using MPI_Request_free).