Point to Point Communication I

2.1.3 Buffered Send

Buffered

Click here for Flash animation

MPI_Bsend (S) copies data to a buffer on the sender's side, then returns

  • timing of corresponding receive is irrelevant
  • data in the original buffer can be modified
  • eliminates synchronization overhead on the sender
  • adds system overhead

The blocking buffered send MPI_Bsend (S) copies the data from the message buffer to a user-supplied buffer, and then returns. The sending task can then proceed with calculations that modify the original message buffer, knowing that these modifications will not be reflected in the data actually sent. The data will be copied from the user-supplied buffer over the network once the "ready to receive" notification has arrived.

Buffered mode incurs extra system overhead, because of the additional copy from the message buffer to the user-supplied buffer. Synchronization overhead is eliminated on the sending task -- the timing of the receive is now irrelevant to the sender. Synchronization overhead can still be incurred by the receiving task. Whenever the receive is executed before the send, it must wait for the message to arrive before it can return.

Another benefit for the user is the opportunity to provide the amount of buffer space for outgoing messages that the program needs. On the downside, the user is responsible for managing and attaching this buffer space. If a buffered mode send requires more buffer space than is available, an error will be generated, and (by default) the program will exit.

Buffer management

User is responsible for managing buffer space with MPI_Buffer_attach (S) and MPI_Buffer_detach (S).

For a buffered mode send, the user must provide the buffer. It can be a statically allocated array, or memory can be dynamically allocated with malloc (C) or ALLOCATE (Fortran). The amount of memory allocated for the user-supplied buffer should exceed the size of the actual message data, as message headers must also be stored.

Memory space is designated as the user-supplied buffer by a call to MPI_Buffer_attach (S) . When it is no longer needed, it should be detached with MPI_Buffer_detach (S) . There can only be one user-supplied message buffer active at a time. It will store multiple messages. The system keeps track of when messages ultimately leave the buffer, and will reuse buffer space. For a program to be safe, it should not depend on such reuse.