This lab should be done after the talk MPI Point to Point Communication II. You should complete the Basics of MPI Programming module and lab exercise before starting this tutorial.

This lab will familiarize you with the four communication modes available for point-to-point message passing in MPI (synchronous, ready, buffered, and standard).

Before You Begin

• blocksends.c
• cbrcv.mak - C Makefile;

• blocksends.f90
• fbrcv.mak - Fortran Makefile;

This exercise looks at one piece of information that relates to the relative performance of the four communication modes (synchronous, ready, buffered, and standard): the minimum time spent at a blocking send call. Note that this is not a measure of the time for the communication to complete, or even a measure of the total system overhead.

The blocksends program reports the elapsed (wallclock) time spent at blocking send calls for the four communication modes. All receives are posted before any messages are sent. Different relative timings might be obtained if receives are not posted first.

1. Read through the program and observe the similarity in syntax between the send calls, the extra steps needed to set up a buffered send, and the use of non-blocking receives.

2. Compile the program using one of the Makefiles:
   For C: nmake /f cbrcv.mak blocksends.exe

   For Fortran: nmake /f fbrcv.mak blocksends.exe

3. Run the program on 2 processors using different message lengths. The commands are:
   mpirun -np 2 blocksends.exe message_length_in_number_of_floats(reals)

   Note the time spent at the blocking send for the different communication modes. You may need to do several runs to get a representative timing.

• brecv.c - initial file;
• new_sleep.c - a simple C routine;
• nbrecv.c - solution file;
• cbrcv.mak - C Makefile;

• brecv.f90 - initial file;
• new_sleep.c - a simple C routine;
• nbrecv.f90 - solution file;
• fbrcv.mak - F90 Makefile;

This exercise demonstrates that replacement of a blocking receive with a non-blocking receive can reduce the synchronization overhead for a corresponding standard blocking send, for message sizes greater than 8KB (default buffer size for VIA communications).

brecv.exe is a very bogus program in which task 0 does a blocking send, and task 1 sleeps for ten seconds before executing a blocking receive. The sleep call is intended to simulate time spent in useful computation.

Both the C and Fortran programs call the new_sleep() function to simulate some computations.

1. Compile the brecv.exe program.

   for C Language :

   nmake /f cbrcv.mak brecv.exe

   or for F90 :

   nmake /f fbrcv.mak brecv.exe

   Specify that two nodes will be used, and run the program. It will report the time spent by task 0 on the blocking send.

2. Edit brecv.c (or brecv.f90) to replace the blocking receive with an MPI_Wait call. Post a non-blocking receive before the sleep. Compile and run the code.

3. Compile the new program (nbrecv.exe):

   for C Language :

   nmake /f cbrcv.mak nbrecv.exe

   or for F90 :

   nmake /f fbrcv.mak nbrecv.exe

   Specify that two nodes will be used, and run the program. It will report the time spent by task 0 on the non-blocking send.

4. Analyze the big difference between brecv.exe and nbrecv.exe when specifying a value within vs above the VIA buffer limit. (2048*4 Bytes - default VIA buffer):
   mpirun -np 2 brecv.exe 2048
   mpirun -np 2 brecv.exe 2049

   mpirun -np 2 nbrecv.exe 2048
   mpirun -np 2 nbrecv.exe 2049

When you are done running programs, delete your subfolder on the T: drives. From a login node, return the nodes to the pool using the ccrm command. You may also wish to delete any files you copied into your \Lab\ folder on H:.