MPI/PRO, from Verari Systems (formerly MPI Software Technology, Inc.), is a message passing library of routines that you can use to parallelize your program. MPI/Pro implements v. 1.2 of the MPI standard, which provides application portability through a high-level interface that also delivers high performance when implemented efficiently. MPI/PRO provides support for TCP and SMP.

 

The MPI/PRO library is installed on the login nodes and the collaboratory.

 

Details:

 

The version of MPI/Pro varies by type of machine as noted on the Software Page.  All batch clusters currently have MPI/Pro 1.7.2.

 

Currently MPI/Pro 1.7.0 is the version on the login nodes.  It is located in C:\Program Files\MPIPro.

Before using MPI/Pro, you have to issue the mpipasswd command on one of the login nodes. It will put an encrypted file, .mpipass, in your home directory. Once you do this, all applications using MPI/Pro should work.

 

As noted in the MPI/Pro Users Guide "MPI/Pro startup procedure exports the entire user environment to the MPI processes. The environment variables and their values that are passed to the MPI processes are the same as in the command prompt window where mpirun is executed." As a consequence, the -export flag to mpirun is no longer recognized by mpirun 1.7.0 and should be removed from your code.

 

Communication Protocols:

 

The only communication protocol that is available on CTC clusters is TCP.

MPI_Barrier:

 

Because MPI/Pro 1.7.0 uses Windows Job Objects to manage MPI jobs, mpirun will terminate tasks on all nodes when the tasks on the master node have terminated. If your code is structured so that the master task might terminate before any of the other tasks, call MPI_Barrier on a communicator that includes the master task before calling MPI_Finalize so that the master task won't finish until all other tasks have been completed.

The MPI calls for MPI_Barrier are as follow, presuming that the communicator is MPI_COMM_WORLD:

Machinemaker:

Machinemaker creates a file with the names of the machines allocated to your batch job.  Set MPI_COMM before calling machinemaker. 

 

Setup file changes:

 

You need to make sure that you are in the root directory of T: and that you do not have a file named T:\%USERNAME% when you try to create a directory of the same name. %USERNAME% is an environment variable that is set to your login id. The first few lines of the setup file should read as below.

       T:
       cd \
       del /Q T:\%USERNAME%
       mkdir %USERNAME%
       cd %USERNAME%
       ...

 

Documentation for MPI/PRO subroutines can be found in the folder: H:\CTC Tools\SoftwareDocs\ .