Prerequisites

You should be familiar with Makefiles, parallel programming, and complete the Large Projects module before starting this lab. A basic sense of scripting languages (PERL) is also useful.

---

Overview

This project is a small prototype of a complex application which involves only command line interface and makefile utilities (MS nmake or GNU make). The application "createpath.exe" creates a random trajectory in N-dimensional phase space of length Np points. The whole trajectory is divided into P parts (P - number of processes), and every process creates its own piece of trajectory (binary file). After all computations have completed, all files are stored in a common place and all binary files should be joined in one big file.

The exercise is divided into two parts: compilation and running of the project. First, the application is created by one of the compilers (using Makefiles). After this a set of batch files are created using a PERL script. The job is submitted using these batch files. After the job is finished, all the binary files are joined by another PERL script.

---

Exercise

Initial Setup

Copy the archive file Proj01.tar which contains all source into a separate directory.

      copy H:\VWLabs\Large\Proj01.tar .

Unpack the archive using a command:

      tar -xvf Proj01.tar

and change to the root project directory:

      cd Proj01

The top project directory contains a few directories:

  Proj01  ------+-------- bin (empty)
                |
                +-------- data (initial data)
                |
                +-------- jobs (empty)
                |
                +-------- src ( Fortran90 sources)
                

Project Makefiles

• ifltopwin.mak (header for Intel Fortran Compiler 8.*) - for use with MS nmake
• generic.mak (contains all the generic targets and dependences)

and two PERL script files

• prepjob.pl (script to be used to create run batch files in directory jobs)
• joinpath.pl (script to join all binary files into one file)

are also in the top directory Proj01.

---

Compiling

Open a command shell window with a specific compiler environment ( Intel Fortran 8.*).

If you intend to use the Windows command shell type one of the following commands depending on your compiler preferences.:

  nmake /f ifltopwin.mak

Otherwise you may try the Cygwin Unix environment:

  make -f pgitopwin.mak

which launches the GNU make utility to create the project application.

All the above Makefiles include the generic makefile generic.mak with portable dependencies suitable for both nmake (MS) and make (GNU) utilities. Using such a simple prototype a new compiler can be added very easily by editing one of the custom Makefiles (name of the compiler, compiler flags etc.). This will not affect the generic complexity of the project.

From the other point of view, adding new source and dependencies to the generic makefile (generic.mak) will be applied instantaneously to all of the specific Makefiles (due to the "include"). In this way, updating the project can be separated into two independent tasks: OS and compiler specificity (*top*.mak) and internal complexity (generic.mak).

---

Solution

The script prepjob.pl will create a set of 3 batch files to run a project application created with specific compilers:

   perl prepjob.pl --l=df --t=3 --p=6 --n=2 --u=bob --c=v1

The parameters for the script will create different run configurations in directory jobs:

   --l= (df,pgf90,ifl) - specify the directory where the application 
                          "createpath.exe" is created;
   --u= username;
   
   --t= time in minutes;
   
   --p= number of processes to be created with "mpirun";
   
   --n= numbers of nodes to be allocated;
   
   --c= (v1,vplus,workshop) -types of the nodes.
                            

For this specific set of parameters a subdirectory df will be created in directory jobs:

   jobs ----+---- df ------------ output
            |      |
            |      +-- run.bat
           ...     |
                   +-- setup.bat
                   |
                   +-- cleanup.bat

The subdirectory output will be used to store the results of the submitted job:

       ccsubmit run.bat 

A set of binary files (equal to number of processes) are stored in the subdirectory output as a result of the program "createpath.exe" :

     lcbin000.pth
     lcbin001.pth
       ......
     lcbin005.pth  
     

Run the PERL script if you need to gather all these files in one binary file:

     perl joinperl.pl
     

---

Cleanup

After you are done, you can remove the top directory Proj01 with all subdirectories and files:

        rmdir /S /Q Proj01       
        

---