|
Many significant problems, over the entire computational complexity scale, fall into the data parallel model, which basically stands for "do the same thing to all this data":
- Explicit data distribution (via directives)
The data is assumed to have some form of regularity, some geometric shape or other such characteristic by which it may be subdivided among the available processors, usually by use of directives commonly hidden from the executable code within program comment statements.
- Single thread of control
Each processor in the distributed environment is loaded with a copy of the same code, hence single thread of control; it is not necessary, nor expected, that all processors will be synchronized in their execution of this code, although the amount of instruction-separation is generally kept as small as possible in order to, among other things, maintain high levels of processor efficiency (i.e., if some processors have much more work to do than others, even though they're all running off the same code, then it'll turn out that some processors get finished long before the others do, and will simply be sitting there spinning, soaking up cycles and research bucks, until the other processors complete their tasks ... this is known as load-imbalance, and we'll talk more about this later, but it should be obvious even now that it is a bad thing).
- Examples:
- HPF
High Performance Fortran (HPF) is a standard of Fortran language extensions and features supporting, among other things:
- data parallel programming,
- top performance on MIMD and SIMD computers,
- code tuning for various architectures,
- minimal deviation from Fortran 77 and Fortran 90.
Click here for more information on HPF.
- CM Fortran
Connection Machine (CM) Fortran is a Thinking Machines, Inc. Fortran compiler suite optimized for the SIMD and SPMD environments encountered on the CM-2 and -5 series of systems, based on Fortran 8x, a precursor to Fortran-90.
- Cray CRAFT
CRAFT is a Cray Research, Inc. Fortran environment that combines aspects of message passing, explicit shared memory, and implicit global address space paradigms, for the T3D MPP system. The programmer is allowed a wide range of control, from very low-level where virtually every data-distribution and work-partitioning decision is provided, to very high-level where the programmer simply identifies where the parallelism exists and leaves it to the system to determine how to best exploit it.
Click here for more information.
- C*, pC++, ...
Various flavors of "C" and "C++" have been modified to fit into data-parallel environments; C* is the hybrid that runs on CM systems, while pC++ is a data-parallel extension to C++.
|
