CTC VW: Distributed Memory Programming

Virtual Workshop Modules

Distributed Memory Programming

2. Trends
    2.1.1 Technologies
    2.1.2 Internal Parallel
    2.1.3 Clock Rates
    2.1.4 Memory Rates
    2.1.5 Multilevel
    2.1.6 Interleaved
    2.2 Shared Memory
    2.3 Distributed Memory
3. On the Horizon
4. Time Hierarchy
5. Memory Architectures
    5.1 Multicomputers
    5.2 Distributed Shared
6. Programming Models
    6.1 Data Parallel
    6.2 Message Passing
    6.3 Active Messages
7. Strengths, Weaknesses
8. Key Principles
    8.1 Program Hygiene
    8.2 Addressability
    8.3 Communication
    8.4 Collective Operations
    8.5 Blocking
    8.6 Non-Blocking
    8.7 Synchronization
    8.8 Pitfalls
9. Considerations
    9.1 Owner Rule
    9.2 Strategies
    9.3 Scaling
    9.4 Computation
    9.5 Load Balance
    9.6 Models
    9.7 Models (continued)
    9.8 Conclusions
    9.9 Speedup
10. Conclusion

References
Quiz

Evaluation
Questions?

J. Hennessy and D. Patterson, "Computer Architecture" (Morgan Kaufmann, 1990).

R. Hockney and C. Jesshope, "Parallel Computers 2" (Adam Hilger, 1988).

K. Hwang, "Advanced Computer Architecture" (McGraw-Hill, 1993).

G. Almasi and A. Gottlieb, "Highly Parallel Computing" (Benjanim/Cummings, 1994).

W. Gropp, E. Lusk, and A. Skjellum, "Using MPI" (MIT Press, 1994). Using MPI

C. Stunkel, et al, The SP2 Communication Subsystem, IBM Technical Report, Aug. 22, 1994.

T. von Eicken, et al, Active Messages: A Mechanism for Integrating Communication and Computation, 1992 International Symposium on Computer Architecture, pp. 256-266 (ACM Press).



© 2008 Cornell University \| Cornell University Center for Advanced Computing \| Copyright Statement