CTC VW: Distributed Memory Programming

Virtual Workshop Modules

Distributed Memory Programming

2. Trends
    2.1.1 Technologies
    2.1.2 Internal Parallel
    2.1.3 Clock Rates
    2.1.4 Memory Rates
    2.1.5 Multilevel
    2.1.6 Interleaved
    2.2 Shared Memory
    2.3 Distributed Memory
3. On the Horizon
4. Time Hierarchy
5. Memory Architectures
    5.1 Multicomputers
    5.2 Distributed Shared
6. Programming Models
    6.1 Data Parallel
    6.2 Message Passing
    6.3 Active Messages
7. Strengths, Weaknesses
8. Key Principles
    8.1 Program Hygiene
    8.2 Addressability
    8.3 Communication
    8.4 Collective Operations
    8.5 Blocking
    8.6 Non-Blocking
    8.7 Synchronization
    8.8 Pitfalls
9. Considerations
    9.1 Owner Rule
    9.2 Strategies
    9.3 Scaling
    9.4 Computation
    9.5 Load Balance
    9.6 Models
    9.7 Models (continued)
    9.8 Conclusions
    9.9 Speedup
10. Conclusion

References
Quiz

Evaluation
Questions?

6. Programming Models

Distributed memory architectures are fertile grounds for the use of many different styles of parallel programming, from those emphasizing homogeneity of process but heterogeneity of data, to full heterogeneity of both.



© 2008 Cornell University \| Cornell University Center for Advanced Computing \| Copyright Statement