Distributed Memory Programming

8.2 Addressability

As one module in a distributed application, knowing what you know, and, for what you don't who to ask, is one of the central issues in message passing applications. "What you know" is the data you have resident on your own processor; what you don't know" is anything that resides elsewhere, but you've discovered is necessary for you to find out.

  1. CPU can issue load/store operations involving local memory space only

    No big surprise, here: each processor in a distributed architecture is autonomous in the sense that the instructions that it executes, and the memory that it uses, are all under its own control, and no one elses'. By the same token, its own instruction stream, and its own memory are the only things it can be said to control; for everything else, it has to issue the appropriate requests to the appropriate parties.

  2. Requests for any data stored in remote processor's memory must be converted by programmer or run-time library into message passing calls which copy data between local memories.

    You not only have to know that you don't know something, or that something that you used to know is now out-of-date and needs refreshing ... you also need to know where to go to get the latest version of the information you're interested in. This isn't just a "throw-away" remark, it's in fact one of the strongest arguments for the Network-Linda tuple-space model of distributed computing, because, in tuple-space, all you need to do is, in effect, toss something out there with the proper tag, or toss out a request for something with the proper tag, and you get results without ever knowing, or needing to know, who was on the other end of the string. In standard message-passing domains, however, you need to know where you're getting things from, or where you're sending things ... weelllll, and then there's broadcast, but that's another level of complexity, and has its own set of pros and cons -- we'll be getting to that a bit later.

  3. No shared variables or atomic global updates (e.g. counters, loop indices)

    Synchronization is going to cost you, because there's no easy way to quickly get this kind of information to everybody ... that's just one of defining characteristics of this model of operation, and if its implications are too detrimental to the effectiveness of your application, that's a good enough reason to explore other alternatives.