| OCR Text |
Show 3 1.1.3 Multiple Instruction Multiple Data (MIMD) This class includes many different architectures and configurations in which multiple instructions are being applied to multiple data with some degree of concurrency. It can be further subclassified based on a number of criteria such as memory access method, synchronization scheme, communication channels, etc. 1.2 Concurrency, Scalability, Communication, and Synchronization The MIMD and SIMD classes represent what is usually referred to as parallel and/or distributed architectures. It includes various different configurations, but the common characteristic shared by all the various configurations is that they exhibit some degree of concurrency and scalability. Further classifying them and providing precise definitions are rather thorny issues, and beyond the scope of this work. The following discussion will be limited to the general aspects of parallel distributed computation with a subjective lean toward the networked, distributed memory configuration as it is the subject of this work. In this work, concurrency will be loosely defined as having different instructions execute at different computing nodes within the time frame needed to accomplish a certain task or a group of logically related tasks. Informally, scalability is perceived as the ability to divide the work load so it can be carried out by several different computing nodes in an efficient manner, and an algorithm is referred to as scalable over a certain architecture or configuration if the number of computing nodes can be varied over a certain range without inducing prohibitive overhead penalties. This perception is the foundation to the formal definition that an algorithm is considered scalable over a certain range of computing nodes if the parallel speedup is a linear or an almost linear function of the number of computing nodes. Two very important and closely related issues in parallel distributed computing are communication and synchronization. Communication channels between the computing nodes are used to exchange data or protocol messages. They vary from local bus archi- |
