Parallel Processing

1
Parallel Processing

• Presented by Wanki Ho
• CS147, Section 1

2
The single processor problem

• Question Why is parallel processing necessary?

3

• Reasons for needing computational power
• Longer time to completion
• Higher complexity
• More detail
• Solutions must be found within a reasonable time
  by human standards

4

• Examples of computational problems
• Weather prediction
• Air traffic simulations
• DNA mapping
• Big challenges
• Computer AIs

5

• Underlying problem Need more memory and speed
• The solution Parallel processing
• Connect a lot of computers together using an
  interconnection network so that they act together

6
Types of Parallel Processors

• Various types depending on how the processors
  interact with memory
• Flynns Taxonomy
• SISD Single Instruction Single Data Stream
• SIMD Single Instruction Multiple Data Stream
• MISD Multiple Instruction Single Data Stream
• MIMD Multiple Instruction Multiple Data Stream

7

• SISD
• Von Neumann architecture. Your typical everyday
  computer
• MISD
• Wierd stuff. Noone uses.
• SIMD
• Computers with special instructions for handling
  large data in parallel (ex. Array processor in
  Intels MMX chip)

8

• MIMD
• Many processors performing operations on many
  data
• Multiprocessors
• Networks of interconnected computers

9
Parallel Processing Problems

• Multiple processors accessing the same memory.
• What happens if one processor reads and another
  processor writes to the same block of memory?
• How do two processors maintain separate caches
  concurrently?

10
Network Topologies

• Bus Network
• Ring
• Tree
• Mesh
• Hypercube
• Complete Connectivity

11
Ring

• Every computer has two branches connecting it to
  another computer or outside the ring

12
Tree

• Computers are connected to each other in a tree
  structure

13
Mesh

• Each computer is connected to the computer below
  and above and to the left and right of it

14
Hypercube

• Each computer is connected to 4 other computers
• Consistent distance between two computers.
  Useful for weight balancing

15
Complete Connectivity

• Each computer is connected to every other
  computer in the network
• Optimal connectivity
• Complexity grows geometrically

16

• The End