Parallel Thoughts

1
Parallel Thoughts

• James Larus
• Microsoft Research
• Manycore Computing Workshop
• June 20-21, 2007

2
Where Did All Those MIPS Go?

• HW improvement facilitated software bloat
• additional functionality
• improved programming abstractions
• OO, components, libraries, JVM/CLR, etc.

Andrew Tanenbaum (Wikipedia)
3
Which is the Chicken?Which is the Egg?

• Were programming abstractions prerequisite for
  rich software?
• Were programming abstractions created by demand?
• What role did 40/yr performance improvement play
  in this process?
• (hide a lot of sins)

4
Why Multicore is a Challenge (1)

• Few (no?) parallel abstraction mechanisms
• most synchronization is not even composable
  Peyton-Jones and Harris
• Lack of consensus on parallel programming model
  reflects weaknesses of alternatives
• Transactional memory is possible exception
• but does not compose with existing functionality,
  such as IO
• Language design is prerequisite to parallel
  programming

5
Why Multicore is a Challenge (2)

• New features must be computationally intensive
• minor elaborations of existing (sequential)
  functionality are sequential
• New way to design software look for
  computationally intensive tasks
• software development is evolutionary (punctuated
  evolution?)
• Few computer scientists or developers think this
  way
• e.g., where are the parallel algorithms?
• Lack of programming abstractions will hurt

6
Ambitious Functionality

• Old idea computer as intelligent assistant
• opposite of game-like immersive environment
• enable me to do my work with less attention
• watch me (everyone), learn from me, repeat my
  actions
• If a car can drive itself for 50 miles, why cant
  my computer schedule a visitor, etc.?
• Hard problems (AI, UI, ML, etc.)
• computationally intensive

7
Architecture of Multicore Applications

• James Larus
• Microsoft Research

8
Computation
Data
9
Rearchitect Software

• Software as/and a service (aka timesharing, thin
  client, etc.)
• server is embarrassingly parallel and benefits
  directly from Moores law
• Move computation as well as data
• Balance client-server/peer-to-peer
• computationally expensive task
• remote from mobile client
• local on Multicore client
• can service offload computation to clients?
• Network bandwidth is variable
• Every application will be a distributed system!
• concurrency as well as parallelism

10
Programming Model

• Threads?
• probably not error prone, tied to address space
• Message passing?
• essential for distributed systems
• hard to program
• What else?
• Actors?