CS315B Parallel Computer Computing Research Project

1
CS315BParallel Computer Computing
ResearchProject
Bill Dally, Christos Kozyrakis and Kunle
Olukotun Stanford University http//www.stanford.
edu/class/cs315b/
2
End of Uniprocessor Performance
3X
From Hennessy and Patterson, Computer
Architecture A Quantitative Approach, 4th
edition, October, 2006
The free lunch is over! Now in the multicore era
3
The Evolution in the Multicore Era
4
Life in the Multicore Era

• The move to multicore will be a disruptive shift
• Declining single thread performance
• Heterogeneity many thin, few fat, few vector
• A forced shift in the programming model
• New Architecture Opportunities
• A chance for radical redesign of the
  microprocessor
• What are the innovations that will
  reduce/eliminate the extra burden placed on the
  programmer?

5
Novel Opportunities in Multicores

• Dont have to contend with uniprocessors
• Performance plateau
• Perfect scalability not required
• Not your same old multiprocessor problem
• How does going from Multiprocessors to Multicores
  impact programs?
• What changed?
• Where is the Impact?
• Communication Bandwidth
• Communication Latency

6
Communication Bandwidth

• How much data can be communicated between two
  cores?
• What changed?
• Number of Wires
• Clock rate
• Multiplexing
• Impact on programming model?
• Massive data communication is possible
• Data movement is not the bottleneck ? processor
  affinity not that important

1,000X
100 Giga bits/sec
100 Tera bits/sec
7
Communication Latency

• How long does it take for a round trip
  communication?
• What changed?
• Length of wire
• Pipeline stages
• Impact on programming model?
• Fast synchronization communication
• Finer grained parallelism
• Can run real-time apps on multiple cores

10X
200 Cycles
20 cycles
8
Good News High Throughput

• Sun Niagara 2
• 8 cores x 8 threads 64 threads
• Simple cores gt low power
• High throughput/Watt
• Commercial servers
• Request level parallelism
• Performance/Watt important
• Scientific
• Data-level parallelism
• Dense and sparse matrix
• Performance/Watt important

9
Bad News Parallel Programming Gap

• By 2010, software developers will face
• CPUs with
• 20 cores
• 100s hardware threads
• Deep memory hierarchies
• Integrated GPUs with general computing
  capabilities
• 100s hardware threads
• Parallel programming gap Growing divide between
  the capabilities of todays programmers,
  programming languages, models, and tools and the
  challenges of future parallel architectures and
  applications
• MPI, Pthreads, OMP wont suffice
• Automatic parallelization is not general or
  scalable
• parallelism is the biggest challenge since
  high-level programming languages. Its the
  biggest thing in 50 years because industry is
  betting its future that parallel programming will
  be useful. David Patterson

10
Needed Innovations in Parallel Computing

• New application areas that can take advantage of
  parallelism
• Games and virtual worlds
• Data mining
• Cognitive reasoning and machine learning
• New high-level programming models/languages
• Domain specific gt just do it
• Admit high performance implementations
• New medium-level programming paradigms
• Small increase in programming complexity
• Easy to debug functionality and tune performance
• 10 more effort for 90 of potential parallel
  performance
• New architecture support for parallel computing
• Beyond cache coherence
• Easier programming
• High performance
• CS315B Goal generate new ideas

11
Schedule
12
Course Information

• Instructors
• Bill Dally
• Christos Kozyrakis
• Kunle Olukotun
• E-mail cs315b-spr0607-staff_at_lists.stanford.edu
• Office Hours After class or by appointment
• CA
• Jared Casper
• E-mail cs315b-spr0607-staff_at_lists.stanford.edu
• Course Support
• Darlene Hadding

13
Course Information (cont)

• Lectures
• Thursday 415pm-530pm
• Grading
• Class participation 20
• Midterm assignment 40
• Final assignment 40
• Assignments
• 10-15 slides, subset for 5-10 minute
  presentation
• Put slides on Wiki before presentation
• Midterm
• High-level programming languages/models
• Groups of two
• Final
• Application or algorithm analysis
• Individuals

14
What Will You Get Out of CS315B?

• What you put into it
• Research project course
• You do the research
• Little or no traditional lectures gt discussion
• You should be engaged and ready to contribute to
  discussion
• If you have opinions express them!
• Brainstorming we wont be critical