Parallel Programming

1
Parallel Programming

• Henri Bal
• (bal_at_cs.vu.nl)
• Vrije Universiteit
• Faculty of Sciences
• Amsterdam

2
Overview

• What is parallel programming?
• Why do we need parallel programming?
• Organization of this course
• Practicum Parallel Programming

3
ICWall

• A large display suitable for
• Education (classroom)
• Modern scientific applications thatneed to
  visualize large data-sets
• 3D applications

4
ICWall an "off-the-shelf" video wall

• High-end graphics is very expensive
• Use tiled display with many projectors
• Use standard (inexpensive) components
• 9 PCs, graphic and network cards, 16 projectors
• Similar idea as cluster computing, but applied
  to graphics
• Current ICWall is strongly simplified version
• 1 PC with 2 projectors

5
(No Transcript)
6
Parallel Programming

• Sequential programming
• Single thread of control
• Parallel programming
• Multiple threads of control
• Why parallel programming?
• Eases programming? Not really.
• Performance? Yes!

7
Famous quote

• Parallel programming may do something to revive
  the pioneering spirit in programming,which
  seems to be degenerating intoa rather dull and
  routine occupation
• S. Gill, Computer Journal, 1958

8
Why do we need parallel processing?

• Many applications need much faster machines
• Sequential machines are reaching their speed
  limits
• Use multiple processors to solve large problems
  fast
• Microprocessors are getting cheaper and cheaper
• Cheap multiprocessors and multicore CPUsbring
  parallel processing to the desktop!

9
History

• 1950s first ideas (see Gills quote)
• 1967 first parallel computer (ILLIAC IV)
• 1970s programming methods, experimental machines
• 1980s parallel languages (SR, Linda, Orca),
  commercial supercomputers
• 1990s software standardization (MPI), clusters,
  large-scale machines (Blue Gene)
• 2000s grid computing combining resources
  world-wide (Globus)

10
Large-scale parallel machines

• Many parallel machines exist
• See http//www.top500.org
• Current 1 is IBM BlueGene/L with 131072 CPUs

131072
131072
11
Challenging Applications

• Modeling ozone layer, climate, ocean
• Quantum chemistry
• Protein folding
• General computational science
• Aircraft modeling
• Handling use volumes of data from scientific
  instruments
• Lofar (astronomy)
• LHC (CERN, high-energy physics)
• Computer chess
• Analyzing multimedia content
• Generating movies

12
About this Course

• Goal Study how to write programs that run in
  parallel on a large number of machines.
• Focus on programming methods, languages,
  applications
• Prerequisites
• Some knowledge about sequential languages
• Little knowledge about networking and operating
  systems

13
Aspects of Parallel Computing

• Algorithms and applications
• Programming methods, languages, and environments
• Parallel machines and architectures

14
Course Outline

• Introduction in algorithms and applications
• Parallel machines and architectures
• Overview of parallel machines, trends in
  top-500
• Cluster computers, BlueGene
• Programming methods, languages, and environments
• Message passing (SR, MPI, Java)
• Higher-level language HPF
• Applications
• N-body problems, search algorithms,
  bioinformatics
• Grid computing Multimedia content analysis
  on Grids (guest lecture Frank Seinstra)

15
Course Information

• Examination
• Written exam based on
• - Reader available electronically from
  Blackboard
• - Lectures
• More information (and slides)
• http//www.cs.vu.nl/bal/college07.html

16
Practicum Parallel Programming

• Separate practicum (6 ECTS)
• Implement ASP SOR algorithms in C/MPI
• Implement IDA search algorithm in Java/RMI
• Test and measure the programs on our DAS cluster

17
More information

• Register by email to ndrost_at_cs.vu.nl (Niels
  Drost) now
• Starts after MPI lecture (mid October)
• See http//www.cs.vu.nl/pp-cursus/ for a
  complete description