Introduction to Parallel Programming using MPI

1
Introduction to Parallel Programming using MPI

• _at_ CS TechLunch
• Amit Kumar

Source of the entire presentation
http//www.llnl.gov/computing/tutorials/parallel_c
omp/
2

• Why Parallel Computing
• Parallel Computer Memory Architectures
• Parallel Programming Models
• MPI
• MPI Examples

3
Why Parallel Computing

• Primary reasons
• Save time
• Solve larger problems
• Provide concurrency (do multiple things at the
  same time)
• My Motivation Having Parallel Girl Friends

4
Parallel Computer Memory Architectures

• Shared Memory
• Uniform Memory Access (UMA)
• Non-Uniform Memory Access (NUMA)

5
Parallel Computer Memory Architectures contd.

• Distributed Memory

6
Parallel Computer Memory Architectures contd.

• Hybrid Distributed-Shared Memory
• The largest and fastest computers in the world
  today employ both shared and distributed memory
  architectures.

7
(No Transcript)
8
Parallel Programming Models

• There are several parallel programming models in
  common use
• Shared Memory
• tasks share a common address space, which they
  read and write asynchronously.
• Threads
• a single process can have multiple, concurrent
  execution paths. Ex implementations POSIX
  threads OpenMP
• Message Passing
• tasks exchange data through communications by
  sending and receiving messages. Ex MPI MPI-2
  specification.
• Data Parallel
• tasks perform the same operation on their
  partition of work. Ex High Performance Fortran
  support data parallel constructs.
• Hybrid

9
Parallel Programming Models contd.

• Parallel programming models exist as an
  abstraction above hardware and memory
  architectures.
• Which model to use is often a combination of what
  is available and personal choice. There is no
  "best" model, although there certainly are better
  implementations of some models over others.

10
MPI - Message Passing Interface

• MPI or MPI-1 is a library specification for
  message-passing.
• MPI-2 Adds in Parallel I/O, Dynamic Process
  management, Remote Memory Operation, C F90
  extension
• MPI Standard http//www-unix.mcs.anl.gov/mpi/sta
  ndard.html
• MPI Standard 1.1 Index http//www.mpi-forum.org/d
  ocs/mpi-11-html/node182.html
• MPI-2 Standard Index http//www-unix.mcs.anl.gov/
  mpi/mpi-standard/mpi-report-2.0/node306.htm
• MPI Forum Home Page http//www.mpi-forum.org/inde
  x.html

11
MPI Examples

• A few MPI Implementations
• MPICH-1, MPICH2, LAM/MPI ..