Parallel Processing course at an undergraduate college

1
Parallel Processing course at an undergraduate
college

• David E. Hudak
• Chairman, CS Department
• Ohio Northern University

2
Overview

• Motivating undergraduates for the study of
  parallelism
• CS 442 High Performance Computing
• Laboratory exercises
• Conclusions

3
CS 442 High-Performance Computing goals

• Focus on scientific computing domain
• Systems oriented approach
• Lab work on production parallel machines
• OSC was VERY helpful
• Course emphasis comparative programming
• each student writes a program on each machine

4
CS 442 Prerequisites

• Software
• Programming 1-2-3 (was C, now Java)
• Data Structures (C under UNIX)
• Hardware
• Assembly Programming
• Computer Architecture
• Mathematics
• Linear Algebra

5
Overall Course Layout

• Foundations (1 week)
• IBM SP2 (2 weeks)
• Convex Exemplar (2 weeks)
• Cray T3D (2 weeks)
• SGI Power Challenge (2 weeks)

6
Foundations

• Parallelism
• Race conditions and synchronization
• PRAM algorithms
• SIMD/MIMD

7
IBM SP2

• Message-passing systems
• MPI (Message-Passing Interface)
• Application Database management
• Students parallelized mergesort, implemented
  using MPI (in C) on SP2

8
Convex Exemplar

• Shared-memory machine
• 2 hypernodes, 8 processors per hypernode
• NUMA machines
• Application Numeric computing
• Students parallelized FORTRAN code for PDE solver
• Students compared performance of automatic and
  manual parallelization

9
Cray T3D

• Talked about Seymour Cray, CRAY-1
• Cray T3D architecture
• Interconnection networks
• bus, hypercube, toroidal mesh
• Application Numeric computing
• Students parallelized FORTRAN code for LU
  factorization
• Students compared performance of block and cyclic
  data distributions

10
SGI Power Challenge

• Bus-based systems
• Spent a day on BeBox hardware
• SGI bus architecture (split-transaction)
• SGI cache consistency algorithm
• Thread-based SMP parallelism
• Thread at OS level in BeOS, IRIX
• Application Throughput-based parallelism
• thread for each branch of a bank

11
Account Manager Problem
Application
User
12
In Conclusion

• Course was very demanding
• I am glad I took it now that its over
• Students had experience with systems
• Machines made the course
• OSC people were very helpful
• Overall, a good capstone to a Computer Science
  major