MPI IO Perspectives

1
MPI (I/O) Perspectives

• Rajeev Thakur
• Mathematics and Computer Science DivisionArgonne
  National Laboratory

2
Outline

• What is MPI-IO?
• How does it address the scalable parallel I/O
  problem?
• What does it enable you to do?
• Why is it important?
• What is the status of implementations?
• What are the performance and scalability issues?

3
What is MPI-IO?

• A standard, portable API for parallel I/O
• Part of the MPI-2 Standard, released July 1997
• Designed to support a programming model in which
  multiple processes perform I/O to the same file
  concurrently
• Defined by a broadly based group including
  vendors, users, and I/O researchers from academia
  and laboratories
• Incorporates results of research over the last
  8--10 years in the parallel I/O field

4
How does it address the scalable parallel I/O
problem?

• Defines a standard, portable API that enables
  various optimizations that are essential for
  high-performance parallel I/O
• collective I/O
• noncontiguous file accesses
• user-supplied hints
• multiple consistency modes
• Eliminates the performance (and portability)
  problems of using the Unix API for parallel I/O

5
What does it enable you to do?

• Write application programs in a portable way and
  also achieve high performance
• Leave I/O optimizations to the MPI-IO
  implementation
• Library writers can exploit the features of
  MPI-IO to deliver high performance along with the
  convenience of the higher-level library (e.g.,
  HDF, DMF, Exodus, Silo)

6
Why is it important?

• With the advent of MPI-IO, the I/O API is no
  longer the problem
• Given sufficient I/O hardware and a good file
  system, the MPI-IO implementation can deliver
  high performance to applications
• Performance is limited by what is realistically
  achievable on the given hardware and file system
• I have seen 1 Gbyte/s on NEC SX-4,
  800 Mbytes/s on Argonne Origin2000, and 150
  Mbytes/s on one of the NCSA Origins

7
What is the status of MPI-IO implementations?

• I/O was the first part of MPI-2 to be implemented
• Several implementations are available
• ROMIO, developed at Argonne, is a portable
  implementation that runs almost everywhere
• PMPIO from NASA Ames, MPI-IO on HPSS from
  Livermore
• Several vendors (HP, SGI, NEC) have incorporated
  ROMIO into their MPI impl.

8
What is the status of implementations?

• IBM, Compaq, Fujitsu, Hitachi have MPI-IO
  implementations in various stages of completion
• On clusters
• ROMIO has been implemented on the PVFS file
  system from Clemson (and also on NFS)
• ROMIO works on SGIs XFS, therefore should work
  directly on CXFS
• ROMIO is already being used on IBM GPFS at
  Livermore
• Should also work on GFS (Minnesota) because ROMIO
  works on any file system with a Unix/POSIX API

9
What are the performance and scalability issues
(on clusters)?

• On large clusters with sufficient number of I/O
  servers, the aggregate disk bandwidth is quite
  high
• The network, then, becomes the bottleneck
• Scalability and performance depends on
  scalability of network and networking software
• TCP, even on fast gigabit networks, may not be
  good enough
• We are working with the Clemson folks to redesign
  PVFS to use multiple communication mechanisms
  TCP, VIA, Myrinet, ST, ...

10
MPI (I/O) Perspectives

• Rajeev Thakur
• Mathematics and Computer Science DivisionArgonne
  National Laboratory