VODCA:%20View-Oriented,%20Distributed,%20Cluster-based%20Approach%20to%20parallel%20computing

1
VODCA View-Oriented, Distributed, Cluster-based
Approach to parallel computing
Dr Zhiyi Huang Dept of Computer
Science University of Otago New Zealand
2
Motivation

• DSM applications are not as efficient as MPI on
  cluster computers

3
VOPP

• VODCA is a system supporting View-Oriented
  Parallel Programming (VOPP)
• Why a new programming style?
• Improve the performance of DSM applications on
  cluster computers
• Provide a programming style better than MPI
• Message passing is notoriously known as a
  difficult programming style

4
What is a view?

• Suppose M is the set of data objects in shared
  memory
• A view is a group of data objects from the shared
  memory
• ? V, V?M
• Views must not overlap each other
• ? Vi, Vj, i ? j, Vi ? Vj ?
• Suppose there are n views in shared memory
• ? ViM

5
VOPP Requirements

• The programmer should divide the shared data into
  a number of views according to the data flow of
  the parallel algorithm.
• A view should consist of data objects that are
  always processed as an atomic set in a program.
• Views can be created and destroyed anytime.
• Each view has a unique view identifier

6
VOPP Requirements (cont.)

• View primitives such as acquire_view and
  release_view must be used when a view is
  accessed.
• acquire_view(View_A)
• A A 1
• release_view(View_A)
• acquire_Rview and release_Rview can be used when
  a view is only read by a processor.

7
Example

• A VOPP program for a producer/consumer problem

If(prod_id 0) acquire_view(1)
produce(x) release_view(1) barrier(0) a
cquire_Rview(1) consume(x) release_Rview(1)
8
Advantages of VOPP

• Keep the convenience of shared memory programming
• Focus on data partitioning and data access
  instead of data race and mutual exclusion
• View primitives automatically achieve mutual
  exclusion
• View primitives are not extra burden
• The programmer can finely tune the parallel
  algorithm by careful view partitioning

9
Philosophy of VOPP

• Shared memory is a critical resource that needs
  to be used with care
• If there is no need to use shared memory, dont
  use it
• Justification is wanted before a view is created

10
VOPP vs. MPI

• Easier for programmers than MPI
• For problems like task queue, programming with
  MPI is horrific.
• Can mimic any finely-tuned MPI program
• Shared message ? view
• Send/recv ? acquire_view
• Essential differences
• View is location transparent
• More barriers in VOPP

11
Implementation

• VODCA View-Oriented, Distributed, Cluster-based
  Approach to parallel computing
• VODCA version 1.0
• Released as an open source software
• A library run at the user space
• Based on View-based Consistency
• Use an efficient consistency protocol VOUPID

12
View-based Consistency

• Condition for View-based Consistency
• Before a processor Pi is allowed to access a view
  by calling acquire_view or acquire_Rview, all
  previous write accesses to data objects of the
  view must be performed with respect to Pi
  according to their causal order.
• In VOPP, barriers are only used for
  synchronization and have nothing to do with
  consistency maintenance for DSM.

13
Consistency protocols

• They are page based
• Update protocol
• Modify immediately
• Invalidation protocol
• Use a write notice to invalidate a page
• When the page is accessed, a page fault causes
  the fetch of diffs which are applied on the page

14
Consistency protocols (cont.)

• Home-based protocol
• Based on invalidate protocol, but
• For each page, use a copy as its home
• When a diff is created, it is applied to the home
  copy immediately
• When the page is accessed, a page fault causes
  the fetch of the home copy (Pros resolve the
  diff accumulation problem)

15
The VOUPID protocol

• View-Oriented Update Protocol with Integrated
  Diff
• Based on the update protocol
• Diffs of a page of a view are merged into a
  single diff
• The single diff is used to update the page when
  the view is acquired

16
Experiment

• Use a cluster computer
• The cluster computer, in Tsinghua Univ., consists
  of 128 Itanium 2 running Linux 2.4, connected by
  InfiniBand. Each node has two 1.3 GHz processors
  and 4 Gbytes RAM. We run two processes on each
  node.
• We used four applications, Integer Sort (IS),
  Gauss, Successive Over-Relaxation (SOR), and
  Neural Network (NN).

17
Related systems

• TreadMarks (TMK) is a state-of-the-art
  Distributed Shared Memory system based on
  traditional parallel programming.
• Message Passing Interface (MPI) is a standard for
  message passing-based parallel programming. We
  used LAM/MPI.

18
Performance of NN
19
Performance of IS
20
Performance of SOR
21
Performance of Gauss
22
Future work on VOPP

• More benchmarks/applications
• Performance evaluation on larger clusters
• Optimized implementation of barriers for VOPP
• More auxiliary utilities for VOPP programmers
• A view-based debugger for VOPP
• A fault-tolerant system for VODCA

23
Questions?