Implementation of Page Management in Mome, a UserLevel DSM

1
Implementation of Page Management in Mome,a
User-Level DSM

• Yvon Jégou
• Paris Project,
• IRISA/INRIA
• FRANCE

2
Overview

• Mome memory model
• Consistency management
• Multiple writers
• Node page management

3

Background

• Run-time systems for parallel languages
• Object-sharing between parallel programs
• Parallel code coupling
• Persistent data repository
• Checkpointing

4
Mome memory model

• Posix memory model
• Posix mmap Mome MomeMMap
• single node multiple
  nodes

Process 1
Mome space
Process 2
5
Mome memory model
Process 1
Mome space
Process 2
6
F77 common block sharing

• double precision tab(n,m)
• common /arrays/tab
• ..
• call MomeMMap(tab(1,1), sz(tab), WRITE,
• FIXED, seg1, 0)
• ..
• tab(i, j)
• tab(k, l)

7
Mome page management

• Directory-based
• Directory records status of page in each node
• No page home
• (asynchronous) directory migration or
  redistribution

8
Mome page consistency

• call MomeConsistency(tab(1,1),sz(tab),STRONG)
• Each process selects the consistency model at
  page level
• Strong consistency model
• Release consistency model (under development)
• Weak consistency model
• Weak consistency pages made consistent only on
  explicit request
• Example integrate all writes before last barrier

9
Weak consistency management

• Based on a scalar (Lamport) distributed clock
• Page manager valid-before date
• Each node
• Valid-before date
• Consistency-constraint date
• Consistency request updates the
  consistency-constraint date
• Example date of last barrier
• call MomeSyncMem(tab(1,j), n, lastbarrierdate())

10
Weak consistency management

• Valid-before lt Consistency-constraint
• Accesses to page are invalidated
• Access page fault
• Page fault request sent to manager
• Manager new Valid-before date
• Manager new version of page
• Clock shifted on
• Synchronization requests
• Program request

11
Consistency management

• All requests from node to manager contain (date,
  contract)
• Contract consistency model in use during request
• As long as the page is valid on the node, the
  manager guarantees the last contract
• Strong no other writers
• Release no other writer before last
  synchronization
• Weak consistency no restriction
• Manager invalidates the page if the contract
  cannot be graranteed anymore

12
Multiple consistency modes

• Consistency at (page/node) level
• Heterogeneous application on same DSM
• Code coupling (application/coupling library)
• Code optimization optimized code in weak
  consistency
• No global synchronization for consistency
  management

13
Multiple writers in Mome

• Mome favors the single writer case (most frequent
  case)
• No twin on the first writer
• Twinning requested on second writer
• Merging modifications when some consistency
  request is not satisfied
• Modified pages sent to one node with twin
• Result exclusive OR of modified pages and twin
• It is possible to send diffs

14
Multiple mappings

• The same page can be mapped multiple time on the
  same node
• By the same process (synchronous)
• By different processes of the same node
  (asynchronous) persistent data repository
• By a distant process (no shared memory)
• Hierarchical DSM (work in progress)
• Experimented on Mach micro-kernel

15
Internal node page management
Mome Memory
DSM file

• On each node Mome memory is mapped on the DSM
  file (temp file)
• Application pages made accessible through
  aliasing on Mome memory (temp file mmap)

16
Internal node page management
Mome Memory
DSM file
Application space
17
Internal node page management
Mome Memory
DSM file
Application space

• Page fault page made available in Mome memory

18
Internal node page management
Mome Memory
DSM file
Application space

• Page faultapplication page is aliased with page
  in Mome memory

19
Internal node page management
Mome Memory
DSM file
Application space
mmap

• Application process can read/write the page

20
DSM memory management

• Each page reference counter
• Mapping references
• Communication layer references
• Copy-on-write (refcount 1)
• Free page list
• DSM memory management thread
• Free unused pages
• Distant page-out (swapping)

21
Conclusion

• Same DSM kernel for
• Parallel run-time library (HPF, OpenMP)
• OpenMP all shared / explicit sharing
• Library for code coupling
• Persistent data repository
• Close to a well-known memory model (Posix)
• Coupling OpenMP programs

22
(No Transcript)
23
(No Transcript)
24
DSM-based run-time systems for parallel languages

• Run-time library for F77explicitparallel loops
• HPF (SPMD execution model)
• On-going work on OpenMP
• all shared
• explicitly shared

25
Object sharing
D S M
Application 1
Application 2

• Two parallel applications running on the same
  DSM
• Independent adress-spaces

26
Dynamic couplingbetween clusters
Application 1
Application 2
27
Dynamic coupling
Application 1
Application 2
28
Dynamic coupling
29
Persistent data repository
DSM-based data repository
30
Persistent data repository

DSM-based data repository
Application 1
31
Persistent data repository

DSM-based data repository

• Checkpointing of DSM repository