KSIX Parallel Environment on Beowulf Cluster

1
KSIX Parallel Environment on Beowulf Cluster

• Thara Angskun
• Choopan Ratanapocha
• Putchong Uthayopos
• Parallel Research Group, CONSYL, Department of
  Computer Engineering
• Faculty of Engineering, Kasetsart University,
  Bangkok, Thailand

2
Agenda

• Motivation
• KSIX Overview
• Related Work
• KSIX Functionality
• KSIX Implementation
• Experimental Evaluation
• Conclusion and Future Works

3
Motivation

• Beowulf Clusters still have a very primitive
  software environment
• Normal UNIX based OS services is inadequate under
  parallel environment
• No global process, file space, signal
• Many additional services such as naming , events
  can simplify high level software tools

4
KSIX Overview

• A layer of software running in user space
• Provides services to applications and tools
  development in cluster environment
• Provides set of API for applications to access
  the services

API
KSIX
Node
Node
Node
Node
Interconnection Network
5
KSIX Overview

• Support
• Global process space
• Global event delivery
• Global naming
• Global ensemble management

6
Design Goal

• Rich set of services for cluster development
• Focus initially on process control, event, and
  naming
• easy to extended
• portable
• Deliver working implementation first so ,
  scalability is less important for now.

7
Related Work

• Mosix / Nomad
• provides automatics handling of resources
• Process Migration
• work at kernel level
• Bproc
• provides transparent global process control
• lack many features such as naming service, event
  service

8
Related Work

• PVM and MPI
• provide parallel virtual machine view
• portable parallel programming model(message
  passing)
• Partially implement some needed API such as
  global process control, naming and event service,
  fault tolerant RPC

9
Related Work

• Glunix
• focus on process and I/O Control
• available for Network of workstation(NOW)
• Score-D
• support effective gang scheduling for fine grain
  parallelism
• doesnt have enough API for developers

10
KSIX Functionality

• Global process space
• rapid process creation, termination, signals
  delivering across multiple machine transparently
• support 3 type of process
• Normal Normal task
• Restart System restart task when terminate
• Migration Also migrate after restart

11
KSIX Functionality
12
KSIX Functionality

• Naming Services
• Allow dynamic binding of components in the system
• Processes can use logical name to contact each
  other
• Provides APIs that allow the application to
  register, un-register, and search the service
  name for server information.

13
KSIX Functionality

• Naming Services
• The service name will be unregistered
  automatically when directory server detect the
  absent of service server.

14
KSIX Functionality

• Fault Tolerant RPC
• Example use of dynamics binding to provides
  fault tolerance connection between stateless
  server and client..

15
Server
16
KSIX Functionality

• Event Services
• support event delivering between process
• crucial part of high level services

17
KSIX Functionality

• Event Services
• Application can register and un-register as event
  interceptor.
• Event generator can raise an event which will be
  delivered to event interceptor

Raise event
Event Generator
Event Manager
Event Interceptor
register
18
KSIX Functionality

• Ensemble Management
• acknowledge the change in cluster topology
• automatic add/delete node from the ensemble

19
KSIX Implementation

• KSIX consists of 2 software layers
• KSIX Kernel Take care of core service and KSIX
  internal management
• KSIX services Extensible services

20
KSIX Kernel

• Main Components
• KXPM (KSIX Partition Manager)
• keeps track processes status
• maintain global process ID
• KXD (KSIX Daemon)
• create, terminate, send signal to processes
• monitor processes on each node
• map between global process ID and local process
  ID

21
KSIX Kernel

• KXPM and KXD is organized into a tree to
• scalable due to fixed degree of connection

22
KSIX Implementation

• KSIX tree are maintain dynamically by ensemble
  management subsystem
• always maintain KSIX tree as a complete tree
• add node to end of tree
• delete node by request of automatics delete dead
  node
• KSIX tree is also used as a heart beat to detect
  dead node. And multicast communication
  infrastructure

23
KSIX Services

• Provide services to application
• Each service consist of service server and API
  library
• Currently available services are event service
  and directory service
• Can be used by applications on any node in the
  cluster

24
KSIX Implementation
Application
Service Lib
KSIX Service
Service Server
KSIX Kernel
25
KSIX Target Application

• Support the development of task scheduler
• We are working on a task scheduler called SQMS
• Support future implementation of MPI2

MPI (MPI 2)
KSIX
26
Experimental Evaluation

• Experimental System
• PIRUN Beowulf Cluster
• 72 Node Pentium III 500 MHz , 128 MB
• 3 File Server Pentium Xeon 500 MHz , 1 GB
• 100 Mbps Fast Ethernet Switch

27
KSIX Bootstraping

• KSIX is started by a bootstrapping utility called
  ksixboot and stop with a utility called
  ksixhalt
• After KSIX have been loaded. Application can use
  KSIX services by linking to KSIX library

28
KSIX in operation

• KSIX system comes with command line utilities
  that control KSIX
• KSIX Process Monitoring
• monitor status of process
• terminate process

29
Experiment

• Simulate the working of scalable unix command
• Start parallel task that do nothing on 1 to 64
  processors using KSIX call and rsh
• compare the time used by rsh based code and KSIX
  based code
• KSIX is much faster

30
Experimental Evaluation
31
Experimental Evaluation
32
Conclusion

• Our contribution is to build a workable parallel
  environment for Beowulf cluster that help enable
  users to develop much more complex system
  software tools and applications
• KSIX version 1.1 beta2 is now available for
  download at http//smile.cpe.ku.ac.th
• very experimental state

33
Future Works

• Extend the APIs to meet more needs in cluster
  development eg. Simple file services
• Enhancing the scalability
• Adding high-availability support
• implement an extensible service architecture that
  allows developer to plug in new services to KSIX

34
Future Works

• SCE SMILE Cluster Environment
• Powerful set of tools that allows user to easily
  build and use Beowulf Cluster
• Consists of
• Beowulf Diskless Cluster Suite
• SCMS Management tool
• KCAP Web/VRML for SCMS
• SQMS Batch scheduling
• KSIX Cluster Virtual Machine

Cluster Application
SCMS
KCAP
SQMS
KSIX
Beowulf Cluster
35
THE END