Scalable Cluster Software Tools

1
Scalable Cluster Software Tools

• Putchong Uthayopas
• Kasetsart University

2
Scenario

• Beowulf cluster is now a very popular platform
• Excellent price/performance
• Scalable performance to teraflops on hundreds of
  nodes
• Good technology tracking
• large selection of open source software
• the only hope for developing country to do HPC
• What is wrong with this picture?

3
Issues

• Is is still very painful to build and operate
  medium and large cluster
• Large collection of software that are not talk
  to each other
• Cluster is still difficult to use
• Writing program on cluster is still quite a
  painful experience
• very primitive environment
• How to use cluster power efficiently
• It is difficult to keep the cluster reliable for
  long term execution
• Hardware
• Software problem

4
Solutions

• Building a powerful scalable software tools
• Management environment
• User Environment
• Problem solving environment
• Integration and standardization
• Interoperable components
• based on standard such as LDAP, XML
• Related Attempt
• OSCAR, ROCK, SCYLD and SCE

5

• Goal
• Creating a truly integrated and scalable cluster
  environment for cluster computing community
• Infrastructure Management Tools
• Administration and Monitoring Tools
• Resource Management and Batch Scheduling Tools
• User and Problem Solving Environment
• User can quickly and automatically build a
  cluster up to the point they can run MPI task

6
SCE ideas

• Simplicity and Completeness of the environment
• No kernel modification
• Create user level global environment, most thing
  can be done in user level
• Maintain full software compatibility
• Rich set of tools
• Command line, GUI, Web
• Rich set of API, extensible

7
SCE Ideas

• Infrastructure builder is used to Create and
  maintain
• cluster configuration and database (in XML
  format)
• Cluster software configuration that common to all
  components

8
SCE Ideas
Application and Tools
MPI
Instrumentation
Process Control
Fault Detection
Event Services
Naming Service
Cluster Hardware
9
(No Transcript)
10
SCE and GRID
GRID
SCE Cluster
SCE Cluster
SCE Cluster
SCE Cluster
SCE Cluster
Grid Middleware
SCE
SCE
Node
Node
Node
Node
Node
Node
11
Future Direction

• The need for standard
• Data format(XML) , interaction and communication
  among tools
• Easier and more integrated user environment
• How to help user debug program easier, run
  program easier, visualize their result easier
• Migration of PC/Windows/MAC technology and ideas