Computational Grid Using Commodity Systems

1
Computational Grid Using Commodity Systems

• Vijay Potnis
• 03/31/03

2
The Concept

• Grid -A pool of computing power created using
  multiple computers
• Goal is to solve a problem that requires enormous
  amount of computing power
• User submits his work to this pool and gets
  computed results
• Data Grid - Grid environments tailored to the
  needs of the processing of huge data samples
• Computational Grid Grid environment tailored to
  the needs of intensive computation

3
Grid Architecture

• How to Build a High-Performance Compute Cluster
  for the Grid
• Alexander Reinefeld and Volk ErLindenstruth
• Framework for high performance cluster that can
  be used as a reliable computing node in the grid
• Goals
• Low total cost of ownership, High Degree of
  scalability , Maximum performance for parallel
  applications
• Experiment at European research institution CERN
• Large Hadron Collider high energy physics
  collider experiment
• Huge data generated is reduced by Data selection,
  filtering Compression using this grid
• Simple compute farm architecture fails for
  analysis under quasi real-time condition

4
Grid Architecture

• Components
• Computing nodes Intel 1.7 Ghz P4, AMD Athlon
  1.3 Ghz, Alpha 212644 833 Mhz, with either SDRAM
  or DDRAM, OS Linux, PCI(X) bus
• Interconnect Protocol To reduce cost Gigabit
  Ethernet was chosen. Used Scheduled Transfer
  Protocol within the cluster and TCP for
  communication with outside world
• Fault Tolerance Sensor and actuator function
  are implemented. Micro-controller based PCI
  devices are installed which operated through a
  private control network. Repair server take care
  of other complex failures such as race
  conditions.
• Integration Local cluster management system is
  used to integrate these individual pieces into a
  single computational node

5
Scheduling

• Scheduling in a Grid computing environment using
  Genetic Algorithm
• VincenZo Di Martino and Marco Mililoti
• Purpose - Investigate the possibility to use the
  computing GRID in a flexible way to permit the
  maximum usage of resources
• Approach - Central repository of resources and
  resources requests
• Two level system of scheduling
• First formed by the set of computing nodes each
  one with a local scheduling policy. Local
  scheduler accepts a single job at a time and
  allocate it on the local hardware with respect to
  the local information
• The second level formed by the superscheduler. A
  superscheduler (metascheduler) is a centralized
  scheduler which optimize the allocation of a job
  allowing the execution on the fittest set of
  resources
• Developed a simulator to study the usefulness of
  genetic algorithm for this scheduler. Simulator
  is written in C using Parallel Genetic Algorithm
  Package (PGAPack) library.

6
Scheduling

• Given a set of jobs and computing node,
  superscheduler find the allocation sequence on
  each node of a computational grid that minimize
  the release time of jobs.
• Solution is represented using chromosome encoding
  a schedule. For n jobs the chromosome will hold
  2 n genes with couples of two genes representing
  the job and the node on which it will be
  scheduled
• Decoding is done by scanning the chromosome and
  scheduling each job on the corresponding node
  with respect to the local scheduling policy.
• Results for simple configuration and simple
  problem the system was able to allocate resource
  optimally but in complex configuration the
  optimum was not achieved.

7
Developing Applications for a grid

• The GrADS Project Software Support for
  High-level Grid Application Development
• Francine Berman, Andrew A. Chien, Keith Cooper,
  Jack Dongarra, Ian Foster, Dennis Gannon, Lennart
  Johnsson, Ken Kennedy, Carl Kesselman, John
  Mellor-Crummey, Dan Reed, Linda TorcZon, and Rich
  Wolsky
• Goal To ease process of applications
  development for grid
• Focus
• Grid software architectures
• Base software technologies, such as scheduling,
  resource discovery, and communication
• Programming models, languages, compilers,
  environments, and other tools
• Mathematical and data structure libraries,
  including numerical methods for adaptivity,
  control of accuracy, and latency tolerance

8
Developing Application for Grid

• Result - GrADSoft an application framework
• It provides an environment where an unmodified
  parallel application can be scheduled,
  instrumented, executed, and monitored in a Grid
  environment with no user intervention.

9
Grid Management

• A System for Monitoring and Management of
  Computational Grids
• Warren Smith
• Describes a framework for a System to monitor
  and manage computational grids
• Work done at NASA on Informational Power Grid
  which is based on Globus toolkit.
• Framework - Control and Observation is
  Distributed Environment (CODE)
• User creates his own sensors and actuators and
  add to the base framework, giving more
  flexibility to the user in managing any component
  on a Grid
• This framework is embedded in Globus Resource
  Allocation Manager (GRAM) management agent

10
Grid management
11
Security

• A Security Architecture for Computational Grids
• Ian Foster, Carl Kesselman, Gene Tsudik and
  Steven Tuecke
• Problem - Security concerns when using widespread
  computational resources
• This paper analyses security requirements of a
  large-scale distributed computing and develops
  security policy and a corresponding security
  architecture.
• Following are security requirements on a
  computational grids
• Single sign-on
• Protection of credentials
• Interoperability with local security solutions
• Exportability
• Uniform credentials/certification infrastructure
• Support for secure group communication
• Support for multiple implementations

12
Security

• Security policies focuses on authentication of
  users, resources, and processes and supports
  user-to-resource, resource-to-user,
  process-to-resource and process-to-process
  authentication
• Architecture to Enforce these

13
Grids using commodity systems

• SETI_at_home
• SETI team at UC Berkeley discovered thousands of
  computers sitting idle.
• Most of them having screen savers and wasting
  precious computing resource
• This is where SETI_at_home (and you!) come into the
  picture.
• The SETI_at_home project uses these free cycles to
  search out new life and new civilizations."
• On the users acceptance an agent (screen saver)
  is downloaded.
• The screen saver gets a chunk of data over the
  internet, analyze that data, and then report the
  results back to SETI team.
• But this is not about using free cycles only

14
Future Work

• Condor Grid Computing from Mobile Handheld
  Devices
• Francisco J. Gonzalez-Castano, Javier
  Vales-Alonso and Miron Livny
• Title is misleading
• Goal - Provide user interface to access Condor
  and not provide computing power using Mobile
  Handheld Devices.
• But in future you may see SETI running on a PDA
  or Cell phones