Technologies for Grids and eBusiness GridComputing Introduction 23'10'07

1
Technologies for Grids and eBusinessGrid-Comput
ing Introduction23.10.07

• Dr. Ramin YahyapourComputer Engineering
  InstituteUniversity Dortmund

2
Overview

• What is the Grid?
• Application Examples
• Architecture of Grids
• Current Development
• Similarities to eBusiness Solutions and
  Ubiquitous Computing

3
Technological Trend - Computational Power

• Computational power follows Moores Lawdouble
  of transistor density every 18 months.

• Do we need this computational power?

4
Applications

• Several relevant problems are still not
  adequately solved.
• molecular, atomic simulation (chemistry, physics)
• pharmaceutics
• climate research
• artificial intelligence
• Grand Challenge Problems

5
Technological Trend - Networking

• Network performance doubles every 9 months.

• the difference will yield several dimensions in a
  few years!

Source Scientific America 2001
6
Consequence

• When the network is as fast as the computer's
  internal links, the machine disintegrates across
  the net into a set of special purpose appliances
  (George Gilder)

• Parallel programming
• Distributed applications
• Decentral problem solving

7
The Grid

• Resource sharing coordinated problem solving
  in dynamic, multi-institutional virtual
  organizations (Ian Foster)

Source Globus, Ian Foster
8
Grid Vision

• Simple, transparent access to resources without
  central control
• dynamic coordination and combination of services
  on demand
• easy additionof resources
• autonomic management of Grid components
• complexity of the infrastructure is hidden from
  user or resource provider.
• Analogy Power-Grid
• Transformation to e-Science
• Support of Virtual Organizations

9
Computational Grids

• Connection of
• High-performance computers, parallel computers
• Workstations-/PC-Cluster
• in the future single PC systems
• The Grid automatically dertermins on request,
  where, when, which computers are available for a
  task.
• Idea orginitates from metacomputing (90)

10
Trend of Online-Data

• The storage density doubles every 12 months.
• Example online-data
• 2000 0.5 Petabyte
• 2005 10 Petabytes
• 2010 100 Petabytes
• Not only computational performance but also data
  management is a key challenge in the future!

11
Data-intensive Applications

• Medical Data
• Digital X-Rays
• Brains-Scans
• Petabytes of Data

12
Data-intensive Applications

• Medical Data
• Digital X-Rays
• Brains-Scans
• Petabytes of Data
• Molecular Data
• Genom
• Proteine
• Drug analysis
• Environmental Data
• Weather-/Climate
• Geophysics
• Astronomy
• Physics
• High-Energy Physics
• Astronomy

typical time-variant, 3D models require large
volumes of data (simulated, recorded)
13
Example CERNs Large Hadron Collider

• 1800 Researchers, 150 Institutes in 32 Countries
• work on 100 PetaByte of data until 2005 with
  several 10.000s of processors!

14
Example CERN-LHC Data-Grid
PBytes/sec
100 MBytes/sec
Offline Processor Farm 20 TIPS
100 MBytes/sec
Tier 0
CERN Computer Centre
622 Mbits/sec
Tier 1
FermiLab 4 TIPS
France Regional Centre
Italy Regional Centre
Germany Regional Centre
622 Mbits/sec
Tier 2
622 Mbits/sec
Institute 0.25TIPS
Institute
Institute
Institute
Physics data cache
1 MBytes/sec
Tier 4
Physicist workstations
Source Ian Foster, DataGrid, CERN
15
Grid-Ressources

• Computer und data management ultimately require
  alsonetwork management
• Considerations of network connections
• Reservation of network properties
• Quality-of-service features
• e.g. GMPLS/MPLS
• other resources
• Visualization (3D-Cave, Video-Conferencing),
• Experimental devices and instruments
• Software (Licences)
• Services
• etc.

16
Hype or Reality?
17
Grid-Initiatives

• Several Grid projects in research and academics
• Globus Toolkit
• Open-Source solution
• defacto standard for several protocols and
  services
• Global Grid Forum
• Forum similar to IETF
• Definition and standardisation of Grid protocols
  and services
• Commercial support
• IBM, Microsoft, Sun, Compaq,
• Platform, Avaki, Datasynapse, ...
• Combination of technological interests from
  eScience and eBusiness

18
Intelligent Infrastructures
19
Architecture of a Grid

• Situation
• different types of components,
• different rules, policies, provider
• Requirement of standardisation
• description of and
• access to resources
• independent, distributed, and scalable services
• shared protocols

20
Grid Functions (1)

• Security
• Authentication
• Privacy
• Information Services
• Information about existing resources or services
• Lookup and discovery of functionalities
• Resource Management
• Integration of resources

21
Grid Functions (2)

• Data Management
• Data transferring
• Data localisation
• Replication
• Synchronisation
• Scheduling
• automatic selection of resources
• coordination of resource allocation
• Accounting
• Billing

22
Future Grid Applications

• Use of Grid-APIs for portable access to Grid
  functionalities
• Example
• dynamic resource discovery
• data transfers
• requesting network services
• higher abstraction level in software development

23
Multi-level Infrastructure
Source Ian Foster
24
Open Grid Service Architecture

• OGSA is currently standardised in GGF.
• Goal is the simple integration of new services
  into the Grid.
• service-oriented approach
• Communication via
• WebServices, SOAP
• Similarity with eBusiness solutions
• Similar protocols
• Application Server

25
Globus Toolkit

• Open-Source software package with basic services
• usually the reference implementation of services
  that are standardized by the Global Grid Forum.
• Primarily, just a bag of services to implement
  Grid projects
• Security
• Information Services
• Resource management
• Data transferring
• no directly usable/installable product

26
Example for Application Software Cactus

• modular framework for creating parallel
  applications for multi-dimensional simulations
• small kernel with management functions
• modules (Thorns) with single simulation functions
  (visualization, numerical methods etc.)
• Used in the area of astrophysics
• but also usable for other projects.

Thorns
Cactus flesh
27
Example of a Grid Job

• Required Resources
• needs 48 processing nodes of a specified
  architecture/properties for 6 hours
• a nearby visualization device is requested during
  execution
• Allocation Time Requirements
• should be executed between 8am and 6pm the
  following day
• Data Requirements
• needs a specified data set
• Storage Requirements
• needs 1 GB of storage during execution
• Software Requirements
• utilizies a specified licensed software package
• Network Requirements
• a network connection with a given bandwidth
  between the VR deviceand the application is
  needed
• Cost Requirements
• The user is willing to pay at most 4 Euro
• Objectives
• He prefers cheaper job execution over an earlier
  execution.

28
Automatic Allocation and Reservation

• Goal
• Automatic planning of the resource allocation

time
Network 1
Prefetching Input-Data
Storing Output-Data
Computation
Reservation
Computer 1
Computation
Computer 2
Network 2
Communication
Visualization Cave
Visualization
Software License
Software Usage
Storage
Storage
29
Future Grid Scenario
WAN Transfer
Remote CenterReads and Generates TB of Data
Visualization
LAN/WAN Transfer

• In the near future the Grid offers users
• online access to
• Petabyte of data storage,
• Teraflops of computational power,
• on-demand access to QoS-network connections
• on-demand software,
• automatic access to arbitrary services.

Difference to Internet, anticipated potential !
30
eBusiness Applications

• Internet Business services B2C
• eCommerce-Solutions
• Portals
• Internet Business services B2B
• Inter-operation between companies
• Supply-Chain Management
• Intranet Solutions
• Enterprise Application Integration EAI

31
Trend in eBusiness Solutions

• Previously
• use of the internet as add-on technology to
  existing business models
• often stand-alone solutions for specific prupose
• Current and future
• integration of different services (in-house,
  between companies) to improve efficiency
• orientation of internal and external business
  processes and workflows
• leveraging new business models

32
Technological Challenges

• Similar to Grid standards
• Standardization of services and protocols
• similar technologies
• XML-Processing
• Web-Services, SOAP
• Enterprise Java Beans
• Requirements
• security
• reliability
• performance
• scalability
• interoperability

33
Pervasive/Ubiquitous ComputingMobile Solutions

• Easy and transparent integration of different
  mobile components
• Requirements
• security
• reliability
• performance
• scalability
• interoperability
• Same Requirements similar solutions
• but often additional constraints
• devices may be subject tighter limitations in
  size, cost, performance

34
Course Content

• Current Technologies for building dynamic and
  distributed systems
• Focus on software aspects
• Web Technology Stack
• Hands-on experience of practically usable
  software skills
• exercise and practical sessions