Introduction to the GRID

1
Introduction to the GRID

• Satoshi Sekiguchi
• Director, Grid Technology Research Center,
• Advanced Industrial Science and Technology, Japan

2
What is the Grid (1)?

• The World Wide Web provides seamless access to
  information that is stored in many millions of
  different geographical locations
• In contrast, the Grid is a new computing
  infrastructure which provides seamless access to
  computing power and data distributed over the
  globe
• The name Grid is chosen by analogy with the
  electric power grid plug-in to computing power
  without worrying where and how it comes from,
  like a toaster

Slide by courtesy of Fabrizio Gagliardi
3
What is the Grid? (2)

• New applications enabled by the coordinated use
  of geographically distributed resources
• e.g., distributed collaboration, data access and
  analysis, distributed computing
• The Grid relies on advanced software, called
  middleware, which ensures seamless communication
  between different computers and different parts
  of the world
• e.g., certificate authorities and policies,
  protocols for resource discovery/access,
  programming environment
• Original motivation, and support, from high-end
  science and engineering but has wide-ranging
  applicability
• e.g., from Grand Challenge to Everyday Business

Slide courtesy by Ian and Carl
4
Who will use the Grid?

• Computational scientists engineers large scale
  modeling of complex structures
• Experimental scientists storing and analyzing
  large data sets
• Collaborations large scale multi-institutional
  projects
• Training education virtual learning rooms and
  laboratories
• Environmentalists climate monitoring and
  modeling
• Corporations global enterprises and industrial
  partnership

Slide by courtesy of Fabrizio Gagliardi
5
Grid a quick look

• No clear definition, changes day by day

Grid
Meta computing
Computing Grid (high speed computing service)
Virtual Computing Center
Bio, High energy, Astro Large scale data analysis,
Data Grid (File service, DB integration)
Grid
Access Grid, Sensor Grid
Business Grid (Data Center Grid)
Grid Data Center
eGov, Business federation
PC Grid (Mega computing/ Cycle scavenging)
Intra Grid
SETI_at_HOME-like
6
Technology Life Cycle (Grid/Electricity)
Soft PKG Grid XX
Grid software Product
Utility Data Center
Globus, Unicore Ninf, SETI_at_home
Grid Testbed
Today
Business
Pilot user
Technology
Research Lab
Power Plant
Volta's Battery
Lamp
B. Franklin
7
GridWorld2004

• 2004 Apr. 27-28 TFT Hall, Tokyo Japan
• The first national scale event for grids in the
  world
• 3000 registration, 2000 attended, 20 groups
  exhibited

8
GridToday 04

• 2004 May 26-28 Philadelphia CC
• Exhibit for grid technologies in US
• 400 attended, 26 groups exhibited

9
Business with Grids
Business Flow (eCommerce, Business Management,)
Applicatiooon (Stock mgmt, Financial mgmt, CRM, )
System management tool (Work load, DB,
Performance monitoring, Resource management)
Web service system (XML, SOAP, WSDL )
OGSA(Authentication Authorization, Policy,
Resource, )
OS (Linux, Solaris, AIX,, Windows)
H/W(Server, Storage, Network)
10
Economic Impact in IT Market in Japan
by NAREGI report
2099
100 Million JPY 0.9 M USD
2000
1886
Grid service
1774
1800
System - Grid
1547
1600
IDC
1383
1500
1285
1191
1292
1200
1245
1100
1200
1156
1114
1074
1000
System non Grid
1035
987(2002???)
0
Y
2004
2005
2006
2007
2008
2009
2010
807
100 Million JPY
Grid Market Growth
700
?
Grid Business
? Grid System ? Grid Service
600
529
594
500
347
400
391
300
227
?
171
258
200
117
170
65
100
128
88
49
0
Y
2007
2004
2005
2006
2008
2009
2010
11
Adopt Grid Technology to Business Scene

• IT resource Virtualization/ provisioning/Integrati
  on of Web services
• Higher resource utilization usage for saving TCO
• Automate IT resource management in Data centers
• Business Continuity, Business opportunity

Business Scene
Science grid
HPC Application Bio informatics Weather
forecasting, climate simulation Structure
Analysis

• Business Application
• Web Service
• Engineering/Technical

• Adopt Grid technology to business
• Scene
• Higher utilization of IT resources
• Dynamic load balancing

Global load distribution Global IT resource
sharing
12
More complex, higher TCO

• Larger in system scale
• 3-tier model
• Buy more servers for sustain performance
• More complex heterogeneity
• Processor architecture, performance
• OS (Windows, Linux, Unix, )
• Non standard APIs
• DB, AP server, system management

TCO increases significantly
Save TCO by Web service, Grid
13
Autonomous IT Infrastructure

• Dynamic provisioning according to the demand for
  the resources in case for access spike, system
  fail.
• Virtualizes computing heterogeneity

Server on duty
Stand-by
Over load
A
M
Fail
M Monitor A Analysis P Plan E Execution
Autonomous
E
P
14
Service on demand Pay Per Use (PPU)
Access Spike
Overload
Savings
Option Pay per Use
Basic Charge
1
2
3
4
5
6
7
8
9
10
11
12
Month
15
Intra Grid for higher resource utilization
Problem
Solution
Effect
TCO saving more flexibility in business units
Vertically fixed resource allocation less
flexibility in dynamic allocation
Grid virtualizes IT resource to share
Present in-house system
Solution by the business grid
Personnel
Accounting
Production
CRM
CRM
Personnel
Accounting

shared environment
Business Grid M/W
CRM
Production
General affairs
Infra mgmt
Infra mgmt
Infra mgmt
16
Off loading access spike
Problem
Solution
Effect
Additional resource deals with overflow
traffic Guarantee business continuity
unexpected spike -may lose business
opportunity -may lose reliability
Faster and easier deployment, and provisioning
much less SE work
Settlement
server
storage
Maker P
Hot spot
network
Delivery
Online shop A
Parts Maker
Additional resource maintain business continuity
Maker Q
Online shop B
17
Outsourcing IT resources by the grid service
provider
Subject
Solution
Effect
Systems can cooperate between the in-house and
the data center by the business grid.
Demand of curtailing system possession
cost. Resources are required for load / peak
management.
Constructing system using grid middleware.
ltGrid service providergt
The in-house system of a company
Outsourcing
Data Center X
Business A
Resource offer
Business B
Outsourcing
IDC as a backup center in case of
calamity Disaster recovery
18
Realization of optimizing shared IT resources in
a group company
Effect
Solution
Subject
TCO curtailment Common pool improve utilization
ratio
Grid enables common IT resource pool to shared
back up
Duplicated applications and IT resources Many
unused resources for every system
Web/AP servers (Business system bases)
DB servers
Load increases
Integrated DB bases
Design System
A Corp
Network bases
trouble
Data protection bases
Accounting system
B Corp
A CorpDB
B CorpDB
C CorpDB
ResourcePool
Personnel system
C Corp
Automatic extension / self-restoration
AA Business System
C Corp
Monitor
BB Analysis System
C Corp
AP servers (Parts functional bases)
Pooled resources?A Corp
Automatic extension of pooled resources
AA ComputingParts group
BB BusinessParts group
Restores automatically to pool resources
Pooled resources?B Corp
Adding active resourcesAutomatic inclusion(No
need for stopping other systems)
Pooled resources
CC Business Parts group
OtherParts group
Pooled resources (Additional)
Monitor
Monitor
Integrated Operation Management Bases
19
Karaoke Grid _at_ SC2003
20
National and InternationalGrid Activities
21
Significant Grid Activities in Asia-Pac
1USD110JPY1,100KrW33.8TWD41.3THB1.69SGD
22
ApGrid Asia Pacific Partnership for Grid
Computing

• Open community for Grid researchers in Asia
  Pacific
• ApGrid
• A meeting point for all Grid researchers in
  Asia-Pacific
• A communication channel to the GGF, and other
  grid communities (e.g. TeraGrid, UK-eScience,
  EUGrid, etc.)
• A computing testbed for making real
  grid-collaboration

23
ApGrid Testbed unique features

• Truly (naturally) multi national/political/institu
  tional VO beyond boundaries
• Not an application-dedicated testbed general
  platform
• Diversity of languages, culture, policy,
  interests,
• Grid BYO Grass roots approach
• Each institution contributes his resources for
  sharing
• Not a single source funded for the development
• Physical resources
• Most contributed resources are small-scale
  clusters
• Networking is there, however the bandwidth is not
  enough
• Achievements
• have experiences on running international VO
• verify the feasibility of this approach for the
  testbed development

24

• Non-profit international consortium established
  on 3 June 1997.
• Being a high-performance network for RD on
  advanced next generation applications and
  services.
• Providing an advanced networking environment for
  the research and education community in
  Asia-Pacific.
• Promoting global collaboration.

• Tightly collaborating with TransPAC.
• Having meetings twice a year.

25
Pacific Rim Application and Grid Middleware
Assembly

• NSF-funded project lead by UCSD/SDSC.
• 1st workshop was held in March 2002.
• 7th one Sep.15-17 04
• Establish sustained collaborations and advance
  the use of the Grid technologies for
  applications.
• Expected outcomes
• Advance scientific applications
• Increase productive and effective use of the grid
  by researchers and scientists in the Pacific Rim
• Increase interoperability of grid middleware in
  Pacific Rim and throughout the world

• Tightly collaborating with ApGrid.
• Having workshops 2 times a year.

26
Global Grid Forum
Korea Sangsan Lee (GFAC) Japan Sekiguchi
(GFAC), Matsuoka (GFSG), Itoh (GMAC), and
co-chairs, etc
OASIS
GGF must collaborate with (leverage) a variety
of standards organizations working on
underlying and related technologies.
DMTF
W3C
IETF
WS/I
27
GGF Meeting Participation
13th in Seoul March,05
12th in Brussels Sep,19-22 04
Tokyo March,03
GGF3 shows impact of 9-11
28
Roles of the APEC/TEL Grid Workshop
29
Expected role of APEC/TEL

• Government role includes
• Encourage basic studies
• Remove potential obstacles in advance
• Establish Intl cooperation framework to speak to
  other communities, - US, EU, etc
• NO commitment for each business or project
• Business and Science Grid are inseparable from
  each other
• An ultimate challenge in science grid is always a
  trailblazer for expanding the business field.
• eScience, Large science, Affordable high speed
  network
• Having consensus of shared policy is a MUST for
  promoting future international grid business
  opportunity cooperatively,.
• security, CA ops, SLA, balance, etc

30
Grid Challenges
Business and ENTERPRISE

• Cope with different management policies of
  different centers
• Ensure data security more is at stake than just
  money!
• Guarantee higher reliability for business
  continuity prepare for disaster recovery
• Agility and resiliency for every business
  opportunity

SCIENCE and TECHNOLOGY

• Grow rapidly, yet remain reliable for more than a
  decade
• Share data between thousands of scientists with
  multiple interests
• Link major computer centres, not just PCs
• Ensure all data accessible anywhere, anytime

31
Grid enabled Engineering
Grid enabled Business
Medical Informatics
eBiz, service
Nano-tech Informatics
Bio-Informatics
Solution
Utility
HPC
Chem Infoma
Webservice
Community
Portal/ASP
Personal Wether service
Access Grid
Grid Technology
Community Infra
Science and Engineering platform
Upper Middle
Grid Operation Ctr
Ubiquitous Grid
Peta Scale Grid
Database/Expression
Lower MIddle
sensor
Computing 1PFLOPS PC x 100M SC x 10000
Distributed Storage 10PB
GPS
Earth Q
Wireless network Tremendous Sensor Huge datastream
P2P grid
Mobile
Cam
Server tech
IPv6
Security
Blood pressure
FTTH/xDSL/Wireless AP/100M-
32
Grid Architectures and AIST contributions
Social Framework
Education/training/workshop
Drug Design
Bio Infoma
HEP DG
Applications and Portals
Agriculture
ActiveSheet, BioGrid, Drug Design, HEPGrid,
ThaiGrid Portal, CFD, Chemical Eng.,
Environmental and Ecology Preservation,
QC Portal
High Level Grid Middleware
Programming Tools Ninf-G, Omuni-RPC, Nimrod DPL,
MetaCompiler Simulators Bricks and GridSim, Data
farm Resource Brokers Nimrod-G
Grid RPC/Ninf-G Grid Data Farm
tools
Low Level Grid Middleware
Security, Process, Storage, QoS, etc Globus,
Jxta, CPM, GrACE, OGSA
Grid Fabric
Clusters, MPPs,VP, SMPs
AP partners have Computers PC, WS, Clusters,
Supercomputers SCE manager WAN Internet, APAN,
GrangeNet, APII, Tsukuba WAN Instruments KEK
Accelerators,
1 USD 129 JPY
33
Summery of my talk

• Grid technology has been significantly emerging
  in science scene.
• More demand in business scene.
• Grid expands the IT market size and other areas.
  
• Encourage Grid activities in the region is a MUST
  !
• Lets talk about it for today

34
Back ups