Grid Activities in China

1
Grid Activities in China

• Yongzheng Ma
• CNIC, CAS
• myz_at_cnic.cn
• Aug. 26, 2004, GFK, Seoul

2
Outline

• A Brief Introduction to Principal Grid Projects
  in China
• China National Grid (CNGrid)
• China Grid
• NSFC Grid
• Scientific Data Grid (SDG)
• Shanghai Grid
• Vega Grid
• Total government funding during 2001 2005
  180 million
• e-Science and Scientific Data Grid
• e-Science in CAS
• Scientific Data Grid
• Conclusion

3
A Brief Introduction to Principal Grid Projects
in China
4
China National Grid (CNGrid)

• A key project supported by the Ministry of
  Science and Technology
• 4-year effort
• Total funding 35 million
• 12 million fund from the Chinese government
  (MOST)
• 23 million fund from local governments,
  application organizations and industry

Courtesy to Professor Depei Qian
5
China National Grid

• Enabling resource sharing and cooperative works
  under the Internet environment
• Providing the most powerful computing capability
  to research
• Exploring new models and patterns of applications
  and services in the Internet
• Promoting Chinese industry in high performance
  computers and grids
• Training of talent people in the field of the grid

Courtesy to Professor Depei Qian
6
Three levels of activities
Research
Environment
Manufacturing
Service
Appl. Grids
Development Environment
User Environment
Grid Software
Grid System Software
HPC
Database
Data
Apps
Grid Resources
Internet
7
Grid Test-bed

• Eight nodes across the country
• Two major nodes
• CNIC (Beijing)
• Shanghai Supercomputer Center (Shanghai)
• Ordinary nodes
• Tsinghua University (Beijing)
• Applied Physics Institute (Beijing)
• University of Science and Technology of China
  (Hefei, Anhui)
• Xian Jiaotong University (Xian, Shaanxi)
• National University of Defense Technology
  (Changsha)
• Hong Kong University (Hongkong)

Courtesy to Professor Depei Qian
8
Major tasks of this project

• Develop a test-bed for the new information
  infrastructureChina National Grid (CNGrid)
• Develop a 4TFlops high performance computer to
  equip the grid nodes
• Develop a kind of grid software to support grid
  applications, grid operation and management
• Develop several application grids
• Productive systems
• Demonstrate the future grid applications

Courtesy to Professor Depei Qian
9
Requirement to grid nodes

• Major nodes
• gt1 Tflops
• Equipped with the machine developed by the
  project
• Ordinary nodes
• gt100GFlops

Courtesy to Professor Depei Qian
10
(No Transcript)
11
HPC development

• Two gt4TFlops machines have been developed
• DeepComp 6800
• Intel Itanium 2 processor
• 5.324 Tflops peak performance
• Installed at CNIC, CAS
• Dawning 4000A
• AMD 64-bit Opteron processor
• 11 Tflops peak performance
• Installed at the Shanghai Supercomputer Center

Courtesy to Professor Depei Qian
12
Grid software development

• Grid system software
• Interfacing to heterogeneous systems
• Basic resource management scheduling
• Communication
• Grid application development environment
• Remote debugging
• Performance tuning
• Grid user environment
• Grid browser
• GSML (Grid Service Markup Language)

Courtesy to Professor Depei Qian
13
Research on grid software technology

• Grid software architecture
• Device layer, bus layer, operation environment
• Grid address space model
• Physical address, virtual address, effective
  address
• Mapping between different addresses
• Grid job management
• Virtual address based service invoke
• Job schedule, load balance, fault-tolerance
• Grid information management
• Distributed grid information management
• Grid resource management
• Dynamic resource deployment

Courtesy to Professor Depei Qian
14
Application grids development

• Selected areas
• Resource and Environment
• Research
• Services
• Manufacturing

Courtesy to Professor Depei Qian
15
Application Grids
16
Geological Survey Grid

• Undertaken by China National Geologic Survey
  Bureau
• Nation-wide productive grid
• Data exchange and sharing
• Resource evaluation services
• Shorten the evaluation time from 3 years to 3
  months
• Delivering achievements in
• North China ground water resources evaluation
• Mineral resources evaluation

Courtesy to Professor Depei Qian
17
Meteorological Service Grid

• Undertaken by China National Meteorological
  Center
• Three sites in Beijing, Guangdong, and Hunan,
  forming a grid
• Sharing of computing resources
• Platform for digital weather forecasting pattern
  research
• Provide new weather forecast services (time and
  location-specific)

Courtesy to Professor Depei Qian
18
Bio-informatics Grid

• Undertaken by Genomics Bioinformatics
  Institute, CAS
• Computing grid, data grid, and information grid
  for bio-information research in the country

Courtesy to Professor Depei Qian
19
Manufacturing Grid

• Undertaken by China Aviation industry Corporation
  II
• Support distributed design, simulation, and
  manufacturing
• CAD software sharing
• Data exchange and sharing
• Computing resource sharing

Courtesy to Professor Depei Qian
20
China Grid

• China Education and Research Grid
• About 3.5 million, Funded by the Ministry of
  Education
• Based on CERNET (China Education and Research
  Network)
• First Phase
• From 2003-2005
• 12 key universities as initiative
• More than 6Tflops w/60TB
• 20 key universities by the end of 2004

Courtesy to Professor Hai Jin
21
Northeastern Univ
Tsinghua Univ/Peking Univ/Beihang Univ
Shandong Univ
Xian Jiaotong Univ
Southeast Univ
Huazhong Univ of Science and Technology
Shanghai Jiaotong Univ
National Univ of Defense Technology
South China Univ of Technology/ Sun YAT-SEN Univ
22
Main Research Tasks

• Campus grid platform
• Common platform for China Grid
• Grid application platform
• Representative grid applications
• Image processing grid
• Bioinformatics grid
• Course on-line grid
• Computational fluid dynamic grid
• Large scale information processing grid

Courtesy to Professor Hai Jin
23
NSFC Grid in a Nutshell

• Sponsored by the National Natural Science
  Foundation of China (NSFC)
• ?30 million (3.5 million)
• Goal
• research into key technologies of grid and
  e-Science
• fundamental technologies

24
Scientific Data Grid (SDG)

• Sponsored by Chinese Academy of Sciences
• Based on Scientific Database
• SDG Middleware would have been developed
• Several Application has been or will be
  established
• Virtual Observatory
• Will be discussed in detail in the second part

25
Shanghai Grid

• Sponsored by the Shanghai government
• About 7.5 million fund
• 2003-2005
• Characteristics
• The first city grid
• An information grid

26
Vega Grid Project Overview

• Developed by the Institute of Computing
  Technology, CAS
• Vega Service Grid
• Versatile Services for applications
• Providing minimum common services
• Enabling Intelligence for reducing cost
• Automatic, self-aware, dynamic, interactive
• Providing minimum common supports
• Global Uniformity for ease of use
• Connectivity, single system view,
  interoperability
• Autonomous Control for management
• Autonomous, user-centric, open architecture

Courtesy to Professor Zhiwei Xu
27
e-Science and Scientific Data Grid
28
e-Science and Scientific Data Grid

• e-Science in CAS
• Scientific Data Grid - An exploration towards
  e-Science

29
Whats e-Science? Why e-Science?

• e-Science
• Informatization of Research Activities
• Challenges in modern research
• problems are more complex than ever
• research object is not isolated, but
  cross-discipline and large-scale
• data processing, simulation and computing become
  indispensable methods
• more and more communication and collaboration
  among scientists

30
The Background of e-Science for CAS

• CAS launched Knowledge Innovation Program in
  1998, its time NOW to push it forward in all
  aspects.
• Scientists demand a higher level Informatization
  to meet their requirements in research
  activities.
• CAS started the Informatization Program in the
  10th Five-year Plan (2001-2005)
• Informatization will make great effects on
  promotion of technology innovation and knowledge
  innovation.

31
Informatization of Research Activities

• Bridge the gaps of time, space and environment,
  enable global, cross-discipline, large-scale
  collaboration among scientists
• Change the way how scientists do research,
  greatly improve communication and collaboration,
  advance the development of science and technology
• Informatization of Research Activities is the
  pioneer of Informatization of the whole society

32
Features of e-Science

• Open
• Resource sharing
• Supercomputer, Data, Instruments,
• Coordinated research
• working with a colleague across an ocean as if
  they were within a same lab
• cross-discipline, complex, coordinated
  problem-solving

33
Infrastructure for e-Science

• Computing resources
• Data resources
• Software resources
• Communication resources
• Human resources
• Scientific Instruments
• particle accelerators, telescopes, sensors,

34
e-Science and Application

• e-Science provides an informatized environment
  and platform for research
• Individual applications for fields and areas
  should be developed case by case
• Application is key

35
Milestones of e-Science in CAS

• In 2000, proposed Informatized Research
  Environment in the SDB project
• In March 2001, proposed Scientific Data Grid
• In August 2001, the project funded by the CAS
  Informatization Program
• In December 2001, proposed China Science Grid
• In October 2002, Scientific Data Grid joined
  the China National Grid and became a key component

36
e-Science Activities in CAS (2001-2005)

• Upgrading IT Infrastructure
• Constructing Scientific Research Environment
• Developing Key IT Technologies
• Demonstrating Science Applications

37
Upgrading IT Infrastructure

• Networks
• CSTNET
• Domestic links 155M-2.5G
• International links 310M
• CNGI (China Next Generation Internet)
• Supported by National Development and Reform
  Commission
• 12 GigaPoPs, 2.5-10G links will build by CAS
• Scientific Database
• 10TB
• Supercomputing Environment
• 5 TFLOPS
• Mass Storage System
• 100TB
• Visualization Environment
• SGI Oynx3000

Lenovo 6800, Installed at CNIC
38
DeepComp 6800

• Developed by the Lenovo Group Corp, China
• Completed in Nov. 2003
• Installed at CNIC, CAS in December, 2003
• 2.6TB memory
• 81TB disks
• 4.183TFLOPS Linpack performance (78.5
  efficiency)
• Ranked at 14th in the Top500 list (in Nov, 2003)

39
Lenovo DeepComp 6800
40
Constructing Scientific Research Environments
based on the Internet

• Network of Field Observatories
• Ecology network
• Astronomical Observatories
• Weather stations
• Mountain disaster stations
• Network of Digital Libraries of Specimen
• 24 (zoology, botany, fossil, mineral, ), 80 of
  the whole country
• Digital Library of Specimen is starting
• Network of Digital Libraries
• National Science Technology Digital Library
• Network of Scientific Instruments
• LAMOST, BEP-II, Electron Microscopes,

41
Key IT Technologies

• NGI Technology
• IPv6/IPv4 Transition
• Network Measurement
• IPv6 Root DNS
• Multicast
• Hierarchy Network
• Security
• Resource Location Addressing

• Grid Computing
• Data Grid Middleware
• Data Integration
• Grid Information Service
• Grid Security
• Metadata
• Grid-enabled application

42
Grid-enabled Applications

• Virtual Observatory
• Digital Earth
• HEP Data Grid
• Bio Grid
• Chemical Integrated Information System

43
China Science Grid

• By 2005, Scientific Data Grid will have been
  built. Sharing of scientific data resources and
  collaboration based on it are achieved.
• Then, computing resources and scientific
  instruments will be integrated into. China
  Science Grid will be built on the SDG.
• Also, develop grid-enabled applications and
  establish application grids bio grid, astro
  grid, etc.
• China Science Grid an instance of e-Science

44
International Collaboration

• PRAGMA, 2002
• GLORIAD, Jan, 2004
• NCSA(US), Kurchatov Institute(RU)
• KISTI (Korea)
• Internet2
• APAN

45
GLOBAL RING NETWORK FOR ADVANCED APPLICATIONS
DEVELOPMENTRussia-China-USA Science Education
Network
46
e-Science Planning in Future

• Starting to plan the 11th Five-year
  Informatization Program (2006-2010)
• Focus on e-Science in CAS
• Work with CNGI (China Next Generation Internet)
• International Collaboration
• GLORIAD
• PRAGMA
• APAN
• Potential Killer Science Applications
• Virtual Observatory
• High Energy Physics
• Bioinformatics

47
Scientific Data Grid (SDG)

• An exploration towards e-Science
• Undertaken by CAS
• Background
• Current Status
• Resource constructing
• Middleware developing
• Experimental applications

48
Background

• Scientific Data Grid (SDG) is built upon the mass
  scientific data resources of the Scientific
  Database (SDB).
• SDB is a long-term project since 1983, in which
  there are multi-disciplinary scientific data
  accumulated through the course of science
  activities in CAS.
• The vision of SDG is to take valuable data
  resources into full play by benefiting from
  advanced information technologies, in particular,
  the Grid technology.

49
Data Resources

• Scientific Database (SDB)
• 45 institutions across 16 cities
• 313 databases
• 10TB total volume
• Cover a lot of disciplines
• Chemistry, Biology, Geosciences, Environment,
  Astronomy, High energy physics,

50
SDG Platform

• Data Center
• Part nodes of DeepComp 6800
• 20TB SAN Storage
• TFLOPS-scale computing

51
SDG Software Modules
52
SDG Middleware and ToolKits

• Grid Middleware
• Grid Information System
• SDG Uniform Access Interface
• SDG Security System
• SDG Toolkits

SDG GIS V1.0 Universal Metadata Tool
V2.0 Statistics Tool V1.1
53
SDG GIS V1.0
SDG Applications

• Backend MDS/LDAP
• Two types of Information
• System info
• Metadata
• Management and Service
• Centralized
• Distributed

Query GRIP GRRP MDR
P
SDG GIIS
MDW
SDG Sub-GIIS
MDIS
DCIS
I-MDIS
C-MDIS
MDIS
DCIS
C-MDIS
I-MDIS
54
SDG Universal Metadata Tool

• metadata is tree-like and more flexible than
  fix-column tables, difficult to deal with on web
  UI
• use xml files to store interim results

55
Universal Metadata Management Tool
56
Statistics Analysis Tool (SAT) for Data Volume

• Features
• Win2000/XP, Linux
• Java 1.4
• Globus Toolkit 3 Core
• Oracle, SQL Server, File System
• Deploy
• Data nodes 45 institutes at CAS, across 16
  cities in China
• Mediator CNIC
• Service Monitor

57
Windows 2k/xpJava 1.4GT3 Core
Statistics Services
58
(No Transcript)
59
SDG Middleware and ToolKits

• SDG Middleware
• Grid Information System
• SDG Uniform Access Interface
• SDG Security System
• SDG Toolkits

Data Access Subsystem 1.0
60
SDG Data Access Service Framework
Application Clients
Grid Level Services
Internet
Information Service
Internet

Oracle
mySQL
Member Institutes
Member Institutes
DB2
SQLServer
Node Level Services Data Resources
Foxpro
FileSystem


61
Data Access
62
SDG Middleware and ToolKits

• SDG Middleware
• Grid Information System
• SDG Uniform Access Interface
• SDG Security System
• SDG Toolkits

SDG CA V1.0 Access Control Toolkit V1.1
63
SDG Security System

• GSI based
• Use certificates to identify users
• Role-based local access control

Full Process of security-related operations under
SDG Security System
64
Security Subsystem
65
SDG Middleware and ToolKits

• SDG Middleware
• Grid Information System
• SDG Uniform Access Interface
• SDG Security System
• SDG Toolkits

SDG Portal (prototype) Image Process Tool
1.0 Storage Sharing Service
66
(No Transcript)
67
Pilot Applications

• Virtual Observatory
• High Energy Physics
• Global Climate Data Integration
• Bioinformatics Integration
• Resources and Environment Monitoring

68
China Virtual Observatory Demo
69
Conclusion

• A whole picture of grid activities in China
• CNGrid,
• China Grid
• NSFC Grid
• SDG
• Shanghai Grid
• Vega Grid
• e-Science and Scientific Data Grid
• Today and tomorrows research demands global
  collaboration e-Science.
• The progress of Information Technology make it
  possible.
• CAS is making great efforts on e-Science with its
  Informatization Program in the 10th Five-year
  Plan.
• The e-Science Program in the 11th Five-year Plan
  (2006-2010) is being worked out. e-Science will
  become the groundwork of research in the future
  five years.
• Scientific Data Grid is the first experimental
  project for CAS e-Science.
• A few of science applications on SDG would be our
  exploration towards e-Science.

70
Thank you!