Grid Database Access and Integration

1
Grid Database Access and Integration
2
Aim to

• Requirement and potential functionalities of Grid
  database Service
• The relationship between Grid database service
  and other Grid services

3
A Scenario

• Access an information service
• Select a service based on some criteria
• Authorized to use the service?
• Has access permissions on the data
• How much relevant data is available
• The way the data is stored and queried
• Formulate a request, and dispatch the request

4
Requirements for GDS(1)

• Data Sources and Resources
• Data Sourcesthe stream data in real time from
  instruments and devices
• Data resourcespersistent data stores in file or
  DBMS
• Virtual databasea logical set of data resources

5
Requirements for GDS(2)

• Data Structure and Representation
• Grid must support access to all types of data
• Numeric data
• Text data
• Multimedia data
• Binary format data

6
Requirements for GDS(3)

• Data Organisation
• Organisation types
• Application-specific file structures
• Held in DBMS(relational and object DBMSs)
• Held in XML repositories
• Data States
• Raw data
• Reference data
• Result data set
• Derived data setCreated from other existing data

7
Requirements for GDS(4)

• Provenance
• A record of the origin and history of the data
• The audit trail of the data
• Grid must provide the ability to record and
  access data provenance

8
Requirements for GDS(5)

• Data Access Control
• Data owner can grant?revoke access permission,and
  delegate authority
• Requernment
• Flexiblecombinationsof restrictions and the
  level of granularity
• effectivegrantand revoke all types of
  privileges dynamically
• Easilyprovide facilities or tools to owners

9
Requirements for GDS(6)

• Data publishing and Discovery
• Challenge
• lack of agreed standards for structuring and
  representing data
• Lack of standards for describing published data
• Requenment
• Flexible
• Publish all types of data
• Discoveryname and location

10
Requirements for GDS(7)

• Data Publish Functionality
• Register and deregister data resources
  dynamically
• Schedule and propagate to site holding replicates
• Metadata definition, publication and
  specification processed automated
• Provide intelligent tools to process existing data

11
Requirements for GDS(8)

• Data Discovery Functionality
• Frame the discovery search criteria
• Defined naming conventions and ontologies

12
Requirements for GDS(9)

• Data Retrieval
• Translate metadata terms into physically
  addressable data
• Construct search rules and matching criteria from
  the parameters
• Link the relevant data resource
• Construct distributed queries, heterogeneous and
  federated queries
• Merge and aggregate data from different resource
  to a single, logical set of result data

13
Requirements for GDS(10)

• Modes of Working with Data
• Background workingpredominant method for process
  large volumes of data
• Interactive working
• Data Management Operations
• Unlimited resources to create, process, analyse
• Unlimited volumes of data

14
Requirements for GDS(11)

• Required Attributesarchitectures for gird data
  access and management should have
• Flexibilitylocal changes with the remainder to
  operate unchanged
• Functionalitydata source not reduce the
  functionality of applications
• Performanceenable high-performance applications
• Dependabilityintegrity, availability and
  security
• Manageability
• Composabilityarchitecture must be able to
  efficiently combine computation and data

15
Architectural Principals

• Heterogeneity Transparencyaccess mechanism
  should be independent of the structure of the
  data source
• Location Transparencyirrespective of data
  location
• Name Transparencyaccess data without knowing its
  name or location
• Distribution Transparencymanage data in a
  unified fashion
• Replication Transparencyaccess the replicate
  without knowing them
• Ownership Costing Transparencyspan
  organizational boundaries, and involve multiple
  autonomous data sources

16
Database Access and Integration Functionalities(1)

• Publication and Discovery
• The scope of the DAIS-WG includes
• Defining a registry in which descriptions of data
  services can be published
• Defining standard structures and terms through
  which data services can be described
• The three phases of each operation
• Preparation and validationoperation is checked
• Application
• Result delivery

17
Transactions

• Difference between the Grid and traditional
  transaction environments
• Concurrently executing collaboration chained
  together
• Span multiple regions of control
• Long duration short duration
• Implement approach
• Construction of a core activity service
• Development of a High Level Services

18
metadata

• Types
• Technical metadata
• Contextual metadata
• Derived metadataabout data derived from any
  other data
• Mapping metadatamap relationship between
  contextual metadata

19
ManagementOperation and Performance

• Grid management services must provide
  functionality to integrate database into grid
• Database management services should facilitate
  both operational management and database
  management

20
Data Replication

• Replication service invoked dynamically by
  schedulers or workload managers
• Replication QOS latency of end-to-end data
  propagation

21
Connections and Sessions

• Session a interaction with a service
• Connection a communication with a specific
  database instance
• Difference
• Session have security implications
• Connection established under the context of the
  current user session
• A session may hold many connections
• Connections lifetime is limited to that of the
  session

22
Integration

• Necessity combine information from multiple data
  sets
• Difference with database federation
• Highly dynamic federations In Grid, applications
  dynamically select a set of database to access
• Extreme performance
• Alternative source selection
• Use of Grid standards and Services Federation
  middleware conform to the OGSA

23
Grid database services specification

• GGF 4th Oct 2002
• Generic Definitions
• Relational database services
• XML Database services

24
Thanks !