OGSA-DAI Users

1
OGSA-DAI Users MeetingIntroductionMalcolm
AtkinsonDirectorwww.nesc.ac.uk7th April 2004
2
Overview

• Todays Meeting
• Users meet each other
• We hear Users Complaints, Requirements
  Priorities
• Users understand better the limits of one teams
  work
• We jointly identify the way ahead
• OGSA-DAI Vision
• What we aspire to achieve
• Short, medium and long term
• Relationship with GGF DAIS WG Standardisation
  effort
• OGSA-DAI Today
• Current and Imminent Releases
• The Road Map
• Which you the Users should influence

3
OGSA-DAI History Lesson

• DB BiGUM meetings
• August 2001 Paul Watson paper prime mover
• Database Task Force
• 01Q3 02Q3
• Norman Paton, Dave Pearson Paul Watson
• http//www.cs.man.ac.uk/grid-db/dbtf.html
• Architecture Task Force
• 01Q3 02Q2
• Urged UK engagement in OGSA
• OGSI Sep 01 to Sep 03 http//www.nesc.ac.uk/teams/
  atf/01-03/
• OGSA-DAI project
• Start Feb 02 end Sept 03
• Produced releases R1 to R3.1
• Produced Alpha Release of DQP
• http//www.ogsadai.org.uk/
• DAIT project
• Start Nov 03 end March 06
• Continues OGSA-DAI development

4
OGSA-DAI Partners
IBM USA
EPCC NeSC
Glasgow
Newcastle
Belfast
Manchester
Daresbury Lab
Cambridge
Oxford
Oracle
Hinxton
RAL
Cardiff
London
IBM Hursley
Southampton
5
OGSA-DAI Vision 1

• There are many sources of Data
• Structure is the essence of using data
• Structure supported by DBMS Relational XML
• Structure also recognised in files BinX GGF
  DFDL
• Sources of Data are Autonomously Managed
• Vary in model, structure, schema
• Vary in content
• Vary in Access Use policies
• Vary in Platform
• Discovery Design enabled by Combining Data
• From multiple sources
• Recorded in multiple data resources

6
OGSA-DAI Vision 2

• Developing applications using heterogeneous data
  sources is difficult
• Dealing with all those variations
• Negotiating with data owners and data platforms
• Communication protocols and data movement
• Data translation
• Distributed operations
• Scale, reliability, coping with change, failure
  recovery, security, privacy
• Currently solved incrementally on a per
  application basis
• OGSA-DAI to the rescue

7
OGSA-DAI Vision 3

• OGSA-DAI will provide a generic framework
• Standard components for dealing with common
  requirements
• Query, update, bulk load, schema edit, metadata
  access, transactions,
• Agreed protocols and standard agreements with
  DBMS
• Established mechanisms for handling owner policy
  requirements
• Provides data translation
• Supports distributed operations
• Delivers
• Scale, reliability, coping with change, failure
  recovery, security, privacy
• Costs of developing a robust OGSA-DAI amortised
  over many applications
• Development of Applications significantly
  accelerated
• Accepted de facto standard by DBMS vendors and
  data owners
• Efficiency from direct support and system or
  resource knowledge
• Collaborating International Community Extending
  the Framework
• E.G. the work of OGSA-WebDB

8
OGSA-DAI Vision 4

• Progressively develop the functionality
• Progressively raise level of operations
• Progressively improve quality
• Scale, Performance, Reliability, Flexibility,
  Usability
• Adapt to changing context
• Support user community
• Training, Documentation, Consultation,
  Collaboration
• Place High Value on Users Application
  Developers Ingenuity
• They know about the data resources
• They invent new ways of combining and analysing
  data
• Therefore Extensibility and Flexibility MUST
  always be supported

9
OGSA-DAI today
1919 Downloads so far from http//www.ogsadai.org.
uk/
Release 4 April 04 from http//www.ogsadai.org.uk
/
Release 5 October 04 from http//www.ogsadai.org.
uk/
Releases 67 2005 Many options You must help
us choose
10
Download analysis
11
And Now

• Users Presentations
• What are you doing with our baby
• What do you like
• What do you want changed, extended or added
• What consensus is there
• Team Presentations
• More details of current and planned release
• The issues that face us
• Choosing Interoperation Platforms
• Choosing which requirements get priority
• Working Groups
• Collation of the Groups conclusions
• Ideas for Future User Group Activities

URGENT Questions ?
12
Reserve Slides Follow
13
OGSA-DAI Behaviours
Request to Registry for sources of data about x
Registry GDSR
Request to Factory for access to database
Registry responds with Factory handle
Factory GDSF
Analyst
Factory returns handle of GDS to client
Factory creates GridDataService
Client queries GDS with SQL, XPath, XQuery etc
Database (Xindice, MySQL Oracle, DB2)
Query results returned XML
Grid Data Service GDS
ORdelivered to consumer as XML
Consumer
GDS interacts with database
14
Delivery Patterns
15
Statement Operations

• Receives statement
• Performs statement against a data resource
• Returns results

Statement Component and Database Connections
Statement
Results
Data Resource
16
Statement Document
lt?xml version"1.0" encoding"UTF-8" ?gt
ltGridDataServiceScriptgt ltHeadergt Name,
versioning, configuration specification
lt/Headergt ltBodygt ltDocumentationgtUpdate
with data delivered with the scriptlt/Documentation
gt ltDefineParameter name"table"gtmyimageslt/De
fineParametergt ltDefineParameter
name"id"gtid 2lt/DefineParametergt
ltStatement name"xyz" dataResource"MyDataResource
"gt SELECT FROM ltUseParameter
reference"table" /gt WHERE
ltUseParameter reference"id" /gt
lt/Statementgt ltDelivery name"delivery1"gt
ltMechanism type"bulk" /gt ltMode
type"full" /gt ltFromgtxyzlt/Fromgt
ltTogtresponselt/Togt lt/Deliverygt
ltExecute name"execute1"gtxyzlt/Executegt
lt/Bodygt lt/GridDataServiceScriptgt
17
Statement Architecture
Engine/GDS-S
Statement
Statement Activity
Statement Handlers
JDBC Statement Handlers
XMLDB Statement Handlers
JDBC Connection
XMLDB Connection
XML DB
Relational DB
JDBC API
XMLDB API
18
OGSA-DAI Security

• Grid Security Infrastructure Authentication
• The role mapper takes credentials and a database
  name and returns a dbRole
• dbRole contains username and password

Role Mapper
dbRole
username
Credentials
Database Name
password
19
Data Access Integration Services
20
Future DAI Services?

1a. Request to Registry for
sources of data about x
Data

y

Registry

1b. Registry

responds with

Factory handle

2a. Request to Factory for access and

integration from resources Sx and Sy

Data Access Integrationmaster

2c. Factory

returns handle of GDS to client

3b. Client
2b. Factory creates

tells

GridDataServices network

analyst

Client

3a. Client submits sequence of

scripts each has a set of queries

GDTS

to GDS with XPath, SQL, etc

1
XML
Analyst

GDS

GDTS

database

GDS

2
S

x
GDS

S

y
3c. Sequences of result sets returned to

Relational
analyst as formatted binary described in

GDTS

GDS

GDS

2
3
a standard XML notation

database

1
GDS

GDTS