UDAL Unified Data Access Layer

1
UDAL Unified Data Access Layer
Lukasz Dutka, Jacek Kitowski, Renata Slota, Darin
Nikolow, Bartosz Kryza Cyfronet AGH Krakow

2
Agenda

• Problem description
• Architecture
• Functionality

3
Problem description

• Huge amount of data requires optimization
• Data stored in heterogeneous environment
• Disk Drives
• Disk Arrays
• HSM systems like Castor, DiskXtender and so on.
• Data replicated and distributed over the world
• Overloaded storage hosts load balancing
  required
• Decision where to get data from, where to store
  data is necessary
• To take the decision we need to know prediction
  about current storage performance

4
Goals

• A1 Unified Data Access Platform building the
  unified and highly expandable data access
  platform for grid storage nodes, selected
  database, optimized data access and predicted
  data access factors for data,
• A2 Data Access Estimators development of
  data access estimators for HSM system, disk
  drives and data bases, compatible with the
  unified platform,
• A3 Data Access Handlers development of data
  access handlers compatible with the platform and
  realizing some optimization and unification
  tasks,
• A4 Integration with EDG integration of the
  unified data access platform with EDG-Replica
  Manager.

5
UDAL - Unified Data Access Layer
6
UDAL Deployment
Selects replica on the base of gathered costs
Application of UDAL for Estimation of Data Access
Cost in Heterogeneous Environment
Application of UDAL for Data Access Stored in
Heterogeneous Storage
7
UDAL - Unified Data Access Layer
UDAL is supposed to cover heterogeneity of
Storage. It is based on Component Expert
Architecture allowing easy adaptation to new
problems.
8
ECMS processes request and fires Expert System
Get additional information form an external
databases
Component (TID1, SPECa)
UDAL is based on Component-Expert Architecture
introducing new paradigm in component selection
problem
Rule base expert-system
Rule base expert-system
Component (TID1, SPECa)
Component (TID2, SPECb)
Take into account only components with TID1 type
Component (TID3, SPECg)
Component (TID3, SPECe)
Component (TID3, SPECd)
Component (TID3, SPECa)
9
(No Transcript)
10
(No Transcript)
11
Functionality

• Universal and flexible Component-Expert
  Architecture base programming framework
• Built-in expert system with easy to modify rules
  selection the best components for current context
  and purpose
• Set of data access estimators for
• All secondary storage devices
• Estimator for Castor
• Estimator for DiskXtender
• Set of sample data access handlers providing data
  in unified way from storages available trough
  different protocols RFIO, FTP etc. as well as
  specialized for different hardware installation
  disks, disk arrays, HSMs and so on.

12
Most Important Achievements

• Flexible component platform - UDAL ready for
  future changes and adaptations
• Precise estimators for
• Secondary Storage
• Tertiary Storage as Castor and DiskXtender
• Integration UDAL with EDG Optor and Reptor making
  the whole thing more universal as well as more
  transparent

13
More information

• Technical contact
• dutka_at_agh.edu.pl

14
(No Transcript)