Coordinating BackupRecovery and Data Consistency Between Database and File Systems

1
Coordinating Backup/Recovery and Data Consistency
Between Database and File Systems

• Suparna Bhattacharya
• IBM Software Lab, India
• C. Mohan, Karen W. Brannon, Inderpal Narang,
  Hui-L Hsiao, Mahadevan Subramanian
• IBM Almaden Research Center

2
Outline

• Intro
• Configuration
• Backup
• Restore
• Consistency
• QA

3
1.Intro

• Motivation
• Much of the worlds data lives in files and most
  data would continue to live there and volumes
  will grow.
• Related Corba and OLE DB framework -gt provide
  uniform access to database and nondatabase data.
• File system do not provide sufficient metadata,
  no query.

4
1.Intro (cont.)

• Motivation
• DBMS may not always be appropriate for the
  storage of very large objects.
• LOB type for storing large obj in DB, but LOB
  require the use of SQL API -gt compatibility
• iFS oracle internet FS supports file system
  interface SHORE supporting both DB/FS
  interface.
• Using LOB actually.
• They dont supporting hierarchical storage
  management, such as Tivoli TSM.
• Performance degradation due to LOB.
• Legacy data in files should be loaded into DBs.

Xiang Xiaojia
Department of Computer Science Slide 4
5
1.Intro (cont.)

• Motivation
• The importance of integrating files containing
  unstructured and semi-structured data with
  business applications is growing.

Xiang Xiaojia
Department of Computer Science Slide 5
6
1.Intro (cont.)

• Motivation
• Datalinks extends DBMS functionality of
  management of data to files stored in file
  systems while providing
• Referential integrity
• Access control
• Coordinated backup/restore
• Advantage
• Good Performance.
• Supporting existing/new applications based on the
  file paradigm.

Xiang Xiaojia
Department of Computer Science Slide 6
7
1.Intro (cont.)

• Motivation
• Datalinks
• Basic semantic
• Consistency between reference objecte
• Three Issues
• Backup
• Restore
• CM-Consistency

Xiang Xiaojia
Department of Computer Science Slide 7
8
2.Configuration

• Overview

Xiang Xiaojia
Department of Computer Science Slide 8
9
2.Configuration(cont.)

• Overview
• Introduce new datatype in DBMS, but not any new
  API for DB/file access
• Advantages
• Current DBMS apps can be easily extended
• Apps to work on file paradigm need not change
• Negligible impact on performance

Xiang Xiaojia
Department of Computer Science Slide 9
10
2.Configuration(cont)

• DLFM (DataLink File Manager)
• Supporting transactional properties to DB data
  and references to files
• Supporting coordinated backup/recovery between DB
  and files.
• Acts as subordinate in TPC.
• Keeping information about operations(LinkFile/Unli
  nkFile).
• Interfacing with a backup/archive server.

Xiang Xiaojia
Department of Computer Science Slide 10
11
2.Configuration(cont)

• DLFF (DataLink File System Filter)
• Interceptor open rename delete
• Dose not come in the normal file r/w path
• Implementation layered approach, installable fs.
• Access control token handle expire time,
  validating checksum.

Xiang Xiaojia
Department of Computer Science Slide 11
12
2.Configuration(cont)

• Link API
• Linking a file and placing it under the control
  of the DBMS.
• Applying certain constraints.
• DB-Owner The DBMS becomes the owner of the
  linked file.
• RO the linked file is read only, guarantee the
  integrity of the indice
• Referential Integrity Constraint Metadata is
  maintained to indicate that the file is
  referenced by the DBMS

Xiang Xiaojia
Department of Computer Science Slide 12
13
2.Configuration(cont)

• Link API
• Constraints

Xiang Xiaojia
Department of Computer Science Slide 13
14
2.Configuration(cont)

• Link API
• Example

Xiang Xiaojia
Department of Computer Science Slide 14
15
2.Configuration(cont)

• Link API
• Recovery identifier

Xiang Xiaojia
Department of Computer Science Slide 15
16
2.Configuration(cont)

• Link API
• Copy Table
• Copy in batch mode asynchronously.

Xiang Xiaojia
Department of Computer Science Slide 16
17
2.Configuration(cont)

• Unlink API
• Backup Number is assigned when unlinking

Xiang Xiaojia
Department of Computer Science Slide 17
18
3. Backup

• Datalinks supports both online/offline backups
• Disadvantages of copy referenced files directly
• Reading each record at backup time is time
  expensive. Uncommited updates impact
• Large files
• Our solution Backup when linking

Xiang Xiaojia
Department of Computer Science Slide 18
19
3. Backup (cont.)

• Coordinated Backups
• Backup Verify
• If all files whose recovery identifier satisfies
  the above formula are backed up, then the verify
  call return success
• Online backup the DBMS will issue backup verify
  on completing the backup on the database side.

Xiang Xiaojia
Department of Computer Science Slide 19
20
3. Backup (cont.)

• Coordinated Backups
• Backup End
• DBMS-gtall DLFMs - distributed transaction

Xiang Xiaojia
Department of Computer Science Slide 20
21
4. Restore

• Recovery states

Xiang Xiaojia
Department of Computer Science Slide 21
22
4. Restore (cont.)

• Restore for OBS
• DBMS provide Restore_Recovery_ID
• DLFM do the following jobs
• Relinking a file implies restoring the file
  version when it was linked

Xiang Xiaojia
Department of Computer Science Slide 22
23
4. Restore (cont.)

• Restore for PTS and QPS
• Log rollforward

Xiang Xiaojia
Department of Computer Science Slide 23
24
5. Loosely Coupled Transaction Model
Xiang Xiaojia
Department of Computer Science Slide 24
25
5. Loosely Coupled Transaction Model

• Loosely Couple
• No db locks are held
• SQL INSERT - Link
• SQL SELECT - Access
• SQL UPDATE Update
• SQL DELETE Deassociate
• No lock -gt Consistency Detection

Xiang Xiaojia
Department of Computer Science Slide 25
26
5. Loosely Coupled Transaction Model

• Basic Detection

Xiang Xiaojia
Department of Computer Science Slide 26
27
5. Loosely Coupled Transaction Model

• Encoding version number in the Token

Xiang Xiaojia
Department of Computer Science Slide 27
28
QA

• ??!