Research topics in data warehouse

1
Research topics in data warehouse

• Directed By Dr Rahgozar
• Mostafa h.Chehreghani

2
List of research topics

• Lineage tracing
• Incremental view maintenance
• Indexing in data warehouse
• Data quality

3
Lineage tracing

• List of papers
• Using AutoMed Metadata in Data Warehousing
  Environments
• A Tutorial on the IQL Query Language
• Practical Lineage Tracing in Data Warehouses
• Incremental view maintenance and data lineage
  tracing in heterogeneous database environments
• A Framework for supporting data integration using
  the materialized and virtual approaches

4
Lineage tracing

• Automed model for metadata in data warehouse
• Use tag for relations
• Use a language such as IQL
• Node , Edge , Constraint
• IOL
• Functional and typed language
• Prefix and Infix functions
• New functions by lambda
• lambda x,y,z (() (() x y) z)

5
IQL

• let v q1 in q2
• let v (() 200 500) in (() v v)
• union R S
• duplicate elimination distinct (R)
• setUnion R S ? distinct (R S)
• difference R S
• projection x,z x,y,z lt- R
• Cartesian product and Joins
• gc agFun xs
• map f xs
• Grouping and Aggregation Operations

6
Using IQL in Automed

• Example Enforce unique key constraint
• () (count (distinct n s,n lt-
  ltltStudent,namegtgt)) (count ltltStudentgtgt)
• Name field
• Student table

7
Example of lineage tracing
8
Example of lineage tracing

• TS1,S2
• addNode (dept,Maths,CompSci)
• addNode (person, x x mathematician x x
  compScientist)
• addNode (avgDeptSalary, avg s (m,s)_,
  mathematician, salary
• avg s (c,s)_, compScientist, salary)
• addEdge (_, dept, person, ( Maths, x) x
  mathematician
• (CompSci, x) x compScientist)
• addEdge (_, person, salary, _,
  mathematician,salary _, compScientist,
  salary)
• addEdge (_, dept, avgDeptSalary, ( Maths,
  avg s (m,s) _, mathematician, salary),

9
Example of lineage tracing

• (CompSci, avg s (c,s)_, compScientist,
  salary))
• delEdge (_, mathematician, salary, (p, s) (d,
  p) _, dept, person (p, s) _, person,
  salary
• d Maths p p)
• delEdge (_, compScientist, salary, (p, s) (d,
  p) _, dept, person (p, s) _, person,
  salary
• d CompSci p p)
• delNode (mathematician, p (d, p) _, dept,
  person d Maths)
• delNode (compScientist, p (d, p) _, dept,
  person d CompSci)

10
Incremental view maintenance

• List of papers
• Incremental view maintenance and data lineage
  tracing in heterogeneous database environments
• View maintenance in a warehousing environment
• A System Prototype for Warehouse View Maintenance

11
Incremental view maintenance

• Di set of base relations
• ?Di bags inserted into Di
• Di bags deleted from Di
• V materialized view
• ?V bags inserted into V
• V bags deleted from V
• Vnew (V ?V) -- V
• Minimality condition
• ?V C V
• ?Vn V Ø

12
Incremental view maintenance
13
Incremental view maintenance
14
Indexing in data warehouse

• Paper
• Bitmap Index Design and Evaluation
• Advantages
• Compact size
• Efficient hardware support for bitmap operations
  (AND, OR, XOR, NOT)
• Fast search

15
Bitmap Index
16
Bitmap Index
17
Data quality in data warehouse

• List of papers
• Towards Quality-Oriented Data Warehouse Usage and
  Evolution
• Data Quality Problems and Proactive Data Quality
  Management in Data-Warehouse-Systems
• Data Warehouse Data Policy
• Fitness for use
• Subjective
• Related to end users
• Objective
• Definition of system
• Models
• GQM Goal Question Metric
• English

18
GQM

• Goal factor
• Importance of each factor determined respect to
  Goal
• Quality dimension
• Data coherence
• Data Completeness
• Data freshness

19
GQM