Data mining and discovery of access patterns

1
Data mining and discovery of access patterns

• 3a.i) Adaptive file caching in a distributed
  system (LBNL)
• 3b.i) Dimension reduction and sampling (LLNL)
• 3c.i) Multi-agent based high-dimensional cluster
  analysis (ORNL)
• 3c.ii) Analysis of application level query
  patterns (LLNL, NWU)

SDM kickoff meeting July 10-11, 2001
2
People involved

• Adaptive file caching in a distributed system
  (LBNL)
• Ekow Otoo, Frank Olken
• Dimension reduction and sampling (LLNL)
• Chandrika Kamath, Imola Fodor
• Multi-agent based cluster analysis (ORNL)
• Nagiza Samatova, George Ostrouchov
• Analysis of application level query patterns
  (LLNL, NWU)
• Terence Critchlow, Ghaleb Abdulla, Alok
  Choudhary,
• Agent technology (ORNL, NCSU)
• Tom Potok, Mladen Vouk

3
Targeted Application Area(s)

• First Year Climate, HEP, Astrophysics
• Future Years others (to be determined)

4
Application(s) contact people

• High Energy Physics
• Ask Arie?
• Climate (SciDAC)
• John Drake David Erikson Compute Science and
  Mathematic Division, ORNL
• Ben Santer Program for Climate Model Diagnosis
  and Intercomparison (PCMDI), LLNL
• Astrophysics (SciDAC)
• Tony Mezzacappa Physics Division, ORNL

5
Application Scenario
6
System Architecture
7
Year 1 Deliverables
Distributed simulation product query, search, and
retrieval engine (proof-of-principle
climate-centric search engine)

• VIPAR-based system architecture (Tom)
• Develop and test similarity measures
  clustering algorithms for climate time series
  data comparison (Nagiza)
• Identify, collect begin analyzing meta-data
  from simulation application (Terence)
• Optimization of file migration and replacement
  policies in distributed disk caching using
  simulation models (Ekow)
• (Alok)
• Serial implementations of climateappropriate
  non-linear non-orthogonal dimension reduction
  methods Start dynamic EOFs (Chandrika)