Goal 8

1
Goal 8

• Summary by
• Arie Shoshani and Valerio Pascucci (for Wes
  Bethel)
• Goal 8
• Integrate large, complex, and potentially
  distributed software systems with components
  derived from multiple applications domains and
  with distributed data gathering and analysis
  tools.
• Includes 3 areas
• Software integration infrastructure
• Data and storage management
• Analysis and visualization

2
Software Integration Frameworks - 1

• Vision
• Support end-to-end capabilities
• Ask what I want, not how to do that
• Make the data deluge problems invisible
• Broader community simplicity and dependability
• Debugging complex processes, recovery, etc.
• Trust in software VV
• Integration of multiple models and data bridge
  transformers

3
Software Integration Frameworks - 2

• Challenges
• So far this effort did not succeed, lots of
  pieces
• Multiple pieces need to be integrated in a
  single environment
• Need technology to promote reuse and workflow of
  software components
• Need to invest in integrative systems
• Tools for software development lifecycle
• Standards is a challenge rich interfaces for
  effective coupling of components
• Gain trust by showing value

4
Data and Storage Management - 1

• End-to-end capability needs
• Orchestration of multiple tasks
• Workflow at a high level of components
• Automatic data format and transformation
• Uniform treatment of all resources compute,
  storage, network
• Resource management and schedulers that work
  together
• Unified authentication and authorization
  mechanisms
• Ask what I want, not how to do that
• e.g. Want to view a climate image
• Decisions made by system on
• Where is the data
• Which subsets to extract
• Where to do the summarization

5
Data and Storage Management - 2

• Make the data deluge problem invisible
• Vision eliminate the file paradigm
• No need to break large data into files,
  directories, etc.
• Provide disk-based archival storage at 10-100
  GB/s
• Provide large scale data movement
• Energy efficient storage
• Effective parallel indexing and subsetting tools
• Performance estimation

6
Data Analytics and Visualization - 1

• Vision integrative development environment
• There is a hidden costs in vis and analysis
• data exploration, prepare presentations, visual
  debugging, comparative analysis, quantitative
  analysis
• Users want/need robust, reliable systems that are
  simple to use
• Require platforms with massive amounts of I/O and
  memory
• Shared disks - moving exabytes of data
  prohibitive
• Vision do data analysis and visualization on
  SuperComputer
• Vision tools framework for Exascale analysis and
  vis
• Need to integrate vis, analysis, workflow tools
• Vision dynamic custom interfaces for app
  communities
• Put together very quickly alternative
  interface-tools connections
• Share infrastructure between application domains
• Layering of interfaces gt data structures gt
  system
• Standards are needed to enable reusability

7
Data Analytics and Visualization - 2

• Vision Develop tools providing insight for
  exascale data
• Challenges sheer volume, complexity of features
• Current tools do not scale
• Need to visualize and analyze both details and
  coarse scales
• Need scaling to tools for parallel analysis
• Many algorithms need to be redesigned to scale
• Uncertainty characterization
• Robust computations
• Shorten gap between research and deployment
• Reward dissemination and testing of new tools
• Integration of formal mathematical models and
  computer science

8
Data Analytics and Visualization - 3

• Vision Interface research needed to improve
  interaction with users
• Visual interface evaluation
• Commercial applications 50 of software in UI.
• Need languages capturing high level concepts,
  e.g. parallelism
• Vision interfaces that speak to the scientific
  work at hand
• You-to-computer
• Now keyboard, maybe voice
• Computer-to-you
• Printout, monitors,
• Will have better hardware
• High resolution monitors, connectivity,
• Can we take advantage of new hardware coming
• High-res, platforms with 100K cores/nodes,
  multi-modal,
• Interactivity important web is limiting
• Latency, availability, speed, .
• Not having to be concerned with find data,
  reorganize, fit it in available space

9
All Areas

• Vision acceptance and use by broader community
• Need to provide simplicity and dependability
• Debugging complex processes, recovery, etc.
• Methodology for systematic development and
  robustness
• Trust in software VV
• Simple user interfaces for complex task
  orchestration
• Find ways to export expertise, such as feature
  identification analysis