Dr. Frederica Darema

1
Dynamic Data Driven Applications
Systems (DDDAS) Novel Drivers for the Future
InterNets PERCOM2007
Dr. Frederica Darema Senior Science and
Technology Advisor NSF
2
Science, Engineering, and Commercial
Applications Environments how are they
shaping in the future

• What does it entail forLarge-Scale Computing
• and
• for Large-Scale Networking

3
Compelling Applications Requiring New
Networking Capabilities New Direction for
applications/simulations and measurement
methodologies Dynamic Data Driven Application
Systems (DDDAS) Multi-Directorate/Multi-Agency
DDDAS program NSF, NIH, NOAA with cooperation
with the EU/IST e-Sciences Programs (16M, 32
Projects Funded - www.cise.nsf.gov/dddas)
DDDAS ability to dynamically incorporate
additional data into an executing application,
and in reverse, ability of an application to
dynamically steer the measurement process
Dynamic Integration of Computation
Measurements/Data (from the Real-Time to the
High-End) Unification of Computing Platforms
Sensors/Instruments DDDAS guides sensor systems
architectures
Challenges Application Simulations
Methods Algorithmic Stability Measurement/Instrum
entation Methods Computing Systems Software
Support
Software Architecture Frameworks Synergistic,
Multidisciplinary Research
4
LEAD Users INTERACTING with Weather
InfrastructureNSF Engineering Research Center
for Collaborative Adaptive Sensing of the
Atmosphere (CASA)

• Current (NEXRAD) Doppler weather radars are
  high-power and long range Earths curvature
  prevents them from sensing a key region of the
  atmosphere ground to 3 km
• CASA Concept Inexpensive, dual-polarization
  phased array Doppler radars on cellular towers
  and buildings
• Easily view the lowest 3 km (most poorly observed
  region) of the atmosphere
• Radars collaborate with their neighbors and
  dynamically adapt the the changing weather,
  sensing multiple phenomena to simultaneously and
  optimally meet multiple end user needs
• End users (emergency managers, Weather Service,
  scientists) drive the system via policy
  mechanisms built into the optimal control
  functionality

NEXRAD
CASA
5
LEAD Users INTERACTING with WeatherThe LEAD
Goal Restated - to incorporate DDDAS -
Droegemeier
Interaction Level II Tools and People Driving
Observing Systems Dynamic Adaptation
NWS National Static Observations Grids
Virtual/Digital Resources and Services
ADaM
ADAS
Mesoscale Weather
Tools
Remote Physical (Grid) Resources
Local Physical Resources
Local Observations
Sensor Networks Computer Networks
6
LEAD Service-Oriented Architecture

• Desktop Applications
• IDV
• WRF Configuration GUI

User Interface
LEAD Portal
Crosscutting Services

• Why A Service-Oriented Architecture?
• Flexible and malleable
• Platform independence (emphasis on
  protocols, not platforms)
• Loose integration via modularity
• Evolvable and re-usable (e.g. Java)
• Interoperable by use of standards ?
  robustness

Control
Visualization
Workflow
Education
Browse
Portlets
MyLEAD
Monitor
Query
Control
Ontology
Client Interface
Workflow Monitor
Application Resource Broker (Scheduler)
Stream Service
Control Service
Authorization
Workflow Services
Workflow Engine/Factories
Query Service
Ontology Service
Application Configuration Services
Configuration and Execution Services
Data Services
Host Environment
Execution Description
Authentication
Decoder/Resolver Service
Transcoder Service/ ESML
VO Catalog
Application Description
Application Host
Catalog Services
WRF, ADaM, IDV, ADAS
THREDDS
GPIR
Geo-Reference GUI
Monitoring
Resource Access Services
Scheduler
OPenDAP
Grid FTP
Generic Ingest Service
OGSA-DAI
RLS
SSH
LDM
GRAM
Notification

• Observations
• Streams
• Static
• Archived

Data Bases
Distributed Resources
Steerable Instruments
Specialized Applications
Computation
Storage
7
Beyond Grid Computing Extended Grid
SuperGRID the Application Platform is the
computationalmeasurement system
Applications
Archival/ Stored Data
Computational Platforms
Instruments
Sensors
Measurements
Computational Grids
8
Examples of Areas of DDDAS Impact Funded
Projects span many areas, including

• Physical, Chemical, Biological, Engineering
  Systems
• Chemical pollution transport (atmosphere,
  aquatic, subsurface), ecological
  systems,molecular bionetworks, protein folding..
• Medical and Health Systems
• MRI imaging, cancer treatment, seizure control
• Environmental (prevention, mitigation, and
  response)
• Earthquakes, hurricanes, tornados, wildfires,
  floods, landslides, tsunamis, terrorist attacks
• Critical Infrastructure systems
• Electric power systems, water supply systems,
  transportation networks and vehicles (air,
  ground, underwater, space)
• condition monitoring, prevention, mitigation of
  adverse effects,
• Homeland Security, Communications, Manufacturing
• Terrorist attacks, emergency response Mfg
  planning and control
• Dynamic Adaptive Systems-Software
• Robust and Dependable Large-Scale systems
• Large-Scale Computational Environments
• List of Projects/Papers/Workshops in
  www.cise.nsf.gov/dddas, www.dddas.org

9
Measured Response (A Homeland Security
Simulation)Synthetic Environments for Analysis
and Simulation(SEAS)
Alok Chaturvedi, Director Shailendra Mehta,
co-Director Purdue Homeland Security Institute
10
Interaction between Fire and Structure Models
11
Evacuation Integration of Science-based and
Agent-based Models
12
Virtual Ground Zero

• Integrate independent cross platform simulation
  models
• Provide data exchange capabilities across models
• Enable mutual dependencies between models
• Integrated visualization view of multiple
  simulations

13
(No Transcript)
14
Citizen Agents
15
SEAS Synthetic World
16
SEAS SoS Concept
Web-based Collaborative Modeling Tools
Pandemic
Endemic
Epidemic
Continuous Experimentation
Gaming, Simulation, Experimentation Framework
Content
Contagion
Agents Traits, Emotions, Contagion Models
Sad
Happy
Traits Behaviors
Emotions
aroused
Well-being Emotion
Integrated Infrastructure Models
Media Effect Models
Population Infrastructure Models
Religion Models
Behavior Models
Geography Population Demographics
Computational Models -- Palm top to Teraflop
17
CapWin Project (Gaynor/BU Seltzer/Harvard) Dynam
ically Sharing Sensor and other Data between
heterogeneous resources
Sensor Entry Point
18
A Dynamic Data Driven System for Structural
Health Monitoring and Critical Event Prediction
Seeding effort DDEMA A Data Driven Environment
for Multiphysics Applications
C. Farhat (Stanford), M. Lesoinne( University of
Colorado at Boulder) J. G. Michopoulos (Naval
Research Laboratory) E. N. Houstis, P.
Tsompanopoulou (University of Thessaly) J. R.
Rice (Purdue)

• International Conference for Computational
  Sciences 2003 Melbourne Australia2-4 June 2003

This material is based upon work supported by the
National Science Foundation under Grant No.
0205663
19
PROGNOSIS
20
Real-Time Support for supersonic/hypersonic
multiphysics simulation-based paltform
management Flutter, Temperature Softening of
Skin Material Degredation etc.
Simulation Environment
Physical system
User Interface
Sensor DataHandler
1
9
Basis Solutions Database
6
5
SolutionComposer
2
4
7
ComputationalModel
VisualizationModel/Behavior
3
8
21
Micro-future Simulation of Submarine Room
fuel-leak fire
Location aware mobile or static distributed
sensor network
Before Door Opening
After Door Opening
Red surface 25 C Green Surface 55 C
22
DDEMA Fire-Fighting Scenario
The Grid
23
Fire Model

• Sensible and latent heat fluxes from ground and
  canopy fire - heat fluxes in the atmospheric
  model.
• Fires heat fluxes are absorbed by air over a
  specified extinction depth.
• 56 fuel mass - H20 vapor
• 3 of sensible heat used to dry ground fuel.

• Ground heat flux used to dry and ignite the
  canopy.

Kirk Complex Fire. U.S.F.S. photo
Slide Courtesy of Coen/NCAR
24
Dynamic Data Driven Application System Wildfire
Weather data
Weather model
Map sources (GIS)
Dynamic Data Assimilation
Aerial photos, fuel
Fire model
Sensors, telemetry
Visualization
Supercomputing
Communication
Software engineering
Jan Mandel and Team
25
MIPS A Real-Time Measurement-Inversion-Prediction
-Steering Framework for Hazardous Events
V. Akcelik (Stanford SLAC) G. Biros
(University of Pennsylvania) A. Borzi
(Graz) A. Draganescu (Sandia, NM) J. Hill
(Sandia, NM) O. Ghattas (UT Austin) B. Van
Bloemen Waanders (Sandia, NM) K. Willcox (MIT)
26
Sensitivity to sensor array density
27

Sensor and Computational Grids for Dynamic
Data-Driven Contaminant Dispersion Prediction
Farhat Michopoulos, Naval Research Laboratory
Objective Development of methodology for
achieving real time detection and prediction of
Chemo/Bio-contaminant dispersion under various
weather conditions, enabling the protection of
warfighters and civilians in urban or industrial
environments.
Benefit to warfighter Information superiority,
C4IR integration, rapid and accurate assessment
of COP and CBRN, and automated decision support.
28
WIPER - DDDAS Three Layer Architecture

• Data Source and Measurement
• Detection, Simulation, and Prediction
• Decision Support System (DSS)

29
DDDAS SERVICE ORIENTED ARCHITECTURE
30
DynaCode A General DDDAS Framework with Coast
and Environment Modeling Applications
Hurricane ensemble modeling Coupling ocean
circulation, storm surge, wave generation models
for the Gulf Integrating data from regional
observing systems for realtime coastal forecasts
in SE Event driven, dynamic component framework
with algorithm selection, optimization tools,
workflow, data assimilation, result validation
with sensor/satellite. Ecological restoration
and control Coupled models (hydrodynamic,
salinity, geomorphic, sediment) control
diversion, sensors/wind fields inject real
time data.
Louisiana Coastal Area
Infrastructure algorithms to couple models, to
each other and to external inputs from sensors,
wind databases to optimize execution of complex
workflows on grids, invoking appropriate models,
meshes, and algorithms, depending on current
conditions.

• Multidisciplinary Team
• Gabrielle Allen (LSU)
• Greg Stone (LSU), Johannes Westerink (Notre
  Dame), Burak Aksoylu (LSU), Ivor van Heerden
  (LSU), Ed Seidel (LSU), Robert Twilley (LSU)

31
Critical Infrastructure SystemsSurfaceTransportat
ion (eliminating the tyranny of commuters safer
response evacuation of cities in crisis
situations)
Washington Post Feb 20, 2006 Article that little
voice

• Richard Fujimoto, et al (Georgia Inst of Tech)
• Delays in surface transportation systems today
  cost tens of billions of dollars annually in the
  U.S. in lost productivity, wasted fuel, and
  pollution. In times of crisis, delays can result
  in lost lives.
• The project developing novel ad hoc distributed
  simulations that feature dynamic collections of
  autonomous in-vehicle simulations interacting
  with each other and real-time data in a
  continuously running distributed simulation
  environment. Each simulator models some portion
  of the transportation network, and exchange data
  with other simulators through a mobile, wireless
  network to predict future states of the overall
  system.
• Ad hoc distributed simulations combine elements
  of conventional distributed simulations and
  replicated simulation runs, together with dynamic
  and continuous monitoring. Incorporating
  dynamically monitoring data poses challenges of
  data distribution and synchronization a
  synchronization protocol based on rollback
  mechanisms has been designed for use in these
  systems.

32
Critical Infrastructure SystemsElectrical
PowerGrids

• Auto-Steered Information-Decision Processes for
  Electric System Asset Management
• James McCalley, et al (Iowa State University)
• Multi-disciplinary research and industry
  collaboration

• Electrical Engineering Power Systems
• Computer Sciences Data Integration, ML, Agents
• Statistics Reliability, Decision
• Computer Engineering Sensor Networks
• Aerospace Engineering Nondestructive Evaluation
• Industrial Engineering Stochastic Optimization

AREVA (Energy Company)

• Layer 1 Long-term power system simulation
• Areva commercial grade simulator (DTS), Iowa/ISU
  grid
• Layer 2 Sensing and communications
• One or two field installations on campus,
  wireless sensors
• Layer 3 Data integration
• Ontology-based, query-centric, federated
• Layer 4 Converting condition data into failure
  predictors
• Steady-state transient failure probabilities
• Layer 5 Integrated decision algorithms
• Interacting, rolling, multi-objective, stochastic
  optimization
• Two stage analysis for uncertainty reduction to
  decide new sensor measurements

Advances through the project are aimed to enable
enhanced electrical power-systems management
Enable economic and efficient management of
electrical power-grids, foresee and mitigate
failures and widespread blackouts. Enhance the
nations electrical energy distribution health
and preparedness in cases of natural and man-made
disasters
33
Critical Infrastructure SystemsUrban Water
Distribution Management Systems (WDS)

• Kumar Mahinthakumar, et. al. (NCState University,
  University of Chicago, University of Cincinnati,
  and U of South Carolina)

• Multidiciplinary research collaboration with
  industry partners from the Greater Cincinnati
  Water Works and the Neptune Technology Group to
  implement and test the cyberinfrastructure for a
  working WDS.

• Threat management in WDSs involves real-time
  characterization of any contaminant source and
  plume, design of control strategies, and design
  of incremental data sampling schedules.
• Requires dynamic integration of time-varying
  measurements along with analytical modules that
  include simulation models (evolutionary
  algorithms), adaptive sampling procedures, and
  optimization methods.
• A live demonstration of this preliminary
  cyberinfrastructure using Suragrid resources was
  carried out at the Internet2 meeting in Chicago
  in December 2006.

34
The Instrumented Oil Field PI Prof. Mary
Wheeler, UT AustinMulti-Institutional/Multi-Resea
rcher Collaboration
Detect and track changes in data during
production Invert data for reservoir
properties Detect and track reservoir
changes Assimilate data reservoir properties
into the evolving reservoir model Use simulation
and optimization to guide future production,
future data acquisition strategy
35
CS advances for Curing Cancer - I

• Mesh Generation and Optimistic Computation on
  the Grid, enables recent multidisciplinary
  collaborative research for Advances in Brain
  Tumor Neurosurgery
• N. Chrisochoides, et al (College of William and
  Mary Harvard Medical School Aerospace
  Inria/Sophia Antipolis France)
• Problem brain-neurosurgery challenges tissue
  deformation, due to swelling, fluid loss
• Solution image-guided neurosurgery,
• The novel approach uses dynamic data
  (intra-operative brain images) for landmark
  tracking across the entire brain volume, and the
  cyber-infrastructure together with distributed
  Dynamic Data Driven Application (Software) System
  (DDDAS) capabilities, developed under this
  collaboration between the researchers at College
  of William and Mary, Harvard Medical School, and
  Inria (Sophia-Antipolis) in France.
• Using software methods developed by the
  investigators enabled remote use from Harvard, of
  269 processors at CWM reducing the per image
  registration time from 3000 secs to 30secs.
• Such DDDAS environments will drive the next
  InterNet requirements GENI.

• These these recent advances though DDDAS systems
  have enabled delivering in real-time registration
  of pre-operative and intra-operative images.
  First time ever in clinical practice, presented
  on-time brain shift/deformation displacements to
  neurosurgeons at Brigham and Women's Hospital
  (BWH) during tumor resection procedure.

36
Beyond Grid Computing Extended Grid
SuperGRID the Application Platform is the
computationalmeasurement system
Applications
Archival/ Stored Data
Computational Platforms
Instruments
Sensors
Measurements
Computational Grids
37
DDDAS New Directions Capabilities

• Dynamic Integration of Computation
  Measurements/Data
• Pervasive computing pervasive data/monitoring
• from the Real-Time to the High-End
• from the sensors to the High-End
• from the PDA to the High-End
• Unification of Computing Platforms
  Sensors/Instruments
• Dynamic Data Driven
• at the application level
• at the systems level (computational platforms,
  networks)
• Pervasive monitoring of resources and dynamic
  adaptation
• DDDAS for management and control of
  sensor-network systems
• architecture (e.g combining network)
• optimization of configuration of heterogeneous
  sets of sensors
• Placement (mobile sensors) re-configuration
• Prioretization of which sensors to monitor, at
  what frequency
• Power management (which sensors to turn on/off,
  frequency of measurement, etc)

38
Large-Scale Systems -

• Processing at multiple levels
• Computation and data processing both at the
  application and the instruments/sensors side
• New Computational Units
• Beyond commodity microprocessors /superscalar /
  (D)MT GPU/(GP)2Us (MC-P), MT, FPGAs, GPUs,
• Populating high-end platforms, workstations,
  visualization servers, data servers, etc,
• Potentially
• MC-Ps, FPGAs, GPUs at application side
• MC-Ps, FPGAs, GPUs at the data acquisition side
• One winner EVERYWHERE???
• Or Mix of MC-Ps, FPGAs, GPUs???
• Main Engine or Co-Processors???
• Pros deficiencies in each - advances close
  gaps

39
Large-Scale Systems - SuperGrids

• Multiple types of such processors together
• Tightly coupled loosely coupled
• clusters of a kind, or processor and
  co-processors,
• hierarchically coupled (Platform Heterogeneity)
• Applications complex, multi-components
• Map applications - different components of an
  application on the appropriate processing units
• Specifically GPUs - (GP)2Us or Co-processors?
• GPUs beyond graphics and games
• Sorting (GPUTeraSort), GAMESS,
• GPU co-processors
• BLAS, FFT, CFD,
• Challenge irregular (mesh) and Monte-Carlo
  applications
• Significant excitement - the number of GPU-based
  platforms (built planned) is increasing

40
Vision for the Future InterNets

• The networking infrastructure will be
• Heterogeneous (nodes, connectivity), dynamic
  (ad-hoc), scalable
• wired, wireless mobile/fixed devices and sensors
• from cellular to satellite, from the PDA to the
  High-End
• Autonomic Management of Networks
• Continuous resource monitoring
• Adaptive management of resources
• Performability (performance, dependability,
  fault-tolerance), Security,
• Dynamic Data Driven Applications Systems are the
  drivers for the next InterNet
• (the future InterNets will need to support DDDAS
  environments)
• www.cise.nsf.gov/dddas -- www.dddas.org

41
back-up