SCEC Community Modeling Environment SCECCME: Advancing SCEC Simulationbased Earthquake System Scienc

1
SCEC Community Modeling Environment (SCEC/CME)
Advancing SCEC Simulation-based Earthquake
System Science
Philip Maechling SCEC/CME Project Manager 17 July
2006
2
Recent Project Activities

• Jacobo Bielak, Leonard Ramirez-Guzman Hercules
  FE AWM toolchain
• Kim Olsen TeraShake 2 and TeraShake 3
  activities
• Ralph Archuleta, Shuo Ma F.E. Dynamic Rupture
  Code with Topography
• Steve Day Semi-analytical solution to a dynamic
  rupture
• Yolanda Gil, Jihie Kim Wings ICAP Workflow
  Construction Tool
• Tim Ahern, Joanna Muench SCEC Earthworks
  Science Gateway Portal
• Han Chalupsky, Tom Russ DataFinder Metadata
  Search Tool
• Bernard Minster, Geoff Ely SORD Mimetic Dynamic
  Rupture Code
• Carl Kesselman, Ewa Deelman, Gaurang Metha
  CyberShake workflow
• Reagan Moore, Yuanfang Hu SCEC Digital Library
  Developments
• Yifeng Cui, Amit Majumdar TeraShake,
  CyberShake, Petascale benchmarks
• Ned Field, Nitin Gupta, Vipin Gupta OpenSHA,
  CyberShake, SCEC-VDO
• Sue Perry UseIT Intern Program, SCEC-VDO for
  Multi-media literacy
• Dave OHallaron, Julio Lopez Etree velocity
  mesh w/ 10 billion mesh points.
• Robert Graves, Li Zhao CyberShake Code and
  Analysis
• David Okaya, Hunter Francoeur SCEC Earthworks
  Software System
• Amit Chourasia, Steve Cutchin TeraShake
  Animations and Videos
• Marcus Thiebaux, Sridhar Gullapalli TeraShake 2
  Volume Images

3
OpenSHA Preparing Deliverable Software
4
Recent Project Activities
TeraShake 1 Spectral Acceleration at 10 Seconds
with Bump Mapping (Amit Chourasia, Steve Cutchin,
Kim Olsen)
5
CMU Hercules Tool chain
Highly scalable AWM software tools that uses new
technique called In-situ parallel mesh generation
David OHallaron et al (CMU) Etree Mesh
Representation Jacobo Bielak et al (CMU) AWM
6
Earthworks Development (Robust Pathway 2
Capabilities)
Olsen AWM
CVM 3.0
Harvard VM 1.0
Hadley-Kanamori
Hollywood EQ Mw 4.23, 6.98 depth 24x24x12 km
region (160x160x80 nodes) (Dx150 m)
CMU AWM
7
Recent Project Activities
SCEC Earthworks Science Gateway Portal for
Configuring a Pathway 2 Simulation (Joanna
Muench, Hunter Francoeur, David Okaya et al)
8
Recent Project Activities
CyberShake Hazard Curves (3sec SA) for 4 sites
with truncation of low probability events (Robert
Graves, Li Zhao, Scott Callaghan, Nitin Gupta,
Ned Field et al)
9
Recent Project Activities
Combined Cybershake Workflow SGT Generation
Post Processing (Gaurang Mehta, Ewa Deelman,
Yolanda Gil, Jihie Kim, Carl Kesselman, Sridhar
Gullapalli et al)
10
Olsen Dynamic Rupture Simulations
11
Full 3-D Tomography Model (Preliminary)
Po Chen Inversion-based updates to SCEC CVM3.0
Velocity Model.
12
Recent Project Activities
TeraShake 2 Animation with Narration showing
Dynamic Rupture and Wave Propagation (Amit
Chourasia, Kim Olsen, Yifeng Cui et al)
13
Recent Project Activities
TeraShake 2 X Component Velocity Volume Data with
Blue-gtGreen () and Yellow-gtRed (-) motions
(Marcus Thiebaux, Carl Kesselman, Kim Olsen,
Yifeng Cui et al)
14
(No Transcript)
15
SCEC 2 Proposal Year 2000

• SCEC 2 Submitted Dec 2000
• Science plan identified goals as
• System-level basic research to develop a
  predictive understanding of earthquake processes
  using a physics-based approach to 3 main problem
  areas
• Fault systems
• Rupture dynamics
• Wave propagation
• Identified need for substantial IT resources
  including data and modeling requirements.

16
SCEC/CME Proposal Year 2001

• SCEC/CME Submitted April 2001
• Identified goals as
• Improve Seismic Hazard Analysis by using a more
  physics-based approach.
• Introduced concept of SCEC Computational Pathways
  as guide to improving PSHA
• Formed strong interdisciplinary team of
  geoscientists and computer scientists.

17
SCEC/CME Project
Goal To develop a cyberinfrastructure that can
support system-level earthquake science the
SCEC Community Modeling Environment (CME)
Support 5-yr project funded by the NSF/ITR
program under the CISE and Geosciences
Directorates Oct 1, 2001 Sept 30, 2006
NSF CISE GEO
SCEC/ITR Project
USGS
ISI
Information Science
Earth Science
SDSC
IRIS
SCEC Institutions
www.scec.org/cme
18
Information Technology (IT) to Seismic Hazard
Analysis
19
SCEC/CME Computational Pathways
Pathway 1 Standard Seismic Hazard Analysis
20
SCEC/CME Project Development

• SCEC/CME ITR is an National Science Foundation
  (NSF) Information Technology Research (ITR) Large
  Project funded in 2001.

Oct-2002
Oct-2003
Oct-2004
Oct-2001
Oct-2005
Oct-2006
July, 2006 57 months out of 60 months completed
21
Project Assessment

• We maintained a focus on physics-based
  improvements to seismic hazard analysis.

Metadata for Maps IMT 1 Sec Spectral
Acceleration POE 10 TimeSpan 50 Years IMRs 1)
Abrahamson and Silve (1997) 2) Boore, Joyner, and
Fumal (BJF 1993, 1994a) with later
modifications to differentiate thrust and
strike-slip faulting (Boore et al., 1994b), 4)
Sadigh et al. (1993) 4) Campbell and Bozorgnia
(1994). etc ERF (and more)
22
Project Assessment

• Large Collaborative Projects involving computer
  scientists and geophysicists worked together.

• TeraShake
• Kim B. Olsen (SDSU), Bernard Minster (IGPP),
  Reagan Moore (SDSC), Steve Day (SDSU), Phil
  Maechling (USC), Tom Jordan (USC), Marcio Faerman
  (SDSC), Geoffrey Ely (IGPP), Boris Shkoller
  (IGPP), Carey Marcinkovich (EXxonMobil), Jacobo
  Bielak (CMU), David Okaya (USC), Ralph Archuleta
  (UCSB), Steve Cutchin (SDSC) , Amit Chourasia
  (SDSC), George Kremenek (SDSC), Yuanfang Hu
  (SDSC), Arun Jagatheesan (SDSC), Nancy
  Wilkins-Diehr (SDSC), Richard Moore (SDSC), Bryan
  Banister (SDSC), Leesa Brieger (SDSC), Amit
  Majumdar (SDSC), Yifeng Cui (SDSC), Giridhar
  Chukkapalli (SDSC), Qiao Xin (SDSC), Donald Thorp
  (SDSC), Patricia Kovatch (SDSC), Larry Diegel
  (SDSC), Tom Sherwin (SDSC), Christopher Jordan
  (SDSC), Marcus Thiebaux (ISI), Julio Lopez (CMU)
• Workflow Systems Including CyberShake and
  Earthworks
• Hans Chalupsky (ISI), Maureen Dougherty
  (USC/HPCC), Ewa Deelman (ISI), Yolanda Gil (ISI),
  Sridhar Gullapalli (ISI), Vipin Gupta (USC), Carl
  Kesselman (ISI), Jihie Kim (ISI), Gaurang Mehta
  (ISI), Brian Mendenhall (USC/HPCC), Thomas Russ
  (ISI), Gurmeet Singh (ISI), Marc Spraragen (ISI),
  Garrick Staples (USC/HPCC), Karan Vahi (ISI),
  Yifeng Cui (SDSC), Thomas Jordan (SCEC), Li Zhao
  (USC), David Okaya (USC), Robert Graves (URS),
  Ned Field (USGS), Nitin Gupta (SCEC), Scott
  Callaghan (USC), Hunter Francoeur (USC), Joanna
  Muench (IRIS), Philip Maechling (USC)

23
Project Assessment

• We performed large collaborative projects
  involving computer scientists and geophysicists
  worked together.

• 2003
• OpenSHA
• 2004
• OpenSHA, TeraShake
• 2005
• OpenSHA, TeraShake, CyberShake
• 2006
• OpenSHA, TeraShake, CyberShake, Earthworks

24
Project Assessment

• Project is well received and recognized by the
  funding agency.
• Margaret Leinen Geoinformatics 2006, Discussion
  of SCEC Collaboratory and TeraShake including
  animation from SDSC. (http//www.geongrid.org/geoi
  nformatics2006/presentations/slides/Margaret_Leine
  n/GeoinformaticsWkshp.5.2006.ppt)
• National Science Foundation Strategic Plan (2006
  2011) (Draft) Description of SCEC/CME Results
  and Goals on Page 7 (http//www.nsf.gov/about/perf
  ormance/nsfplandraft.pdf)

25
Project Assessment

• We integrated sophisticated geophysics programs
  with sophisticated computer science technology.

26
Project Assessment

• We integrated sophisticated geophysics program
  with sophisticated computer science technology.

• Developed Concept of Computational Platform -
  Definition
• A large-scale implementation of computational
  pathways within a specific computational system
  (hardware software expertise) for producing
  specific knowledge
• Implied components
• Validated simulation software and geophysical
  models
• Broadly useful simulation capabilities
• Imports from other systems. Exports to other
  Systems
• IT/geoscience collaboration involved in operation
• Access to High-performance hardware
• May use Workflow management tools

27
Project Assessment

• We raised SCECs visibility in the computer
  science and high performance computing community.
• Alan Snavely - Congressional Staff Briefing,
  Discussion of TeraShake including animation from
  SDSC. (http//www.sdsc.edu/allans/Snavely-congres
  s-talk.ppt)
• Ian Foster Geoinformatics 2006, Discussion of
  CyberShake (http//www.geongrid.org/geoinformatics
  2006/presentations/slides/Ian_Foster/060512_Geoinf
  ormatics_Foster.ppt)
• Charlie Catlett- TeraGrid06, State of TeraGrid,
  Discussion of TeraShake and CyberShake
  (http//www.teragrid.org/library/StateOfTeraGrid-J
  une2006.pdf)
• IEEE Visualization Contest The TeraShake 2
  volume data is currently being used in an IEEE
  visualization contest. (http//www.cs.unc.edu/7Et
  aylorr/vis06_contest_web_page/data.html)

28
TeraShake / CyberShakeOlsen (SDSU), Okaya (USC)

• Largest and most detailed earthquake simulation
  of the southern San Andreas fault.
• First calculation of physics-based probabilistic
  hazard curves for Southern California using full
  waveform modeling rather than traditional
  attenuation relationships.
• Computation and data analysis at multiple
  TeraGrid sites.
• Workflow tools enable work at a scale previously
  unattainable by automating the very large number
  of programs and files that must be managed.
• TeraGrid staff Cui (SDSC), Reddy (GIG/PSC)

Major Earthquakes on the San Andreas Fault,
1680-present
1906 M 7.8
1857 M 7.8
1680 M 7.7
Simulation of a magnitude 7.7 seismic wave
propagation on the San Andreas Fault. 47 TB data
set.

29
Project Assessment

• We produced physics-based seismic hazard research
  results that advance SCECs earthquake system
  science goals.

30
SCEC/CME Project Developments

• SCEC/CME Project has produced a wide variety of
  useful and impressive results in geosciences,
  computer science, and education and outreach.

Integrated Pegasus/KRR/ICAP Workflow
Docker Knowledge-based UI
SRB-Based Digital Library
CyberShake 1.0
SCEC IT Intern Program
OpenSHA Hazard Maps
TeraShake 1
TeraShake 2
Oct-2004
Oct-2001
Oct-2002
Oct-2003
Oct-2005
Oct-2006
Mid-Term Review and Demos
SCEC Grid
Puente Hills In EQ Spectra
OpenSHA Hazard Curves
SCEC Earthworks On TeraGrid
Gordon Bell Prize For CMUs Quake Group
IRISs SOSA Released
31
Seismic Hazard Analysis Computational Pathways
Standardized seismic hazard analysis Ground
motion simulation Dynamic rupture
modeling Ground-motion inverse problem
1
2
3
Other Data Geology Geodesy
4
Improvement of models
Invert
4
Unified Structural Representation
Ground Motions
Physics-based simulations
AWP
NSR
KFR
AWP
DFR
3
2
Empirical models
Intensity Measures
Earthquake Rupture Forecast
Attenuation Relationship
1
AWP Anelastic Wave Propagation SRM Site
Response Model
FSM Fault System Model RDM Rupture Dynamics
Model
32
SCEC Computational Platform Concept
33
SCEC Community Modeling EnvironmentA
collaboratory for system-level earthquake science
KNOWLEDGE REPRESENTATION REASONING Knowledge
Server Knowledge base access, Inference Translatio
n Services Syntactic semantic translation
Knowledge Base
Ontologies Curated taxonomies, Relations
constraints
Pathway Models Pathway templates, Models of
simulation codes
DIGITAL LIBRARIES Navigation Queries Versioning
, Replication Mediated Collections Federated acce
ss
KNOWLEDGE ACQUISITION Acquisition
Interfaces Dialog planning, Pathway
construction strategies Pathway Assembly Template
instantiation, Resource selection, Constraint
checking
OpenSHA
TeraShake

• Attributes
• System-level scale range
• High-performance hardware
• IT/geoscience collaboration
• Validated software framework
• Workflow management tools
• Well-defined interface

Users
CyberShake
Computational Platform Large-scale implementation
of computational pathways within a specific
computational system (hardware software
wetware) for producing specific results
GRID Pathway Execution Policy, Data ingest,
Repository access Grid Services Compute storage
management, Security
Pathway Instantiations
Storage
Computing
34
SCEC 3 Proposal Year 2005

• SCEC 3 Submitted March 2005
• Science plan identified goals as
• System-level basic research to develop a
  predictive understanding of earthquake processes
  using a physics-based approach to 4 main problem
  areas
• Earthquake Source Physics
• Fault System Dynamics
• Earthquake Forecasting and Predictability
• Ground Motion Prediction

35
PetaSHA Proposal Year 2006

• PetaSHA Submitted Feb 2006
• Proposed a series of large-scale simulations that
  require increasingly demanding computational
  capabilities with Science goals including
• Dynamic Ruptures w/ Topography
• Higher Frequencies Synthetics
• Physics-based PSHA Maps.
• Computing Requirements
• Capability Computing (TeraShake, Quake-type
  simulations)
• Capacity Computing (CyberShake-type workflows)
• Data Intensive Computing (PetaBytes per
  simulation)

36
Making the Case that the SCEC/CME Collaboration
can Benefit SCEC 3 Science

• Wide range of scientific expertise needed for
  complex simulation-based science.
• Strong collaboration with Computer Science
  organizations.
• Experience and ability to perform highly
  collaborative interdisciplinary simulation-based
  projects.
• Participation in SCEC community validation work
  is key to believable simulation results.
• SCEC/CME experience building and using
  cyberinfrastructure.

37
SCEC/CME 2006 All Hands Meeting Overview
38

• Monday 17 July, 2006
• 830 Welcome/Overview/Meeting Goals
• Discussion of SCEC 3 Science and IT Goals
• 1015 TeraShake Platform
• Discussion of TeraShake Platform
• 1200 Lunch and Lunchtime Demo's
• 100 CyberShake Platform
• Discussion of CyberShake Platform
• 200 SCEC Information Technology Capabilities
• Discussion of IT Capabilities
• 400 Planning for SCEC Simulation Science
• Develop CME Phase 2 Science and IT Plan
• 500 Adjourn Day 1
• 600 Cocktails at Faculty Club
• 630 Dinner at Faculty Club

39

• Tuesday 18 July, 2006
• 830 Review of CME Phase 2 Plan
• 845 OpenSHA Platform
• Discussion of OpenSHA
• 935 CME Education and Outreach
• Discussion of CME Education and Outreach
• 1045 Synthetic Seismogram Platform
• Discussion of Synthetic Seismogram Platform
• 1200 Lunch and Lunchtime Demo's
• 130 Develop CME Phase 2 Science and IT Plan
• 230 SCEC/CME End of Year Five Transition Plan
• 300 Adjourn Day 2

40
Working Assumptions
Assumption 1 CME-type Activities will continue
within SCEC to support the SCEC 3 Science
Goals Assumption 2 Experience of CME
collaboration has taught us important lessons
that will benefit CME Phase 2. Assumption 3 No
computing or research is free. Every computing
and research activity (cluster-time, data
storage, visualization, software development,
maintenance of current capabilities, etc.,)
competes for limited IT and or scientific
resources.
41
Meeting Goals

• Discussion periods are built into agenda.
  Discussion during, or in response to,
  presentations is ok.
• If you believe your work (geoscientific or
  computer science) is important to SCEC 3 make
  that case to the group during this meeting.
• By end of the meeting develop a plan CME Phase 2
  Plan with clear priorities indicating our view of
  the Science and IT capabilities needed for SCEC 3
  and CME Phase 2.
• Preserve what works on the CME (Best Practices).
• Drop what has not worked or is not sustainable.
• Establish priorities so we can adjust to levels
  of funding.