Earthquake Simulation

1
The SCEC

• Earthquake Simulation
• SCEC ITR Collaboration

2
Rupture History of Southern San Andreas Fault
Next earthquake?
R. Weldon, T. Fumal, G. Biasi K. Scharer, this
meeting
3
Major Earthquakes on the San Andreas Fault,
1680-present
1906 M 7.8
1857 M 7.9
1680 M 7.7
4
SHA Computational Pathways
Standardized Seismic Hazard Analysis Ground
motion simulation Physics-based earthquake
forecasting Ground-motion inverse problem
1
2
3
Other Data Geology Geodesy
4
Unified Structural Representation
Invert
4
Faults Motions Stresses
Anelastic model
Ground Motions
AWM
SRM
RDM
FSM
3
2
Intensity Measures
Earthquake Forecast Model
Attenuation Relationship
1
FSM Fault System Model RDM Rupture Dynamics
Model
AWP Anelastic Wave Propagation SRM Site
Response Model
5
Scenario ShakeMaps for M 7.4 Southern San Andreas
Rupture
Without soil basin effects
With soil basin effects
Ned Field, USGS, Pasadena
6
SHA Computational Pathways
Standardized Seismic Hazard Analysis Ground
motion simulation Physics-based earthquake
forecasting Ground-motion inverse problem
1
2
3
Other Data Geology Geodesy
4
Unified Structural Representation
Invert
4
Faults Motions Stresses
Anelastic model
Ground Motions
AWM
SRM
RDM
FSM
3
2
Intensity Measures
Earthquake Forecast Model
Attenuation Relationship
1
FSM Fault System Model RDM Rupture Dynamics
Model
AWP Anelastic Wave Propagation SRM Site
Response Model
7
33 researchers, 8 Institutions
Southern California Earthquake Center San Diego
Supercomputer Center Information Sciences
Institute Institute of Geophysics and Planetary
Physics (UC) University of Southern
California San Diego State University University
of California, Santa Barbara Carnegie-Mellon
University EXonMobil
8
Players

• Kim B. Olsen (SDSU)
• Bernard Minster (IGPP)
• Reagan Moore (SDSC)
• Steve Day (SDSU)
• Phil Maechling (USC)
• Tom Jordan (USC)
• Marcio Faerman (SDSC)
• Geoffrey Ely (IGPP)
• Boris Shkoller (IGPP)
• Carey Marcinkovich (EXxonMobil)
• Jacobo Bielak (CMU)
• David Okaya (USC)
• Ralph Archuleta (UCSB)
• Steve Cutchin (SDSC)
• Amit Chourasia (SDSC)
• George Kremenek (SDSC)
• Yuanfang Hu (SDSC)

• Arun Jagatheesan (SDSC)
• Nancy Wilkins-Diehr (SDSC)
• Richard Moore (SDSC)
• Bryan Banister (SDSC)
• Leesa Brieger (SDSC)
• Amit Majumdar (SDSC)
• Yifeng Cui (SDSC)
• Giridhar Chukkapalli (SDSC)
• Qiao Xin (SDSC)
• Donald Thorp (SDSC)
• Patricia Kovatch (SDSC)
• Larry Diegel (SDSC)
• Tom Sherwin (SDSC)
• Christopher Jordan (SDSC)
• Marcus Thiebaux (ISI)
• Julio Lopez (CMU)

9
TeraShake Modeling Challenge

• Outer scale is large 500km.
• Fault rupture is several 100 km long.
• Broad NOAM-PCFC plate boundary zone
• Strong ground motions felt several 100 km away.
• Use absorbing boundary conditions,
• Inner scale is small 200m.
• Physics of rupture scales of 1 m to 200 m.
• Slow shear velocities in shallow soils l lt 200 m
• Impose afloor on shear velocities 500m/s
  
• Restrict frequencies modeled 0.5
  Hz.

10
Domain Expertise

• Built on 10 years of experience within the
  Southern California Earthquake Center
• Description of fault structure
• Description of velocity model (anelastic seismic
  wave propagation speed through sediment basins
  and through rock)
• Validation of the anelastic seismic wave
  propagation code
• Description of the expected fault rupture scenario

11
SCEC Community Fault Model
A. Plesch and J. Shaw (2003)
12
SCEC Community Velocity Model
H. Magistrale et al. (2000)
13
Computer Expertise

• Optimization of code for execution on parallel
  computer
• Management of parallel output generation at the
  rate of 10 TBs per day
• Management of the archiving of the simulation
  output
• Registration of the output into the SCEC
  community digital library
• Visualization of 1 TB surface data set and 43 TB
  volume data

14
TeraShake Performance
Source Yifeng Cui, Scientific Computing, SDSC
15
TeraShake Simulation Area

• Rectangular region parallel to San Andreas fault
  containing
• Los Angeles,
• San Diego,
• Mexicali,
• Tijuana,
• Ventura Basin,
• Fillmore,
• Southern San Joaquin Valley,
• Catalina Island,
• Ensenada
• 600 x 300 x 80 km

16
TeraShake Earthquake Simulation

• Magnitude 7.7 earthquake on southern San Andreas
• Mesh of 1.8 Billion cubes, 200 m in dimension
• 0.011 sec time step, 20,000 time steps 3 minute
  simulation
• Kinematic source (adapted from Denali) Cajon
  Creek to Bombay Beach
• 60 sec source duration
• 18,886 point sources, each 6,800 time steps in
  duration
• 240 processors on San Diego SuperComputer Center
  DataStar
• 20,000 CPU hours, over approximately 5 days
  wall clock
• 50 Terabytes of output (30 million floppies)
• During execution on-the-fly graphics (attempt
  aborted!)
• Metadata capture and storage in the SCEC digital
  library (Er!)

17
How do you get to 47 TBytes?
Total number of files 150,000
18
Ground Velocities
19
Peak Velocity
20
3D visualization (SDSC)
21
Volume Visualization (ISI)
22
One rupture history, two scenarios
23
(No Transcript)
24
One rupture history, two scenarios
25
Peak Ground Velocity Maps
SE to NW rupture
NW to SE rupture
26
Data Management

• Data moved from GPFS to Sam-QFS for archiving
  (10TB day)
• All output being registered into a SCEC community
  digital library
• Storage Resource Broker data grid technology
• NMI portal interface
• Digital library services to display seismograms
• Visualizations of seismic waves at the surface
• Visualization of seismic wave propagation through
  the volume

27
Lessons

• Very large earthquake simulations are possible
  with current supercomputer technology
• Storage of the full 4D volume remains a
  challenge, especially for data mining
• Visualization of the 4D dynamic fields is
  necessary to fully understand how the wavefield
  samples the earth
• Collaboration between seismologists, geologists,
  and IT experts is essential.