Grid Computing for Energy and

1
Grid Computing for Energy and Environmental
Applications Mary F. Wheeler Center for
Subsurface Modeling The University of Texas at
Austin
2
Grid Computing Research Team
Joel Saltz Umit Catalyurek Mike Gray Tahsin
Kurc Shannon Hastings Steve Langella Krishnan
Sivaramakrishnan Biomedical Informatics
Department The Ohio State University
Manish ParasharManish Agarwal Electrical and
Computer Eng. Dept. Rutgers University
Alan SussmanChristian Hansen Computer Science
Department University of Maryland
Nancy Wilkins-Diehr Yifeng Cui Dong Ju Choi San
Diego Supercomputing Center
Mary Wheeler Malgorzata Peszynska Clint
Dawson Xiuli Gai Mrinal Sen Paul Stoffa CSM and
UTIG University of Texas at Austin
Mike Papka Computer Science Department University
of Chicago
3
Outline

• Challenges and Motivation
• Examples Instrumented Oilfield and 4D Seismic
• Grid-enabled Production Optimization and
  Reservoir Management
• Overview
• Computational Portals (IPARS, seismic simulation)
• MACE, DISCOVER, DataCutter
• Grid Collaboration
• Results
• Optimizing Production
• Economic Model Results
• Optimal Well Placement
• Data Assimilation

4
Challenges and Motivation

• Increasing Production from Existing Oil Gas
  Reservoirs is
• Crucial for U.S. Economy and Global Energy
  Consumption

Graphics courtesy of the Institute for Energy and
the Environment, University of Texas at Austin
5
Challenges and Motivation (cont.)

• Demand for Natural Gas Expected to Increase

Conventional Non-conventional
Nat. Gas Coal Bed Tight Gas Sands Gassified
Coal Gas Hydrates
Crude Oil Heavy Oil, IOR Tar Sands Oil Shales
Fossil fuel reserves are not exhausted. There
are 2.02 trillion barrels oil equivalent in
proven reserves and 2.11 trillion barrels in
undiscovered fields. Non-conventional
hydrocarbon sources can be tapped for additional
trillions of barrels of oil equivalent
Graphics courtesy of the Institute for Energy and
the Environment, University of Texas at Austin
6
A Distributed, Collaborative Data Analysis
Scenario
DATASET
VISUALIZATION
CLIENT
CLIENT
VISUALIZATION
DATASET
DATASET
Map courtesy of the United States Geological
Survey, Eastern Energy Resources Team
7
Instrumented Oil Field
Detect and track changes in data during
production Invert data for reservoir
properties Detect and track reservoir
changes Assimilate data reservoir properties
into the evolving reservoir model Use simulation
and optimization to guide future production

• Numerical simulations
• Generate production data and in situ geophysical
  data
• Compute intensive
• Large collections of output datasets
• Integration of software data
• Subsurface Modeling
• Geophysical data modeling
• Non linear optimization and uncertainty
• Data assimilation and visualization
• Distributed databases of reservoir and
  geophysical data
• Storage and computing resources at multiple
  institutions
• Dynamic data driven simulations

8
4-D Seismic
Production Simulation via Reservoir Modeling
Monitor Production by acquiring Time Lapse
Observations of Seismic Data
Revise Knowledge of Reservoir Model via Imaging
and Inversion of Seismic Data
Modify Production Strategy using an Optimization
Criteria
9
Client
Client
. . . .
Grid-enabled Production Optimization and
Reservoir Management
Web Portal, Steering, Collaboration Tools
Production Forecasting
Oil Reservoir Simulation Tools
Reservoir Characteristics
Seismic Data Simulation Tools
Visualization Tools
Reservoir Performance
Data Analysis
Data Analysis
Data Analysis
Grid-based Data Management and Manipulation Tools
Reservoir Monitoring Field Measurements
Datasets from Simulations and Field Measurements
Datasets from Simulations and Field Measurements
10
Integrated Parallel Accurate Reservoir Simulator

• Multiblock
• Multi-model Couplings
• adjacent domains mortars (use MACE) or dual
  formulation
• multinumerics (different numerical algorithms)
• multiphysics for coupling of multiphase flow
  models
• overlapping domains (operator-splitting)
• geomechanics
• transport-chemistry (port from ParSSim) flow
  (any IPARS flow model)
• mix of the above techniques

11
Integrated Parallel Accurate Reservoir Simulator
Boundary conditions
Black Oil 2
Solvers
Keyword Input
Compositional
Wells
Table Lookup
Black Oil
Memory Management
Parallel Processing
WWW interface
Two-Phase
Platforms
Visualization
Two-Phase Impes
Multiblock
Single Phase Implicit
Geomechanics
Air-Water
Single Phase Sequential
Multiphysics
DG Impes
Geomechanics Flow
Flow Reactive Transport
12
Reservoir Simulation to Seismic

• Reservoir characterization use reservoir
  simulation data for 4D seismic analysis of
  parameters to Biot-Gassmann equations (measure
  relative changes in porosities, saturations and
  pressures)

Porosity diff
Gas saturation
Oil saturation
13
Exploration and Visualization of Oil Reservoir
Simulation Data
14
MACE Adaptive Computational Engine

• Data management for distributed adaptive
  applications
• Semantically Specialized DSM
• Application-oriented programming abstractions
• Adaptive mortar grids, adaptive grid blocks, grid
  functions,
• Regular access semantics to dynamic,
  heterogeneous, and physically distributed data
  objects
• Encapsulate distribution, communications,
  interaction
• Automatic and transparent scheduling, load
  balancing
• Coupling/interactions between multiple physics,
  models, structures, scales
• Distributed Shared Objects
• virtual Hierarchical Distributed Dynamic Array
• Hierarchical Index-Space Extendible Hashing
  Heterogeneous objects
• Multifaceted objects
• Integration of computation data visualization
  interaction
• Adaptive Run-time Management
• Proactive/reactive, application and system
  sensitive management
• Algorithms, partitioners, load-balancing,
  communications, etc.
• Policy-based automated adaptations

IPARS Multi-block Oil Reservoir Simulation (M.
Peszynska, UT)
15
MACE Block-Mortar Interactions
16
Seismic Modeling of Reservoirs
17
Seismic Modeling of a Reservoir
Array 1 (top)
Source
Array 3 (vertical)
Array 2 (bottom)
18
Surface Hydrophone Array
19
Bottom Array
20
Seismic Traces
Survey
Line
Sp (or CDP) source position
Array
Traces
Receiver group receiver group position
Component
21
DataCutterSoftware Support for Data Driven
Applications

• Component Framework for Combined Task/Data
  Parallelism
• Core Services
• Indexing Service Multilevel hierarchical indexes
  based on R-tree indexing method.
• Filtering Service Distributed C component
  framework
• User defines sequence of pipelined components
  (filters and filter groups)
• Pleasingly Parallel
• Generalized Reduction
• User directive tells preprocessor/runtime system
  to generate and instantiate copies of filters
• Stream based communication
• Multiple filter groups can be active
  simultaneously
• Flow control between transparent filter copies
• Replicated individual filters
• Transparent single stream illusion

22

• DISCOVER A Grid Computational Collaboratory
  enabling seamless and secure access to and
  interactions between users, applications,
  services, data and resources
• P2P Grid Middleware (PAWN, DISCOVER-COG)
• Peer services (discovery, routing, message
  publication, notification, event), context-aware
  access control, p2p deductive engines.
• Autonomic and Interactive Components (DIOS,
  AUTOMATE)
• Components encapsulate sensors, actuators,
  policies and rules. Distributed control network
  connects sensors, actuators and interaction
  agents.
• P2P deductive shell, control network, rules and
  polices enable autonomic composition,
  configuration, interaction, protection,
  optimization and adaptation.
• Collaborative Portals
• Pervasive (secure) access, monitoring,
  interaction and control

23
Interactive Oil Reservoir Simulations with IPARS
DISCOVER

• Launch jobs

• Remote visualization

• Interactively steer jobs

24
Grid Collaboration
Distributive Collaboration
25
Results

• Optimizing Production
• Economic Model Results
• Optimal Well Placement
• Data Assimilation

26
Optimizing Production

• Max economic value of production (well position,
  rates)
• Min bypassed oil (infill drilling)
• History matching (find reservoir parameters so
  that
• simulation data matches production data)
• ...........

27
Analysis of Oil Reservoir Simulation
DataPrototype Implementation

• Evaluate geologic uncertainty and production
  strategies simultaneously
• Multiple realizations of multiple geostatistical
  models
• Multiple production strategies (number, location
  of wells)
• Dataset Size 5TB
• 500 simulations, selected from several
  Geostatistics models and well patterns
• Each simulation is 10GB
• 2,000 time steps, 65K grid elements, 8 scalars
  3 vectors 17 variables
• Stored at
• SDSC HPSS and 30TB Storage Area Network System
• UMD 9TB disks on 50 nodes PIII-650, 768MB,
  Switched Ethernet
• OSU 7.2TB disks on 24 nodes PIII-900, 512MB,
  Switched Ethernet
• Data Analysis
• Economic model assessment
• Bypassed oil regions
• Representative Realization Selection for more
  simulations

28
(No Transcript)
29
Economic Model

• Economic assessment
• Net Present Value (NPV)
• Return on Investment (ROI)
• Sweep Efficiency (SE)
• Queries
• return R-WP for given GM that has NPV gt average
• return R-WP for all GM which has max NPV

• Economic model uses
• well rates (time series data)
• cost and price (e.g., oil) parameters

30
Bypassed Oil (BPO)

• User selects
• a subset of datasets (D) and time steps (TS),
• thresholds for oil saturation value (Tos) and oil
  velocity (Tov)
• minimum number of connected grid cells (Tcc).
• Query Find all the datasets in D that have
  bypassed oil pockets with at least Tcc grid
  cells.
• A cell (C) is a potential bypassed oil cell if
  Cos gt Tos and Cov lt Tov.
• Algorithm for bypassed oil
• Find all potential bypassed oil cells in a
  dataset at time step Ti
• Run connected components analysis Discard
  pockets with fewer cells than Tcc Mark a cell if
  in a bypassed oil pocket, 0 otherwise.
• Perform an AND operation on all cells over all
  time steps in TS.
• Perform the operations in Step 2 on the result,
  and report back to client.

31
Representative Realization

• Select the simulation/realization that has values
  closest to a user-defined criteria.
• analyze that simulation or use its initial
  conditions for further simulation studies.
• Find the dataset among a set of datasets
• values of oil concentration, water pressure, and
  gas pressure are closest to the average of these
  values across the set of datasets
• User selects
• A set of datasets (D) and a set of time steps
  (T1,T2,,TN).
• Query Find the dataset that is closest to the
  average.
• min S(all grid points) Oc Ocavg Wp
  Wpavg Gp Gpavg

32
Optimal Well Placement

• Optimization algorithm use VFSA (Very Fast
  Simulated Annealing)
• requires function evaluation only, no gradients
• IPARS delivers
• fast-forward model (guess-gtobjective function
  value)
• post-processing
• Formulate a parameter space
• well position and pressure (y,z,P)
• Formulate an objective function
• maximize economic value Eval(y,z,P)(T)
• Normalize the objective function NEval(y,z,P) so
  that

33
Results
Contours of NEval(y,z,500)(10)
Pressure contours 3 wells, 2D profile
permeability
Requires NYxNZ (450) evaluations. Minimum appears
here.
VFSA solution walk found after 20 (81)
evaluations
34
Autonomic Oil Reservoir Optimization on the Grid
35
Data Assimilation
36
Kalman Filter

• K Kalman gain
• d measured data
• HuF models theoretical estimate of what is
  observed

37
Effect of Kalman Filter
38
Effect of Mesh Refinement
39
Summary

• Energy and environmental modeling are important
  to health and well-being of the US as well as the
  economy
• Better knowledge of reservoirs and aquifers
  during production and cleanup will result in
  better engineering decisions for optimizing and
  modifying goals while maintaining safe operating
  conditions in environmentally complex and
  possibly hazardous conditions
• Cost-effective production of oil gas and
  contaminant remediation require real-time
  monitoring and integration of advanced
  technologies a multidisciplinary effort is
  essential
• Computational results for several important
  applications were presented