Computation Fluid Dynamics

1
Computation Fluid Dynamics Modern Computers
Tacoma Narrows Bridge Case Study

• Farzin Shakib
• ACUSIM Software, Inc.
• 2003 SGI Technical Users Conference
• June 13, 2003

2
Content

• Brief Company Product Overview
• Tacoma Narrows Bridge Upgrade Project
• Effectiveness of CFD in a large scale project
• Need for ever larger and more complex solutions
• Design for High Performance Computers
• Conclusions

3
Company Overview

• Founded in 1994
• First product sold in 1997
• Who has continued in production and steadily
  increased usage
• Marketed through distributor channels and direct
  sales
• Success stories
• Visteon employs AcuSolve for
• climate control (ICCE) and
• underhood cooling (UPV)
• analyses of production cars
• ORCA A fully integrated
• mixing package for chemical
• and pharmaceutical companies

4
CFD Product AcuSolve

• A powerful general-purpose finite element
  incompressible flow solver
• AcuSolves differentiation
• Robustness Most problems solved on the first
  attempt
• Speed Coupled solver on distributed parallel
  machines
• Accuracy Highly accurate in space and time
  while
• globally and locally conservative
• Features Rich set of functionality continuously
  growing
• An ideal enabling technology for
• integrated engineering and
• scientific applications

5
Tacoma Narrows Bridge Upgrade Project
Advanced Simulation Bechtel National, Inc.
6
CFD Project

• Use CFD to predict time-varying loads and moments
  on new bridge caissons (East and West) for Flood
  and Ebb flows
• Show CFD can predict known drag coefficients
• Show CFD can match experiment scale model
• Use CFD to predict variation in loads
• due to changing flow directions
• for different caisson drafts
• for full scale geometry

7
Validation flow around standard objects
8
Cylinder Test Case - Re 1.15 e08

• Time history of computed Drag

9
Cylinder Test Case - Performance

• Mesh 500K Tet-elements, 100K nodes
• 2-processors 900MHz Itanium II
• Steady-state RANS Solution (SA turbulence model)
• 50 time steps in 30 minutes
• CD 0.5
• Transient Hybrid RANS/LES Solution (DES)
• 1000 time steps in 8 hours
• CD 0.7
• Same solution with 3.5M Tet-element mesh

10
Square Cylinder Test Case - Re 115000

• Time history of computed Drag

CFD fine mesh Cd 1.98, Sh 0.12
NACA at Re100000 Cd2.0, Parker Welsh Sh
0.12
11
Computational Domain / Mesh
Transient simulation Hybrid RANS/LES model
3M Tet/Prism-elements ICEM-CFD Tetra Module
12
East Side Caisson 4.6 m/s Flood
13
Force Time History East Side 4.6 m/s Flood
14
East/West caissons 3.6 m/s Ebb
Flow streamlines around caisson
East Side
West Side
15
East/West caissons 3.6 m/s Ebb
Velocity profile at 120 ft depth
East Side
West Side
16
Validation Against Experiment

• HR Wallingford (in UK) conducted experiment
• Re 105 Model-Scale
• CFD Solutions at
• Re 108 Full-Scale
• Re 105 Model-Scale
• Comparison
• CFD Re105 compared with experiment
• CFD Re105 / Re108 comparison is used to
• Scale up HRW measurements to Re108

17
Model Scale vs. Full Scale Y Force
18
Frequency Spectrum
19
Observations

• Excellent agreement between CFD and Experiment on
  large scale measures
• Total Drag and Lift
• Drag and Lift RMS (total energy)
• Primary shedding frequency
• CFD solution is too coarse to resolve fine scale
  frequencies

20
Fine Mesh

• Mesh 23M Tet-elements 5M nodes
• 8x finer than previous meshes
• Hybrid RANS/LES transient solution
• Requires fast parallel machine
• 32-processor SGI Altix (900 MHz Itanium II)
• 750 time steps
• 30 hours
• 8.5 GBytes of memory

21
Fine Mesh Solution
22
Fine Regular Mesh Solution Comparison
23
Demand on CFD

• Users are placing ever larger demands on
  performance of CFD solvers
• AcuSolve responds through
• Accurate numerical technology
• Galerkin/Least-Squares Finite Element technology
• Fast/robust linear and nonlinear solution
  algorithms
• Coupled pressure/velocity iterative solution
  algorithms
• Design for High Performance Computers
• Single processor performance
• Parallel performance

24
Single-Processor Performance

• Fast element formation
• Block elements in self similar groups
• Arrange memory with stride 1 block data
• Optimize memory layout for cache access
• Fast linear solution
• Access LHS matrices sequentially
• Reorder equations based on proximity
• Avoid excessive indirections
• Hybrid programming
• C for overall control of program
• Fortran for numerical computation

25
Parallel Processing

• Designed from beginning for coarse grain parallel
  machines
• Host-less architecture
• Domain decomposition distributes elements and
  nodes to different processors
• Transparent to the user
• The number of processors may be changed at
  restart
• MPI for distributed-memory machines
• MPI and/or OpenMP for SMP machines
• All algorithms work seamlessly on parallel
  machines

26
Conclusions

• AcuSolve accurately computed transient turbulent
  flows around caisson and pier.
• Solution was used to design anchoring system of
  new span of the Tacoma Narrows Bridge.
• Solution agreed well with experiment on all
  primary features.
• Fine scale features required finer meshes, which
  required faster and larger machines.
• This trend will continue in the foreseeable
  future.