Physically based simulation on GPU using OpenGL and GLSL

1
Physically based simulation on GPU using OpenGL
and GLSL

• Yalmar Ponce Atencio

2
Outline

• Multitexture mapping
• OpenGL fixed functionality vs. GLSL
• Render to texture
• PBuffers vs. The OpenGL Framebuffer Object
  Extension
• Applications (physical simulation)
• Particle systems
• Cloth
• Rigid and deformable objects

3
Outline

• Multitexture mapping
• OpenGL fixed functionality vs. GLSL
• Render to texture
• PBuffers vs. The OpenGL Framebuffer Object
  Extension
• Applications (physical simulation)
• Particle systems
• Cloth
• Rigid and deformable objects

4
Brief history

• Since the OpenGL 1.0 definition (1992)
• However, limited. Just to apply images to the
  surface of an object
• Later, in OpenGL 1.1 (1995) texture objects was
  added
• In OpenGL 1.2 (1997) 3D textures was added
• In OpenGL 1.3 (1999) new hardware capabilities
  was exposed. Allow access to two or more texture
  objects simultaneously
• In OpenGL 1.4 (2003) was added support for depth
  textures and shadows

5
Simple texturing sample
?

• Good example
• http//www.xmission.com/nate/tutors.html

6
Multitexture with fixed functionality

• Need GL_ARB_multitexture extension (now is part
  of OpenGL 2.0 core)
• glActiveTextureARB (GL_TEXTUREn_ARB)
• glMultiTexCoord

7
Combining textures

• First, a class or function for loading image
  files is needed

void DrawFrame(void) ... glTranslatef(0.0,2.0,
0.0) glBindTexture(GL_TEXTURE_2D, texture1)
drawBox(2.0) glTranslatef(0.0,-2.0,0.0)
glBindTexture(GL_TEXTURE_2D, texture2)
drawBox(2.0) ...
8
Combining textures

• Next, setup for using multitexturing

void drawFrame() ... glEnable(GL_TEXTURE_2D)
glActiveTextureARB(GL_TEXTURE0_ARB)
glBindTexture(GL_TEXTURE_2D, texture1)
glTexEnvf (GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_
COMBINE_EXT) glTexEnvf (GL_TEXTURE_ENV,
GL_COMBINE_RGB_EXT, GL_REPLACE)
glActiveTextureARB(GL_TEXTURE1_ARB)
glBindTexture(GL_TEXTURE_2D, texture2)
glTexEnvf (GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_
COMBINE_EXT) glTexEnvf (GL_TEXTURE_ENV,
GL_COMBINE_RGB_EXT, GL_INCR) drawBox(2.0)
...
void drawBox(float size) // Texture Coordinate
(0,0) is top left. glBegin(GL_QUADS)
glMultiTexCoord2f(GL_TEXTURE0, 0.0, 1.0)
glMultiTexCoord2f(GL_TEXTURE1, 0.0, 1.0)
glVertex3f(0.0, 0.0, 0.0) glMultiTexCoord2f(GL
_TEXTURE0, 0.0, 0.0) glMultiTexCoord2f(GL_TEXTU
RE1, 0.0, 0.0) glVertex3f(0.0, size, 0.0)
glMultiTexCoord2f(GL_TEXTURE0, 1.0, 0.0)
glMultiTexCoord2f(GL_TEXTURE1, 1.0, 0.0)
glVertex3f(size, size, 0.0)
glMultiTexCoord2f(GL_TEXTURE0, 1.0, 1.0)
glMultiTexCoord2f(GL_TEXTURE1, 1.0, 1.0)
glVertex3f(size, 0.0, 0.0) glEnd()
9
Bump mapping example


10
Bump mapping example

• Configuring a given texture unit
• glActiveTexture(GL_TEXTUREn)glBindTexture(GL_TE
  XTURE_2D, texName)glTexImage2D(GL_TEXTURE_2D,
  )glTexParameterfv(GL_TEXTURE_2D,
  )glTexEnvfv(GL_TEXTURE_ENV,
  )glTexGenfv(GL_S, )glMatrixMode(GL_TEXTURE)
  glLoadIdentity()
• Setting texture coordinates for a vertex
• glMultiTexCoord4f(GL_TEXTURE1,s1, t1, r1, q1)
  glMultiTexCoord3f(GL_TEXTURE2,s2, t2,
  r2)glMultiTexCoord2f(GL_TEXTURE3,s3,
  t3)glMultiTexCoord1f(GL_TEXTURE4,s4)glVertex3f
  (x, y, z)

11
Bump mapping exampleMapping code
void display() ... //Set up texture
environment to do (tex0 dot tex1)color
glActiveTextureARB(GL_TEXTURE0_ARB)
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,
GL_COMBINE_ARB) glTexEnvi(GL_TEXTURE_ENV,
GL_SOURCE0_RGB_ARB, GL_TEXTURE)
glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB,
GL_REPLACE) glActiveTextureARB(GL_TEXTURE1_AR
B) glTexEnvi(GL_TEXTURE_ENV,
GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB)
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB,
GL_TEXTURE) glTexEnvi(GL_TEXTURE_ENV,
GL_COMBINE_RGB_ARB, GL_DOT3_RGB_ARB)
glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB,
GL_PREVIOUS_ARB) drawTorus(2.0, 0.5) ...
12
Bump mapping exampleResults

• References
• http//www.paulsprojects.net/opengl/
• http//www.opengl.org/resources/tutorials/sig99/sh
  ading99/course_slides/

13
Why must be used GLSL?

• Programmability allow to exploits much more the
  texture mapping capabilities
• With programmable shaders, APPs can read and use
  the texture values in a way makes sense
• Textures can also be used to store intermediate
  results
• Lookup tables
• normals
• Factors
• Visibility information
• Etc.

14
Combining texturesGLSL version

• A class or function that load vertex and/or
  fragment programs is needed
• Changing the DrawFrame routine

... GLSLKernel myShader ...
myShader.fragment_source("fragment_program")
myShader.vertex_source("vertex_program")
myShader.install() ...
void DrawFrame(void) ... myShader.use()
myShader.setUniform("texture0", 0) // Texture
Unit 0 myShader.setUniform("texture1", 1) //
Texture Unit 1 drawBox(2.0)
myShader.use(false) ...
15
Combining texturesGLSL version

• and the drawBox routine too

void drawBox(float size) // Texture Coordinate
(0,0) is top left. glBegin(GL_QUADS)
glTexCoord2f(0.0, 1.0) glVertex3f(0.0, 0.0,
0.0) glTexCoord2f(0.0, 0.0)
glVertex3f(0.0, size, 0.0)
glTexCoord2f(1.0, 0.0) glVertex3f(size, size,
0.0) glTexCoord2f(1.0, 1.0)
glVertex3f(size, 0.0, 0.0) glEnd()

16
Combining textures GLSL version

• The vertex program
• The fragment program

void main(void) gl_TexCoord0
gl_TexCoord0 gl_Position gl_ModelViewProject
ionMatrix gl_Vertex
uniform sampler2D texture0, texture1 void main
(void) vec4 texel0 texture2D(texture0,
vec2(gl_TexCoord0)) vec4 texel1
texture2D(texture1, vec2(gl_TexCoord0))
gl_FragColor 0.5(texel0 texel1)
17
Using multitexture with several shaders

• References
• OpenGL Shading Language book
• http//www.clockworkcoders.com/oglsl/
• http//www.lighthouse3d.com/opengl/glsl/

18
Conclusions

• GLSL or another GPU programming language allow
  exploit much more texture mapping capabilities
• Several shaders can be used in an APP, anyone to
  a specific APP feature
• Walls with bump mapping
• Water effects
• Fire effects
• Complex material effects (roughness, reflection,
  refraction, glass, etc.)

19
Outline

• Multitexture mapping
• OpenGL Fixed functionality vs. GLSL
• Render to texture
• PBuffers vs. The OpenGL Framebuffer Object
  Extension
• Applications (physical simulation)
• Particle systems
• Cloth
• Rigid and deformable objects

20
Later, PBuffers

• Pixel buffers
• Designed for off-screen rendering
• Similar to windows, but non-visible
• Window system specific extension
• Select from an enumerated list of available pixel
  formats using
• ChoosePixelFormat()
• DescribePixelFormat()
• ...

21
Framebuffer Object

• Only requires a single OpenGL context Only
• switching between Framebuffers is faster than
  switching between PBuffers (wglMakeCurrent)
• No need for complicated pixel format selection
• Format of Framebuffer is determined by texture or
  Renderbuffer format
• Puts burden of finding compatible formats on
  developer
• More similar to Direct3D render target model
• Makes porting code easier
• Renderbuffer images and texture images can be
  shared among Framebuffers
• e.g. share depth buffers between color targets
• saves memory

22
Framebuffer Object

• OpenGL Framebuffer is a collection of logical
  buffers
• Color, depth, stencil, accumulation
• Framebuffer object extension provides a new
  mechanism for rendering to destinations other
  than those provided by the window system
• Window system independent
• Destinations known as Framebuffer attachable
  images. Can be
• Off-screen buffers (Renderbuffers)
• Textures

23
Framebuffer Object

• Framebuffer-attachable image
• 2D array of pixels that can be attached to a
  framebuffer
• Texture images and renderbuffer images are
  examples.
• Attachment point
• State that references a framebuffer-attachable
  image.
• One each for color, depth and stencil buffer of a
  framebuffer.
• Attach
• The act of connecting one object to another.
  Similar to bind.
• Framebuffer attachment completeness

24
Framebuffer Object

• Introduces two new OpenGL objects
• Framebuffers and Renderbuffers
• Framebuffer (FBO)
• Collection of framebuffer attachable images (e.g.
  color, depth, stencil)
• Plus state defining where output of GL rendering
  is directed
• Equivalent to window system
• When a framebuffer object is bound its attached
  images are the source and destination for
  fragment operations
• Color and depth textures
• Supports multiple color attachments for MRT
• Depth or stencil renderbuffers

25
Framebuffer Object arquitecture
TEXTURE OBJECTS

GL_COLOR_ATTACHMENT0
TEXTURE IMAGE
GL_COLOR_ATTACHMENTn
DEPTH ATTACHMENT
RENDER BUFFER OBJECTS
STENCIL ATTACHMENT
RENDERBUFFER IMAGE
OTHER STATES
26
Framebuffer Object API

• Creating a FBO (similar to create a texture)
• void GenFramebuffersEXT(sizei n, uint
  framebuffers)
• Delete a FBO
• void DeleteFramebuffersEXT(sizei n, const
  uint framebuffers)
• Check if a given identifier is a FBO
• boolean IsFramebufferEXT(uint framebuffer)
• Binding a FBO
• void BindFramebufferEXT(enum target, uint
  framebuffer)
• Check status of the FBO
• enum CheckFramebufferStatusEXT(enum target)

27
Framebuffer Object API

• Attaches image from a texture object to one the
  logical buffers of the current bound FBO
• void FramebufferTexture1DEXT(enum target,
  enum attachment,
  enum textarget, uint texture,
  int level)
• void FramebufferTexture2DEXT(enum target,
  enum attachment, enum textarget,
  uint texture, int level)
• void FramebufferTexture3DEXT(enum target,
  enum attachment, enum
  textarget, uint texture, int level,
  int zoffset)
• Automatically generate mipmaps for texture images
  attached to target
• void GenerateMipmapEXT(enum target)

28
Binding a FBO

• void BindFramebufferEXT(enum target, uint
  framebuffer)
• target must be FRAMEBUFFER_EXT
• framebuffer is FBO identifier
• All GL operations occur on attached images
• If (framebuffer 0), GL operations operate on
  window system provided framebuffer (its default)

29
Attaching textures to a FBO

• void FramebufferTexturenDEXT(enum target, enum
  attachment, enum
  textarget, uint texture, int level)
• target must be FRAMEBUFFER_EXT
• attachment Is one of
• GL_COLOR_ATTACHMENTn_EXT
• DEPTH_ATTACHMENT_EXT
• STENCIL_ATTACHMENT_EXT
• textarget must be one of
• GL_TEXTURE_2D
• GL_TEXTURE_RECTANGLE_ARB
• GL_TEXTURE_CUBE_MAP_...
• level is the mipmap level of the texture to
  attach
• texture is the texture object to attach
• If (texture0), the image is detached from the
  framebuffer

30
Framebuffer completeness

• Framebuffer is complete if all attachments are
  consistent
• Texture formats make sense for attachment points
  i.e., dont try and attach a depth texture to a
  color attachment
• All attached images must have the same width and
  height
• All images attached to COLOR_ATTACHMENTn_EXT must
  have same format
• If not complete, glBegin will generate error
  INVALID_FRAMEBUFFER_OPERATION

31
Checking Framebuffer Status

• enum CheckFramebufferStatusEXT(enum target)
• Should always be called after setting up FBOs
• Returns enum indicating why FBO is incomplete
• FRAMEBUFFER_COMPLETE COMPLETEFRAMEBUFFER_INCOMPL
  ETE_ATTACHMENTFRAMEBUFFER_INCOMPLETE_MISSING_ATTA
  CHMENTFRAMEBUFFER_INCOMPLETE_DUPLICATE_ATTACHMENT
  FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXTFRAMEBUFFER
  _INCOMPLETE_FORMATS_EXTFRAMEBUFFER_INCOMPLETE_DRA
  W_BUFFER_EXTFRAMEBUFFER_INCOMPLETE_READ_BUFFER_EX
  TFRAMEBUFFER_UNSUPPORTED FRAMEBUFFER_STATUS_ERRO
  R
• If result is FRAMEBUFFER_UNSUPPORTED,
  application should try different format
  combinations until one succeeds

32
FBO usage

• FBO allows several ways of switching between
  rendering destinations
• In order of increasing performance
• Multiple FBOs
• Create a separate FBO for each texture you want
  to render to???
• Single FBO, multiple texture attachments
• Textures should have same format and dimensions
• Use FramebufferTexture() to switch between
  textures
• Single FBO, multiple texture attachments
• Attach textures to different color attachments
• Use glDrawBuffer() to switch rendering to
  different color attachments

33
FBO Performance Tips

• Dont create and destroy FBO every frame
• Try to avoid modifying textures used as rendering
  destinations using TexImage, CopyTexImage, etc.
• More references and examples
• http//www.gpgpu.org/developer/

34
Outline

• Multitexture mapping
• OpenGL Fixed functionality vs. GLSL
• Render to texture
• PBuffers vs. The OpenGL Framebuffer Object
  Extension
• Applications (physical simulation)
• Particle systems
• Cloth
• Rigid and deformable objects

35
Brief overview

• Use textures to represent physical features (e.g.
  positions, velocities, forces)
• Fragment shader calculates new values and render
  this results back to textures
• Each rendering pass draws a single quad,
  calculating values to next time step on the
  simulation
• Actually, graphic processors supports textures
  with NPT (non-power-of-two) and signed float
  values in each (RGBA) color channel
• GL_TEXTURE_RECTANGLE_ARB
• Float precision of fragment programs allows GPU
  physic simulation match CPU accuracy
• New fragment programming model (longer programs,
  flexible dependent texture reads) allows much
  more interesting simulations

36
Basic ExampleCloth simulation

• Can be used the Verlet integrator
• Jakobsen, GDC 2001
• Avoids storing explicit velocity
• x 2x x a.?t²x x
• Not always accurate, but stable!
• In a GPU version, current and previous positions
  should be stored in 2 RGB float textures
• Then swap current and previous textures for each
  time step

37
Cloth simulationThe algorithm

• GPU
• Two passes (two shaders)
• Perform integration (move particles)
• Satisfy constraints
• Distance constraints between particles
• Floor collision constraints
• Collision constraints with other objects
  (spheres)
• CPU
• XYZ vertices ? RGB texture pixels (read pixels)
• Compute normals and render final mesh

38
Cloth simulationDiagram
old positions
TEX0
VERLET INTEGRATOR FRAGMENT SHADER
SATISFY CONSTRAINTS FRAGMENT SHADER
curr positions
next positions
next positions
TEX2
curr positions
TEX1
Adjacency
TEX3
39
Cloth simulationIntegration pass code

• uniform sampler2DRect oldp_texuniform
  sampler2DRect currp_texvoid Integrate(inout
  vec3 x, vec3 oldx, vec3 a, float timestep2)
  x 2x - oldx atimestep2 void
  main() float timestep 0.002 vec3
  gravity vec3(0.0, -9.8, 0.0) vec2 uv
  gl_TexCoord0.st // get current and
  previous position vec3 oldx
  texture2DRect(oldp_tex, uv).rgb vec3 x
  texture2DRect(currp_tex, uv).rgb // move
  the particle Integrate(x, oldx, gravity,
  timesteptimestep) // satisfy world
  constraints x clamp(x, worldMin,
  worldMax) SphereConstraint(x, spherePos,
  1.0) gl_FragColor vec4(x, 1.0)

40
Cloth simulationSatisfy constraints code

• // constrain a particle to be a fixed distance
  from another particlevec3 DistanceConstraint(vec3
  x, vec3 x2, float restlength, float stiffness)
  vec3 delta x2 - x float deltalength
  length(delta) float diff (deltalength -
  restlength) / deltalength return
  deltastiffnessdiff// as above, but using
  sqrt approximation sqrt(a) r ((a- rr) /
  2r), if a rrvec3 DistanceConstraint2(vec3
  x, vec3 x2, float restlength, float stiffness)
  vec3 delta x2 - x float deltalength
  dot(delta, delta) deltalength restlength
  ((deltalength - restlengthrestlength) / (2.0
  restlength)) float diff (deltalength -
  restlength) / deltalength return
  deltastiffnessdiff// constrain particle to
  be outside volume of a spherevoid
  SphereConstraint(inout vec3 x, vec3 center, float
  r) vec3 delta x - center float dist
  length(delta) if (dist lt r) x
  center delta(r / dist)

41
Cloth simulationConstraints pass code

• uniform vec2 ms // mesh sizeuniform float
  constraintDistuniform vec3 spherePosuniform
  sampler2DRect x_texuniform vec3
  worldMinuniform vec3 worldMaxvoid main()
  const float stiffness 0.2 // this should
  really be 0.5 vec2 uv gl_TexCoord0.st
  // get current position vec3 x
  texture2DRect(x_tex, uv).rgb // get
  positions of neighbouring particles vec3 x1
  texture2DRect(x_tex, uv vec2(1.0, 0.0)).rgb
  vec3 x2 texture2DRect(x_tex, uv vec2(-1.0,
  0.0)).rgb vec3 x3 texture2DRect(x_tex, uv
  vec2(0.0, 1.0)).rgb vec3 x4
  texture2DRect(x_tex, uv vec2(0.0, -1.0)).rgb
  // apply distance constraints // this
  could be done more efficiently with separate
  passes for the edge cases vec3 dx
  vec3(0.0) if (uv.x lt ms.x) dx
  DistanceConstraint(x, x1, constraintDist,
  stiffness) if (uv.x gt 0.5) dx dx
  DistanceConstraint(x, x2, constraintDist,
  stiffness) if (uv.y lt ms.y) dx dx
  DistanceConstraint(x, x3, constraintDist,
  stiffness) if (uv.y gt 0.5) dx dx
  DistanceConstraint(x, x4, constraintDist,
  stiffness) x x dx
  gl_FragColor vec4(x, 1.0)

42
Cloth simulationA screenshot
128x128 mesh
256x256 mesh
43
Cloth simulationUsed textures
Previous positions
Current positions
44
More complex exampleRigid and deformable bodies

• As cloth modeling
• Vertices ? particles
• Current and previous positions are mapped on
  textures
• Edges ? contraints
• A rigid tetrahedron
• Four vertices
• Six constraints

v3
v1
v2
v0
Texture0
Texture1
ContraintsTexture
45
Rigid tetrahedron simulation
46
ExtensionsRigid cube
Texture0
Texture1
ContraintsTexture
47
Rigid cubesScreenshot
48
Improving the performanceRender to vertex array

• Enables interpretation of floating point textures
  as geometry
• Possible on NVIDIA hardware using the
  NV_pixel_data_range (PDR) extension
• Allocate vertex array in video memory (VAR)
• Setup PDR to point to same video memory
• Do glReadPixels from float buffer to PDR memory
• Render vertex array
• Happens entirely on card, no CPU intervention

49
Videos

• Cloth with 64x64 mesh on CPU
• Cloth with 64x64 mesh on GPU
• Cloth with 128x128 mesh on CPU
• Cloth with 128x128 mesh on GPU
• 400 cubes on GPU

50
ResultsP4 3.6Ghz NVidia GeForce 6800
51
ResultsP4 3.6Ghz NVidia GeForce 6800
52
Conclusions

• GPUs simulates and renders up to 1024x1024
  particles in real-time with effects and simple
  collisions
• High resolution cloths without self-collision can
  be simulated and rendered in real-time
• 256x256 mesh with 195585 constraints in 25 fps
  aprox.
• 128x128 mesh with 48641 constraints in 210 fps
  aprox.
• 64x64 mesh with 12033 constraints in 590 fps
  aprox.
• Other simulations using rigid/deformable objects
  has been successfully conducted
• 400 rigid cubes (6400 constraints) in 570 fps.
• More complex objects would be simulated too
• Tetrahedral models

53
Future work

• Collision detection and response
• Recently has been reported GPU image based
  approaches
• Occlusion queries to fast rejection of
  non-collide pairs can be used

54
References

• Jakobsen, Thomas Advanced Character Physics. In
  GDC Proceedings 2001
• Latta, Lutz Building a Million Particle System.
  Graphics Hardware 2004 conference GDC 2004
  Session