Automatic Generation of Dynamics Models

1
Automatic Generation of Dynamics Models

• John W. Ratcliff
• Most Worshipful Senior Programmer Simutronics
  Corporation

2
Introduction

• Work for this presentation was supported by Ageia
  Technologies.
• This is an open source project to facilitate the
  rapid production of physics content from raw
  source graphics data
• Much assistance provided by Matthias Mueller,
  Pierre Terdiman, Adam Moravansky, Billy Zelsnack
  and Dilip Sequeira

3
The Code Suppositorywww.codesuppository.blogspot.
com

• A Place Where I Insert My Code into the Anus of
  the Internet John W. Ratcliff, Flipcode, January
  12, 2000

Yes, I know what that word means John W.
Ratcliff in response to an email received on
January 13, 2000.
4
The Problem

• The rapid acceptance of pervasive physics in
  games has created new authoring and asset
  requirements.
• There is a general lack of tools to facilitate
  the rapid creation of dynamics assets.
• Many games send their raw graphics assets to the
  physics engine which is highly inefficient and
  often causes significant problems.

5
Where should Physics Content be Authored?

• Physics models are usually derived from a final
  production art asset.
• Iteration on physics assets is primarily a
  design problem rather than an art issue.
• Ideally editing of dynamics data assets should
  occur directly in the game engine/editor for
  rapid iteration cycles to test the behavior and
  interaction of the models in real-time.

6
Existing Tools to Author Physics Content

• Feeling Software www.feelingsoftware.com
• PhysX Plugin for 3D Studio Max (free open source)
• Nima for Maya (Unsupported version Free)
• Lead developer Christian Laforte
• Blender 3D www.blender.org (Free open source)
• Lead developer Ton Roosendaal
• Bullet Physics Library www.continuousphysics.com
  (Free open source)
• Developed by Erwin Coumans
• CreateDynamics www.codesuppository.blogspot.com
  (Free open source)
• Developed by John W. Ratcliff
• Unreal Editor www.epicgames.com
• Proprietary tools.
• Physics authoring happens in the game editor
• Uses some of the CreateDynamics toolkit for
  generating compound objects.
• Lead developer on the physics tools James Golding
• Scythe Physics Editor www.physicseditor.com
• Supports ODE, Newton, PhysX
• Free and Commercial vesions available
• Lead developer Chris Hunt

7
Existing Standardized File Formats for Physics
Content

• COLLADA 1.4.1 www.collada.org
• Supports basic rigid body physics markup in an
  XML format.
• NxuStream www.ageia.com
• Provided as part of the free PhysX SDK.
• Source library.
• Exact reflective object model of every component
  in the SDK, including rigid body, constraints,
  all shapes, including heightfields, cloth,
  softbodies, fluids, collision filters, energy
  fields, and much more.
• XML, Binary, and supports COLLADA 1.4.1 physics
  only specification.
• Scythe www.physicseditor.com

8
Multiple Collision Models Per Asset

• For any given art asset there are usually at
  least three, if not more, physics
  representations.
• Raycast model A static triangle mesh that is
  identical to the original graphics model for
  precise line of sight evaluation as well as
  certain graphics support such as decal
  generation.
• Static collision model An often dramatically
  simplified version which corresponds to what the
  player character would collide with (rarely
  should be the raycast version.) May be a
  simplified mesh, a convex hull, or a compound
  shape.
• Dynamic collision model(s) One or more versions
  to be used when the object is dynamic. Should
  never be a triangle mesh. Typically a single
  convex hull or compound object.

9
Components of a Physics Asset

• Rigid Body properties
• Collision Shapes
• Triangle Mesh
• Box
• Capsule
• Convex Hull
• Sphere
• HeightField
• Constraints
• Cloth properties and topology
• Soft Body properties and topology
• Fluid and fluid emitter properties
• Energy Fields

10
Typical Rigid Body Properties

• World Transform (global pose)
• List of collision shapes
• Collision group and/or other flags
• Dynamics Properties
• Linear Velocity and Angular Velocity
• Mass or Density
• Mass Local Pose and Mass Space Inertia Tensor
• Engine specific components
• Solver Iterations
• Sleep Threshold
• Maximum Angular Velocity
• Etc.

11
Collision Shape Properties

• Local Transform (relative to parent rigid body)
• Material Index
• Collision filtering information
• Contact report flags
• Shape data
• Box dimensions
• Sphere radius
• Capsule height and radius
• Convex hull faces
• Triangle mesh triangles
• Heightfield sample data
• Plane equation

12
Programmable ConstraintCustomizable on all 6
degrees of freedom angular and linearPhysX SDK
implementation by Matthias Mueller-Fischer

• A constraint that can be configured on three
  angular and three linear degrees of freedom.
  Each degree of freedom can be fixed, free, or
  limited and can be configured to support drive,
  motors, and springs.
• Advantages
• A single model can be configured to behave in any
  of the most common custom joint configurations.
• Allows for a much simpler and easier to maintain
  code path.
• Provides new types of constraints not typically
  available with most engines.
• Predictable behavior in all configurations.
• Orthogonal data definition.
• Disadvantages
• Initially can be more difficult to configure.
  This can be improved by providing helper setup
  routines for common constraint types.

13
Authoring Collision Shapes

• For simple objects, when a single shape is
  sufficient, collision fitting is not difficult

Single Collision Shape with Oriented Bounding Box
approximation
Single Collision Shape Convex Hull approximation
14
Compound ShapesApproximate Convex Decomposition

• Published work
• Approximate Convex Decomposition
• Texas AM University
• Algorithms Applications Group
• Jyh-Ming Lien and Nancy M. Amato
• (parasol.tamu.edu/groups/amatogroup/research/app-c
  d)
• Variational, meaningful shape decomposition
• University of British Columbia
• Vladislav Krayevoy and Alla Scheffer
• (www.cs.ubc.ca/vlady/Papers/Segmentation.pdf)

15
Convex DecompositionAlgorithm by John W. Ratcliff

• Inspired by the work of Jyh-Ming Lien and Nancy
  M. Amato at Texas AM. However, this brute force
  implementation has little in common with their
  technique.
• Accepts open meshes
• Not real time
• Available as a plug-in or open source library
• Simple interface

16
The Algorithm

• A recursive routine is passed, as input, the
  source triangle mesh
• The first step is to compute the volume of
  concavity
• Build a convex hull around the mesh.
• Project each triangle on the original mesh onto
  the hull.
• Compute the volume of the mesh that results from
  the projected triangle onto the convex hull skin.
• The sum of the volume of all projected triangles
  to the convex hull surrounding it becomes the
  volume of concavity
• Compute the percentage volume of concavity by
  dividing it by the volume of the hull around the
  original source mesh.
• If the percentage volume of concavity is below a
  user supplied threshold then accept this mesh as
  sufficiently convex.
• By measuring against the total volume this allows
  highly concave pieces to be ignored if they are
  quite small and unimportant relative to the
  overall size of the object.

17
The Algorithm cont.

• Compute the best fit oriented bounding box
• If the mesh is concave then compute the best fit
  oriented bounding box around the mesh.
• Compute a split plane along the long axis of the
  OBB
• Split all of the input triangles against the
  plane producing two output meshes
• Each triangle is tested to see which side of the
  plane it lies on.
• If the triangle straddles the plane it is split
  and one triangle piece is sent to the top mesh
  and the other sent to the bottom mesh.
• The edge of each split triangle is added to an
  edge list.

18
The Algorithm cont.

• Using the edges that lie along the split plane
  compute a delaunay triangulation.
• Project the points onto the plane so the
  operation can be performed in 2d
• Must support multiple polygons
• Must support holes by testing polygons inside
  other polygons (example splitting a glass in
  half)
• See Triangle A Two-Dimensional Quality Mesh
  Generator and Delaunay Triangulator by Jonathan
  Richard Shewchuk http//www.cs.cmu.edu/quake/tri
  angle.html
• Project the polygon points back into 3d and weld
  them into the mesh producing a whole piece.
• Without this operation deep recursion produces
  hollow objects and does not converge to an
  optimal solution set.

19
The Algorithm cont.

• Pass the top and bottom split meshes back into
  the recursive routine.
• Wash, rinse, and repeat until there are no
  concave pieces left or at the maximum recursion
  depth provided by the user.
• This results in an oriented bounding volume tree
  based on the original source mesh.
• The output hulls are over described since an
  optimal split was not performed.
• To correct for this a cleanup phase is run where
  hulls are merged back together again if they are
  largely convex afterwards.
• In short, what the process has done is created
  too many hulls but a straightforward cleanup pass
  can easily stitch them back together again.
  Simply compute the volume of two hulls separate
  then compute the volume of the two hulls
  combined. If the volume is within a percentage
  threshold provided by the user (called the merge
  threshold) then these two hulls are combined back
  into one.

20
The Algorithm cont.

• Attempt to merge hulls by largest volume first.
• At each phase of the merge process select the
  largest hull first and then attempt to merge it
  in order of the largest volume to the smallest.
• The end result is that even though the mesh was
  chopped up more than was necessary, the
  post-process cleanup phase ends up producing
  results as if the most optimal split had been
  performed in the first place. Behold the power
  of brute force, sometimes a blunt instrument
  beats out the most advanced mathematics.

21
Create DynamicsConvex Decomposition User
Interface
22
Approximate Convex DecompositionExamples using
various fitting rules
23
Approximate Convex Decomposition Examples
24
VideoA high poly count helix mesh simulated as a
rigid body composed of only 15 convex hull shapes
25
Automatic Rag Doll GenerationDirectly from
Skeletal Deformed Mesh

• Creating Ragdoll models can be very time
  consuming. Artists spend a great deal of time
  rigging up a skeletal system for an art asset to
  begin with. There should not be a need to spend
  an equal amount of time producing a physics
  representation of the same.
• Embedded within a standard skeletal deformed mesh
  is sufficient information to auto-generate a
  reasonable approximate ragdoll model.

26
Automatic Rag Doll GenerationStep 1 Skeleton
Pruning

• The skeleton used for a ragdoll physics
  simulation is typically far simpler than what is
  used for graphics.
• The graphics version may include many details
  such as fingers, toes, facial bones and marker
  bones for game play elements. None of these
  bones are desirable in the physics representation
• Before attempting to generate constraints for all
  of the bones in the skeleton the skeleton needs
  to be pruned and a new skeleton created which is
  usable for a real-time physics simulation.

27
Automatic Rag Doll GenerationStep 1 Skeleton
Pruning

• Traverse the skeleton and examine the triangles
  which have significant weightings to each bone.
• Bones which are not influenced by any geometry
  are marked for deletion.
• For the rest of the bones compute the volume of
  the convex hull around the triangles weighted to
  it.
• Compare the volume of the geometry associated
  with the volume of the whole. If the volume is
  below a user supplied percentage threshold this
  bone should be marked for removal. This will
  leave only bones associated with larger
  components of the source mesh in a humanoid
  character this would correspond to head, arms,
  legs, and torso, but not eyes, fingers, or toes.

28
Automatic Rag Doll GenerationStep 1 Skeleton
Pruning

• Now a new skeleton must be produced that leaves
  only the bones which were marked for removal.
• Leaf node bones marked for deletion may simply be
  removed.
• Bones marked for removal that are not leaf nodes
  must have their child/parent relationship patched
  up so the skeleton will still be intact. The
  parent should keep track of the now dead child
  bones it has inherited.
• Now that a reduced skeleton has been created the
  deformed mesh geometry has to be revised so the
  bone indices point to the correct locations in
  the new skeleton.
• For each bone referenced in the source mesh
  re-assign it to the corresponding bone in the new
  skeleton for the output mesh.
• If a vertex refers to a bone that has been
  deleted then attach it to the parent bone that
  the original bone was collapsed into.

29
Automatic Rag Doll GenerationStep 2 Generating
Collision Shapes

• Walk the reduced skeleton and collect the
  triangles associated with each bone.
• On a per-bone basis run the CreateDynamics shape
  fitting tool and approximate the shape as a box,
  sphere, capsule, or convex hull based on user
  input selection.
• Using a rules based system, different shape
  fitting techniques can be applied to the skeleton
  using wildcards and inheritance.

30
Automatic Rag Doll GenerationStep 3 Generating
Constraints

• Constraints are created based on the transforms
  of the reduced skeleton.
• Joint types and limits are selected from user
  input using the same rules system for deciding
  shape fitting.
• Though not currently implemented it would be
  feasible to infer joint types and limits by
  interpreting a reference animation.
• Joint limits and types are supported by Maya and
  might be able to be passed as part of the export
  process.

31
Automatic Rag Doll GenerationStep 4 Generating
Collision Filters

• To prevent the ragdoll model from constantly
  colliding with itself collision filters are
  needed to prevent this artifact.
• Walk the skeleton and compute the skeletal
  distance between bones. Bones which are near
  each other along the skeleton should have
  collisions disabled between their bodies.
• For more accurate collision modeling (such as
  convex hulls) the filtering need not be too deep.
  For cruder collision models such as capsules or
  boxes, filtering to avoid the model colliding
  with itself will need to be more aggressive.
• Though not currently implemented, it might be
  best to decide where collision filters are needed
  by detecting contacts while the skeletal model is
  in its rest pose.

32
Simplified Auto Generated RagDoll
33
Detailed Auto Generated Ragdoll
34
Video Auto Generated RagDoll Models Simulated
with the PhysX SDK
35
Auto Generated Constraints

• The previous example showed how to create a
  ragdoll from a skeletal deformed mesh rigged up
  by an artist.
• If you dont have an already rigged up skeletal
  deformed mesh but still want to get a fast
  ragdoll or cheap illusion of soft body then a
  skeletal deformed mesh can be automatically
  generated.

36
Auto Generated Constraints

• First generate an oriented bounding volume system
  for the raw source mesh.
• This is easy to implement because all we have to
  do is disable the merge portion of the convex
  decomposition algorithm.
• By removing the merge option the system is much
  more balanced and will simulate more
  realistically.

37
Auto Generated Constraints
Auto-Generated Oriented Bounding Volume System
using Approximate Convex Decomposition without
Merge
Original Raw Source Mesh No Skeleton No Bone
Weightings
38
Auto Generated Constraints

• Auto-generating the constraints uses a recursive
  algorithm that begins with finding the largest
  volume hull in the decomposed object. The
  largest hull is used as the root node for the
  skeleton.
• From this hull locate all hulls which touch it.
• Touching is determined by finding two triangles
  which have roughly the same vector normal
  pointing in opposite directions.
• If the two triangles lie along the same plane,
  then they are projected into 2d along that plane
  and a 2d triangle-triangle intersection test is
  performed.
• If these two triangles intersect then they are
  considered touching and are added to an
  intersection bounding volume.
• If the two hulls were found to be touching then
  the median intersection point of those touching
  surfaces becomes the anchor point for the
  constraint.
• This same process is repeated for every hull in
  the object, each time starting with the last
  hulls that were connected.
• In the end each hull will have a single
  constraint at the location where it touches a
  neighbor hull.

39
Auto Generated Skinned Meshes

• Once the auto-generated constrained rigid body
  system has been saved there are now run-time
  considerations.
• A skeletal deformed mesh has to be created from
  the original raw source mesh. This could be done
  as a pre-process step but is more powerful to do
  on the fly at run time.
• Converting the raw mesh to a skeletal deformed
  mesh requires that each vertex be assigned bone
  indices and bone weightings.
• For each vertex in the source mesh, compute the
  four nearest bones (rigid bodies) and the
  distance of those bones from the vertex.
• Compute the weighting of each bone based on the
  ratio of the distance of that bone relative to
  the sum total of the distances of all four. An
  additional weighting bias can be applied if
  desired (Example 4x bone1, 2x bone2, 1x, bone 3
  and 4.)

40
Video Auto-Generated constrained system mapped
to an auto-generated skinned mesh
41
Auto-Generated Pre-Fractured Objects

• The same technique used to produce skeletal
  meshes from raw mesh data can be applied for
  pre-fractured objects
• Do not disable the merge in this case.
• Constraints are generated against fewer shapes at
  more natural break boundaries.
• The constraints are rigged up to be fixed joints
  with a particular breaking force based on game
  design.
• The difficult part is producing fracture artwork.
  This is a hard problem, but not unsolvable.
  Currently the CreateDynamics toolkit cannot
  produce pre-fracture graphics, only pre-fracture
  physics.

42
Video
43
Cloth Authoring

• Cloth is simulated based on a source triangle
  mesh.
• Authoring is straightforward as the physics
  representation is simply the source triangle mesh
  with duplicate vertex positions removed.
• Additional data associated with cloth are
  numerous physics properties describing how the
  simulation should behave.
• Attachment points must be captured where needed.

44
Cloth at Run-Time

• The graphics representation of the cloth usually
  does not exactly match up to the physics
  representation due to material assignments.
• A mapping table must exist between the indices in
  the physics mesh to the indices in the graphics
  mesh.
• Additional challenges arise from dealing with
  torn cloth which creates new triangle indices,
  and those indices must be remapped to the
  graphics representation on the fly.
• Double sided cloth can be visualized using a
  shader that renders the cloth twice flipping the
  normals and culling direction on the back side.
• Cloth is typically authored in a rest pose so a
  simulation may need to be run a number of frames
  before rendering it initially to avoid watching
  the cloth fall into place.

45
Cloth Simulation Video
46
Authoring Fluids

• Fluids are represented as a set of simulation
  properties and emitters.
• 3D fluids have numerous and often complex
  physical properties that influence the behavior
  of the simulation.
• Emitters can be attached to rigid body actors.
• Numerous emitter properties can be edited as
  well.
• A real-time preview tool to rapidly iterate on
  fluid properties is essential to achieve good
  results.

47
Rendering Fluids

• The challenges in turning a set of 3d points into
  a compelling visual illusion of fluid surfaces
  would take a whole talk by itself.
• Think outside the box. Just because you are
  using a physics simulation called fluids
  doesnt mean you have to visualize it as a fluid
  surface.
• Smoke
• Sparks
• Micro-Debris
• Fog
• Energy fields
• And whatever your artists imagination can come up
  with.

48
Common Fluid Visualization Methods

• Sprites
• Each fluid particle is rendered using a camera
  facing billboard sprite.
• Excellent opportunity to use hardware point
  sprites.
• Multi-Pass techniques on each sprite can produce
  interesting layered effects.
• Full Screen multipass effects can produce the
  illusion of refraction and reflection by
  rendering sprites as normals which are then
  Gaussian blurred in an off-screen buffer.
• Surface Mesh
• Algorithm by Matthias Mueller
• Performs mesh operations in 2d and then
  back-projects the results producing a highly
  efficient 3d mesh.

49
Video Screen Space Surfaces
50
Video Comparison Fluid Visualization Techniques
51
Video
52
Soft Body Authoring

• Soft Bodies are simulated as a tetrahedral
  volumetric mesh.
• Similar to the cloth simulation presented in the
  paper Position Based Dynamics authors Matthias
  Müller, Bruno Heidelberger, Marcus Hennix, and
  John Ratcliff. (http//graphics.ethz.ch/mattmuel/
  publications/posBasedDyn.pdf)
• Rather than solving for stretching along vertex
  edges instead solve for conservation of volume.
• Highly dependent upon the quality of the input
  tetrahedral mesh.

53
Soft Body AuthoringMathias Mueller-Fischer,
PhDhttp//graphics.ethz.ch/mattmuel/Ageia
Technologies

• Begin with arbitrary 3d mesh
• Does not need to be a closed mesh.
• Generate an isosurface for the input mesh.
• Generate an iso-surface of the distance field to
  the triangle mesh and triangulate it via marching
  cubes.
• Perform a quadric mesh optimization on the
  isosurface
• This is a mesh simplification technique which
  does not affect the morphology of large scale
  features. This will produce fewer tetrahdrons
  without affecting the visual quality of the
  simulation and, therefore, simulate faster.
• Using the optimized isosurface mesh perform the
  delaunay tetrahedralization to produce the set of
  tetrahedrons for simulation. Adding randomly
  distributed vertices inside the isosurface will
  reduce the tetrahedral size.
• Finally allow the user the option of editing
  individual tetrahedron based on game design needs.

54
Soft Body Visualization

• Use free form deformation to produce the
  real-time graphics mesh synchronized to the
  tetrahedral physics simulation.
• For each vertex in the graphics mesh compute the
  nearest tetrahedron to it.
• Compute the set of barycentric coordinates that
  map the four vertices of the tetrahedron to the
  single vertex on the graphics mesh.
• These barycentric coordinates can be pre-computed
  and stored as part of the graphics mesh data, or
  can be built on the fly when the mesh is
  initially loaded.
• During rendering, for each vertex in the graphics
  mesh deform it by the projection of the
  tetrahedron that maps to that vertex using the
  current positions and the previously computed
  barycentric values.
• This deformation is difficult to perfom in a
  vertex shader unless, perhaps under DirectX 10.
  Even then, the operation is so fast in software
  that it difficult to imagine significant
  performance gains.

55
Video
56
Video
57
The Production Pipeline

• For each source asset in the product generate a
  default INI file.
• The default file should correspond to the minimum
  expected set of physics models required for any
  asset. Typically this would include a raycast
  version, a static collision version, and a
  dynamic collision version.
• Provide a user interface within the game editor
  that allows designers to modify all of the
  default settings and add additional physical
  representations.
• Save the INI file as a permanent asset that
  corresponds to the source graphics model. The
  physics assets only need to be regenerated if the
  morphology of the graphics asset changes or the
  user makes explicit editing changes.

58
The Production Pipeline

• During game editing or prior to producing final
  production assets.
• Auto-generate the various physics versions for
  each asset by using the CreateDynamics library.
  This will produce an ASCII version of the physics
  representation as either COLLADA or NxuStream.
• Since the COLLADA specification cannot represent
  many of the types of physics presented here
  cloth, soft body, fluids, heightfields, it is
  recommended to use NxuStream.
• If you do not use the PhysX SDK or COLLADA it is
  easy to modify the source code in CreateDynamics
  to output the physics data in a format that is of
  your choosing. (Accumulate.cpp)
• Store the various ASCII versions of the physics
  asset in your repository for day to day use.

59
The Production Pipeline

• At run-time, and for final production content,
  convert the XML version of the art assets into a
  binary cooked form.
• The XML version can be converted into a binary
  representation very easily and quickly.
• The binary versions are not backwards compatible
  and should never be used as a persistent storage
  format for the physics data.
• It is best to create the binary versions on the
  fly on the users machine and store it in a local
  repository.
• During level load time, the pre-cooked binary
  assets are in a format that can be rapidly
  submitted to the physics engine. Not only is
  parsing of XML and other ASCII data avoided, but
  all mesh preprocessing is bypassed as well.
• Extremely fast level load times can be achieved
  using this technique. Massive game levels can be
  loaded and submitted to a physics engine in under
  a second.

60
The Future of CreateDynamics

• This is a hobby project and while it is being
  used in a production environment no guarantees
  can be made as to a schedule of features, bug
  fixes, or support.
• Open to discussion for collaboration or
  sponsorship of the future development of these
  tools.
• Special thanks to Matthias Muller, Pierre
  Terdiman, Adam Moravansky, Billy Zelsnack, Dilip
  Sequeira, James Dolan, James Golding, David
  Whatley, Simon Schirm, Erwin Coumans, Christian
  Laforte, Stan Melax, Worshipful Brother Lou
  Castle, Jesus Christ, Mahatma Ghandi, etc.

61
Questions???

• Contact
• John W. Ratcliff
• Email jratcliff_at_infiniplex.net
• Coding website www.codesuppository.com
• Skype jratcliff63367
• Skype Phone US (636)-486-4040
• Karma Contribution Site www.amillionpixels.us