Interactive Mesh Sculpting Using a Haptic Device

1
Interactive Mesh Sculpting Using a Haptic Device
2
Goal

• A method to continuously sculpt or deform a
  planar mesh using haptic interaction is
  implemented. Moreover, this mesh has an image of
  the final model mapped on.

3
Approach

• Image mapped on a planar mesh
• Phantom Omni Basic Haptic Device
• NURBS surface is used
• Different tools to improve sculpting

4
Force Feedback Foundation

• It allows the user to interact with it by
  applying and receiving forces.
• Interacts with graphics applications
• Can make objects feel sticky, soft, rough,
  viscous

5
Previous Work
6
Haptic Device

• Avoiding interpenetration between virtual objects
• Allowing the user to paint in virtual
  environments
• Teleoperation of objects
• Touching textures

7
Mesh Sculpting

• Free Form Deformation
• Control points define the vertices of a
  parallelepiped
• Using control points can be cumbersome
• Subdivision solids
• Each subdivided solid has damping, mass and
  stiffness
• Virtual clay

8
Characteristics of the Project
9
Haptic Device

• Movement in 3 dimensions
• Mouse is constrained to 2 dimensions
• Limited range of motion
• Force rendering

10
NURBS

• Provide advantages over other surfaces
• Using OpenGL implementation
• Automatic texture mapping
• Knot array affects continuity of the surface

11
Particle System

• Defined as a set of particles that evolves over
  time
• Control points in NURBS are the particles
• Particles have attributes like position, velocity
  and force

12
Runge-Kutta4

• Method used to solve differencial equations

13
Implementation
14
HLAPI and OpenGL

• High level haptic rendering
• Provides rendering/collision detection and useful
  data structures
• hlBeginFrame and hlEndFrame

15
Data Structures

• Vertex class
• Has attributes like position and velocity
• ParticleSystem
• Contains list of control points
• NURBS
• Draws the mesh

16
Features/Tools

• Sculpting
• Forces are rendered when we touch the surface.
• Sculpting includes both pushing and pulling the
  surface.
• We are using the stylus button to switch between
  push and pull modes.

17
Brush

• Induces force on more than one control point at a
  time

This is the point of contact
The green points are also affected
18
3D Paint

• Painting in the 3D model is reflected on the 2D
  texture

19
Show/Hide Particles

• Showing particles Hiding particles

Closer look at the image
20
Rotations

• It allows us to look the image from different
  angles

21
3D Cursor

• It simulates the shape of a pen

22
Force Feedback

• This is the default mode, but can be disabled.
• If disabled, no forces are rendered, but the mesh
  can still be sculpted.
• Results of this comparison are shown in the
  Results section.

23
Basic Algorithm

• Initialize the application
• Create the mesh and texture map it with the image
• Sculpt the mesh using the haptic device
• Rotate the mesh
• Select a brush or single point sculpting
• Apply forces to the mesh
• The new position for the control point is
  calculated and the mesh is drawn again.
• Repeat step 3 until sculpting is done.

24
Major Issues

• Sense of depth is difficult to implement
• NURBS sampling parameters needs to be adjusted
• Haptic device kicked occasionally while rotating
  the surface

25
Results
26
Comparing the Haptic Device and the Mouse
Mouse Haptic device
No force feedback Points moved one at a time, but can move continuously if implemented User touches the control points Picking control points may be difficult Force feedback exists Points are moved continuously User touches the mesh since the haptic device provides this configuration No need to pick control points
27
Mapping Haptic Device to Graphics Simulation
28
NURBS in OpenGL

• An efficient way to draw NURBS
• Provide different parameters to control the
  surface
• The mesh is 12x12 control points
• Hardware acceleration in the graphics card should
  be disabled

29
Interactive Sculpting

• Initial image Final image

30
Lips Deformation

• Initial image

31
Sculpting the lower lip
32
Side view of the image
33
Common error in the sculpting process
34
Side view of the final mesh
35
Front view of the mesh
36
Comparing Force Feedback and No Force Feedback

• Haptic device as a 3D mouse
• Identifying if we are touching the surface

37
Example
38
Results

• Aesthetic quality of the image and 3D model are
  better when using haptic interaction
• Some curves in the mesh are not steady
• Same results can be achieved without force
  feedback but time will increase.

39
Conclusion
40
Conclusion

• Haptic interaction provides an easier to use and
  faster way to sculpt a mesh.
• We implemented a few of the many tools that will
  enhance the sculpting process.
• Practice is needed to perceive the cursors Z
  location.

41
Conclusion

• Real time interaction is important
• NURBS provided flexibility and important
  properties
• Sense of touch can provide new forms of
  interaction

42
Future Work
43
Future Work

• New editing tools
• E.g. a selection tool
• Adding knots on the fly
• NURBS parameters should be adjusted
• NURBS limitations
• How to model hands, bottles?

44
Future Work

• Different views of the mesh
• Each view has a different angle
• Integration with image editing software
• Attract users attention
• Image recognition
• Identify boundaries and apply rules to the
  control points near the boundary
• 3D Paint