Combating Latency in Haptic Collaborative Virtual Environments

1
Combating Latency in Haptic Collaborative
Virtual Environments
by Chris Gunn, CSIRO

• In Lee
• inism_at_postech.ac.kr
• VR Lab, POSTECH

2
Outline

• Introduction
• Dynamic-Object Instability
• Overcoming Latency
• Implementation Results
• Conclusion
• Reference

3
Introduction

• CSIRO
• Commonwealth Scientific and Industrial Research
  Organization
• It is Australia's national science agency and one
  of the largest in the world
• http//www.ict.csiro.au/

4
Introduction

• Chris Gunn
• http//www.ict.csiro.au/staff/Chris.Gunn/

5
Dynamic-Object Instability

• Haptic feedback loop
• It involves the users hand, arm, muscles and
  voluntary and involuntary reactions as well as
  the robotic force-feedback device, network, and
  software involved in the user interface

6
Dynamic-Object Instability

• Instability sources
• Physical reflection
• Conscious reaction
• Subconscious reaction
• Jitter
• Latency

7
Dynamic-Object Instability

• Force-Reflection Experiment
• Tests were performed on 4 subjects
• Finger tissue
• Hand and finger tissue
• Arm, hand, and finger tissue
• Conscious effort, arm, hand, and finger tissue

8
Dynamic-Object Instability
9
Dynamic-Object Instability
10
Dynamic-Object Instability

• Subconscious reaction
• It seemed that users receiving a regularly
  repeating force pattern are able to control their
  subconscious reactions to the force

11
Dynamic-Object Instability

• Force Reflections in an HCVE
• After a force impulse is delivered to a user from
  a remote system, there is a series of a reflected
  waves passed back to the remote system
• These waves are caused by the elements previously
  mentioned

12
Dynamic-Object Instability

• Jitter
• If date are received in irregular rates, it can
  make the operator to overreact and lead the
  system unstable
• UDP instead TCP can reduce this problem

13
Dynamic-Object Instability

• Latency
• A virtual object grasped by two remote users,
  separated by a latency-affected network, can
  start to vibrate and oscillate when triggered by
  a small force impulse
• Its known that latency of up to 60ms was
  acceptable for some tasks (Matsumoto et al.,
  2000)

14
Overcoming Latency

• Efforts to overcome latency by eliminating
  unintended forces
• Adding a damping factor
• Filtering
• Prediction

15
Overcoming Latency

• Pseudo physics
• Single Physics engine
• Event collection

16
Pseudo Physics - motivation

• Computing full multi-body dynamics may not be
  necessary for surgical simulation
• Body organs are so damped, so that they exhibit
  slow translation in most cases

17
Pseudo Physics

• whereP is new position, P0 is old position,k is
  the pseudo damping factor andF is the input
  force from the ith user
• There is no calculation of acceleration or
  velocity

18
Single Physics Engine - motivation

• If each system calculated its new position
  dependent on its local forces, and then
  transmitted it to the remotes, there could be
  frequently mismatches between the local and
  remote positions
• In this case, some objects can jitter between the
  two positions

19
Single Physics Engine

• One machine was nominated as the physics server
• The server collects all remote users input
  forces as well as its own user input force
• Then, the server calculates the objects new
  position and returns it to the collaborators
• This method also helps to render multiple
  simultaneous interactions with objects

20
Event Collection

• It is necessary to send data describing each
  dynamic objects position and orientation across
  the network, as well as the user forces acting
  upon those objects
• Each client sends user interaction to the server
  and receives dynamic objects new position and
  orientation

21
Event Collection

• A client needs up to three threads to receive
  graphical data, receive and send haptical data
• The server needs up to three threads for each
  client and a thread to handle client-connection
  requests

22
Event Collection

• Data flow

23
Implementation

• Workbench
• A user wears a shutter glass and headset
• A PHANToM 1.5 of Sensable Technology is chosen as
  a haptic device
• To collocate graphical and haptic scenes, a
  display device is specially set up

24
Implementation
25
Implementation

• Application
• A cholecystectomy with an instructor and a remote
  student, both working in the same virtual space
• A user can grasp another users tool
• A user can grasp, move, stretch, and pull the
  bile

26
Implementation
27
Results

• A test was held between Australia and Sweden
• The total distance was about 22,000km, and the
  latency was about 170ms, one-way
• The performance was very nice

28
Conclusion

• In some applications, it is possible to haptic
  collaboration between anywhere of the world!

29
Reference

• C. Gunn, M. Hutchins, and M. Adcock, "Combating
  Latency in Haptic Collaborative Virtual
  Environments," Presence Teleoperators and
  Virtual Environments, vol. 14, pp. 313-328, 2005.