Tracking in VR

1
Tracking in VR

• Agenda
• What is tracking in general?
• Where is it used?
• Why do we need it in VR?
• Keywords in tracking
• Different tracking technologies
• Pros and cons
• Points to take with you

2
What do you think?

• What is tracking in general?
• Where is it used?
• Why do we need it in VR?

3
What do I think tracking is?

• Finding the trajectory of an object over time
• That is, a continuous time-curve in a space
  spanned by the different parameters we are
  tracking (e.g., 3 position 3 rotation)
• Where is it used?
• Long military tradition E.g., radar
• GPS, RFID, ..
• Why do we need it in VR?
• Track the eyes (head) in order to update the
  graphics
• Track body (parts) in order to immersive the user
  in VR and/or in order to recognize the users
  actions (walking, turning, pointing, etc)

4
Keywords in tracking

• Occlusion, line-of-sight
• Invasiveness, amount of mountings, wires, limited
  movements, size, weight
• Cost
• Degrees of Freedom (DoF), 3 positions 3
  rotations
• Speed, measurements per seconds, sampling rate
• Number of objects, Sensor connections (parallel
  vs sequential) Robustness towards noise
• Working area
• Calibration, required time
• Relative vs. Absolute measurements, reference
  frame, world coordinate system
• Hybrid systems
• PRECISION VERSUS ACCURACY, repeatability
• Latency
• Simultaneity assumption, asynchrony vs. synchrony
  measurements
• Outside-in vs. Inside-out

5
Sampling rate
6
Keywords in tracking

• Occlusion, line-of-sight
• Invasiveness, amount of mountings, wires, limited
  movements, size, weight
• Cost
• Degrees of Freedom (DoF), 3 positions 3
  rotations
• Speed, measures per seconds, sampling rate
• Number of objects, Sensor connections (parallel
  vs sequential) Robustness towards noise
• Working area
• Calibration, required time
• Relative vs. Absolute measurements, reference
  frame, world coordinate system
• Hybrid systems
• PRECISION VERSUS ACCURACY, repeatability
• Latency
• Simultaneity assumption, asynchrony vs. synchrony
  measurements
• Outside-in vs. Inside-out

7
PRECISION VERSUS ACCURACY
Accuracy refers to how closely a measured value
agrees with the correct value.Precision refers
to how closely individual measurements agree with
each other. (repeatability)
8
Keywords in tracking

• Occlusion, line-of-sight
• Invasiveness, amount of mountings, wires, limited
  movements, size, weight
• Cost
• Degrees of Freedom (DoF), 3 positions 3
  rotations
• Speed, measures per seconds, sampling rate
• Number of objects, Sensor connections (parallel
  vs sequential) Robustness towards noise
• Working area
• Calibration, required time
• Relative vs. Absolute measurements, reference
  frame, world coordinate system
• Hybrid systems
• PRECISION VERSUS ACCURACY, repeatability
• Latency
• Simultaneity assumption, asynchrony vs. synchrony
  measurements
• Outside-in vs. Inside-out

9
Tracking Latency
10
Keywords in tracking

• Occlusion, line-of-sight
• Invasiveness, amount of mountings, wires, limited
  movements, size, weight
• Cost
• Degrees of Freedom (DoF), 3 positions 3
  rotations
• Speed, measures per seconds, sampling rate
• Number of objects, Sensor connections (parallel
  vs sequential) Robustness towards noise
• Working area
• Calibration, required time
• Relative vs. Absolute measurements, reference
  frame, world coordinate system
• Hybrid systems
• PRECISION VERSUS ACCURACY, repeatability
• Latency
• Simultaneity assumption, asynchrony vs. synchrony
  measurements
• Outside-in vs. Inside-out

11
Simultaneity assumption
12
Error induced by simultaneity assumption?
13
Error induced by simultaneity assumption?
14
Keywords in tracking

• Occlusion, line-of-sight
• Invasiveness, amount of mountings, wires, limited
  movements, size, weight
• Cost
• Degrees of Freedom (DoF), 3 positions 3
  rotations
• Speed, measures per seconds, sampling rate
• Number of objects, Sensor connections (parallel
  vs sequential) Robustness towards noise
• Working area
• Calibration, required time
• Relative vs. Absolute measurements, reference
  frame, world coordinate system
• Hybrid systems
• PRECISION VERSUS ACCURACY, repeatability
• Latency
• Simultaneity assumption, asynchrony vs. synchrony
  measurements
• Outside-in vs. Inside-out

15
Outside-in vs. Inside-out
Is the user wearing the receiver or
transmitter ?
16
A magical device tracker-on-a-chip. (Welch and
Foxlin, 2002)

• This ToC (tracker-on-a-chip) would be all of the
  following
• Tiny - size of an 8-pin DIP (dual in-line
  package) or a transistor
• Self-contained - no other parts to be mounted in
  environment or on user
• Complete - tracking 6 degrees-of-freedom
  (position orientation)
• Accurate - resolution lt1mm in position and 0.1
  in orientation
• Fast 1kHz, latency lt1ms, no matter how many
  ToCs deployed
• Immune to occlusions - needing no clear line of
  sight
• Robust - resisting performance degradation from
  light, sound, heat, magnetic fields, radio waves,
  other ToCs in the environment
• Tenacious - tracking its target no matter how far
  or fast it goes
• Wireless - running for three years on a coin-size
  battery
• Cheap - costing 1 each in quantity.

17
Different technologies

• Accelerometer
• Gyroscopes
• Mechanical
• Acoustic
• Electromagnetic
• Optical

18
Accelerometer

• Principle with strings (show)
• Forces
• In praxis
• a small prism is twisted when moved. These
  movements gt signals
• Pros Small, cheap, fast
• Cons Only measures acceleration gt cannot
  operate by it self

19
Making Accelerometers work

• An advanced accelerometer
• Calculates the position based on the acceleration
  (show)
• Pros Small, cheap, fast
• Cons 3DoF (position), relative measurement gt
  accumulation of the error
• P(t) P(t-1)P_calculated

20
Gyroscopes

• Principle
• Operates as a compas except uses the earths
  gravity
• Example, stabilizing the canon on a tank
• Pros Small, absolute
• Cons 3DoF (rotation)

21
InertiaCube by InterSense

• Combine accelerometers and gyro
• 3DoF

22
Mechanical
23
Mechanical

• Principle
• Measures the physical movement
• For example the change in voltage as a function
  of resistance
• Pros Can be very precise
• Cons Very invasive, very expansive

24
Acoustic

• Sound/sonic system
• Principle
• A transmitter sends out a sound wave
• A receiver measures the time of flight (tof)
• Known speed of wave (sow) sowdist/tof

25
Acoustic

• Pros Can be precise
• Cons Expensive, 3DoF (position) , need to have a
  clear line of sight

26
Acoustic

• We have two InterSense systems in the CVMT lab

Combined with InertiaCube gt Position rotation!
27
Electromagnetic

• Principle
• A transmitter generates three perpendicular
  electromagnetic fields (one at the time)
• Each receiver can measure its pose (6DoF) based
  on the electromagnetic field

28
Electromagnetic

• We have several systems in the Lab.
  visualization arenas

29
Electromagnetic

• Pros 6DoF, independent on line-of-flight
• Cons Expensive, limited range, sequential
  connection gt slow, sensitive to other magnetic
  fields (e.g., PC-monitor) and metal objects The
  reason the CAVE is made of wood

30
Optical tracking

• Classification of optical tracking sources (what
  are we tracking?)

31
(No Transcript)
32
Optical tracking

• Classification of optical tracking sources (what
  are we tracking?)

33
(No Transcript)
34
(No Transcript)
35
Spherical Marker

• Viewpoint invariant
• High reflectance in illumination direction

36
Infrared Illumination

• Fast segmentation by adaptive threshold
• Robust

Visually Opaque IR pass filter
37
3D Head/Hand Position from Stereo

• 3D positions
• Triangulation
• Epipolar constraint
• Camera Selection
• Very accurate

38
(No Transcript)
39
MLD (Moving light displays)
Pros Very precise, as many measurements as
required, high speed Cons The correspondence
problem, calibration, occlusion
40
Hybrid systems in the lab

• Optical for finger electromagnetic for (HMD)
• Mechanical for details (glove for fingers)
  Gyroscope or magnetic for orientation/position of
  hand (interface)
• Acoustic for position gyroscope for orientation
  (HMD tracking)

41
General pros and cons

• Accelerometer fast, relative, inprecise, 3DoF
• Gyroscopes fast, precise, 3DoF
• Mechanical Invasive, precise, track multiple
  objects
• Acoustic line-of-sight, 3DoF
• Electromagnetic 6DoF, sensitive
• Optical precise, track multiple objects, 3DoF

42
Points to take with you

• The keywords provide an inside to the important
  aspects of tracking technologies
• Tracking is Finding the trajectory of an object
  over time
• Many different technologies exist
• Different pros and cons
• Look at the requirements for a particular
  application before choosing a technology!

43
Accelerometer