OnLine Locomotion Synthesis for Virtual Humans

1
On-Line Locomotion Synthesis for Virtual Humans
Ph.D. thesis defense

• Pascal Glardon, Ph.D. candidate
• EPFL VRLab

2
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

3
1. Introduction

• Why virtual human animation?
• Movies and video games
• Locomotion and jump

4
1. Introduction

• Why virtual human animation?
• Movies and video games
• Locomotion and jump
• How to animate?
• Key-frame
• Motion capture

5
1. Introduction

• General objectives
• Versatile (high-level parameters)
• Proactive
• Motion realism
• On-line
• Genericity

6
1. Introduction

• Approach

7
1. Introduction

• Approach

8
1. Introduction

• Approach

9
1. Introduction

• Approach

10
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

11
2. Related work

• Hand-driven method keyframe
• Model-driven method kinematics, physics
• Data-driven method motion capture

12
2. Related work

• Data-driven method

Park02
Rose98
13
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

14
3. Motion Modeling

• Approach

15
3. Motion Modeling

• Data acquisition and processing
• Optical motion capture
• Locomotion (treadmill) / Jump
• Parameter variation
• Personification
• Type of locomotion
• Speed / jump length
• Split in cycles (motion unit)
• Joint angle space (26 joints, 40 DOFs)

Data
16
3. Motion Modeling

• PCA algorithm
• Each dimension most variance

Data
PCA
2 PC
1 PC
17
3. Motion Modeling

• PCA algorithm
• Each dimension most variance
• Dimension reduction (real-time)

Data
PCA
2 PC
1 PC
18
3. Motion Modeling

• PCA algorithm
• Each dimension most variance
• Dimension reduction (real-time)
• PCA on motion data
• Motion matrix M
• Column normalized motion unit ?i

Data
PCA
19
3. Motion Modeling

• PCA algorithm
• Each dimension most variance
• Dimension reduction (real-time)
• PCA on motion data
• Motion matrix M
• Analysis to draw laws for motion modeling

Data
PCA
20
3. Motion Modeling

• Hierarchical PCA
• Separate high-level parametersfor extrapolation

Data
PCA
Rose98Park02
Our method
21
3. Motion Modeling

• Hierarchical PCA
• Separate high-level parametersfor extrapolation
• Efficient motion generation

Data
PCA
22
3. Motion Modeling

• Hierarchical PCA
• Separate high-level parametersfor extrapolation
• Efficient motion generation
• Analysis in smaller dimensions

Data
PCA
23
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA
24
3. Motion Modeling
25
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
26
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1
27
3. Motion Modeling
28
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1

walk
run
29
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1

walk
run
Sub-PCA level 2
30
3. Motion Modeling
31
3. Motion Modeling
32
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1

walk
run
Sub-PCA level 2

speed
33
3. Motion Modeling
34
3. Motion Modeling
35
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1

walk
run
Sub-PCA level 2

speed
speed
36
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1

walk
run
Sub-PCA level 2
Speed

speed
speed
37
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA

subject1
subject2
Sub-PCA level 1
Locomotionweight

walk
run
Sub-PCA level 2
Speed

speed
speed
38
3. Motion Modeling

• Hierarchical PCA

All Data
Main PCA
Personificationweight

subject1
subject2
Sub-PCA level 1
Locomotionweight

walk
run
Sub-PCA level 2
Speed

speed
speed
39
3. Motion Modeling

• Un-normalization
• Space leg length

Data
PCA
Generation
40
3. Motion Modeling

• Un-normalization
• Space leg length
• Time original duration

Data
PCA
Generation
41
3. Motion Modeling

• Un-normalization
• Space leg length
• Time original duration
• Frequency function

Data
PCA
Generation
42
3. Motion Modeling

• Un-normalization
• Space leg length
• Time original duration
• Frequency function
• Phase

Data
PCA
Generation
43
3. Motion Modeling
RHS
RHS
event
phase
44
3. Motion Modeling
RHS
RHS
event
phase
45
3. Motion Modeling

• Un-normalization
• Space leg length
• Time original duration
• Frequency function
• Phase

Data
PCA
Generation
46
3. Motion Modeling
47
3. Motion Modeling

• Results
• Real-time
• Entire motion

48
3. Motion Modeling

• Results
• Real-time
• Entire motion
• Limitations
• Personification parameter
• Class of movements

49
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

50
4. Motion Correction

• Approach

51
4. Motion Correction

• Problem
• Foot sliding (or skating)
• Treadmill
• Interpolation

52
4. Motion Correction

• Problem
• Foot sliding (or skating)
• Treadmill
• Interpolation
• Foot position

53
4. Motion Correction

• Problem
• Foot sliding (or skating)
• Treadmill
• Interpolation
• Foot position
• Angular speed parameter

54
4. Motion Correction

• Correct footplant

Right foot
RHS
RTS
RHO
RTO
55
4. Motion Correction

• Footplant detection
• Threshold Men04, Lee02
• Foot position and speed
• Dependent on motion parameters

56
4. Motion Correction

• Footplant detection
• Threshold Men04, Lee02
• Foot position and speed
• Dependent on motion parameters
• Our approach only one threshold (position)
• Rough approximation begin/end frame

57
4. Motion Correction
58
4. Motion Correction

• Footplant detection
• Threshold Men04, Lee02
• Foot position and speed
• Dependent on motion parameters
• Our approach only one threshold (position)
• Rough approximation begin/end frame
• Adaptive position threshold

59
4. Motion Correction
60
4. Motion Correction
61
4. Motion Correction
62
4. Motion Correction
63
4. Motion Correction
64
4. Motion Correction

• Footplant detection
• Threshold Men04, Lee02
• Foot position and speed
• Dependent on motion parameters
• Our approach only one threshold (position)
• Rough approximation begin/end frame
• Adaptive position threshold
• On-line Anticipation

65
4. Motion Correction
66
4. Motion Correction

• Footplant enforcement
• Analytic IK Kovar02, Park02

67
4. Motion Correction

• Footplant enforcement
• Analytic IK Kovar02, Park02
• Numerical IK with priorities Baerlocher04,
  Callennec04
• 1 Ankle
• 2 Toe

68
4. Motion Correction

• Footplant enforcement
• Analytic IK Kovar02, Park02
• Numerical IK with priorities Baerlocher04,
  Callennec04
• 1 Ankle
• 2 Toe
• Anticipation forease-in phase

69
4. Motion Correction

• Footplant enforcement
• Yellowcurrent posture
• Greenanticipated posture with IK

pos
E0(t)
E0(t1)
time
t0
t1
70
4. Motion Correction

• Footplant enforcement
• Yellowcurrent posture
• Greenanticipated posture with IK

pos
E0(t)
E0(t1)
Ec
time
t0
t1
71
4. Motion Correction

• Footplant enforcement
• Yellowcurrent posture
• Greenanticipated posture with IK

pos
E0(t)
E0(t1)
E (t)
Ec
time
t0
t1
72
4. Motion Correction
73
4. Motion Correction

• Results
• Smooth correction
• Straight curved motion

74
4. Motion Correction

• Results
• Smooth correction
• Straight curved motion
• Limitations
• IK per frame
• Only for slight corrections

75
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

76
5. Adaptive Motion Control

• Approach

77
5. Adaptive Motion Control

• Automatic on-line motion transition
• Locomotion / Jump

Parameterized jump
Parameterized locomotion
Walk / Run
Jump
78
5. Adaptive Motion Control

• Automatic on-line motion transition
• Locomotion / Jump
• Time and duration identical footplant

RTO
RHS
Walk / Run
Jump
LTO
LHS
79
5. Adaptive Motion Control
jump
loco
loco
80
5. Adaptive Motion Control

• Automatic on-line motion transition
• Locomotion / Jump
• Time and duration identical footplant
• Dynamic coherence type, run-up speed

81
5. Adaptive Motion Control

• Motion capture observation (gt200 jumps)

82
5. Adaptive Motion Control

• Motion capture observation (gt200 jumps)

83
5. Adaptive Motion Control

• Automatic on-line motion transition
• Locomotion / Jump
• Time and duration identical footplant
• Dynamic coherence type, run-up speed

84
5. Adaptive Motion Control

• Speed variation
• Linear variable distance during adaptation

adaptation
85
5. Adaptive Motion Control

• Speed variation
• Linear variable distance during adaptation
• Other model ensure in on-line correct take-off
  foot position

86
5. Adaptive Motion Control

• Approach
• Quadratic speed profile
• Duration

87
5. Adaptive Motion Control

• Approach
• Quadratic speed profile
• Duration
• Parameters
• Current speed
• Final speed
• Distance
• Current phase
• Final phase

88
5. Adaptive Motion Control

• Behavior

Jump over ?
89
5. Adaptive Motion Control

• Behavior

Jump over ?
Speed profile
Loco Jump
90
5. Adaptive Motion Control

• Behavior

91
5. Adaptive Motion Control

• Behavior

Jump over ?
Speed profile
Way points
Loco Jump
Loco
92
5. Adaptive Motion Control

• Behavior

93
5. Adaptive Motion Control
94
5. Adaptive Motion Control
95
5. Adaptive Motion Control

• Results
• Transition locomotion jump
• Real-time
• Flexibility
• Limitations
• Obstacle as box in front
• Quadratic model

96
Contents

• Introduction
• Related Work
• Motion Modeling
• Motion Correction
• Adaptive Motion Control
• Conclusion

97
6. Conclusion

• Goals
• On-line and versatile animation model
• Environment

98
6. Conclusion

• Goals
• On-line and versatile animation model
• Environment
• Choices
• Motion capture data
• PCA
• IK for footplants

99
6. Conclusion

• Summary of contributions
• Motion parameterization
• Footplant detection and enforcement
• Coherent motion transition
• Dynamic obstacle handling
• Integration and applications

100
6. Conclusion

• Perspectives
• Applications
• Video game (autonomous character)
• Urban traffic simulation (crowd)
• Orthopaedics

101
6. Conclusion

• Perspectives
• Applications
• Video game (autonomous character)
• Urban traffic simulation (crowd)
• Orthopaedics
• Extensions
• Locomotion personification
• Physical validation
• Jump variety

102

• Questions ?

103

• IK formulation
• f non-linear
• Second priorityprojected null-space

104

• Rodrigues Formula
• Skew-symmetric matrix B

105

• Quaternion and axis-angle

106

• Speed profile

107

• Linear interpolation of quaternions

108
RTO p 1
RHS p 0
109
f
110
Approximation onsynthesized data
Approximation onsynthesized data
Original Data