Spatio-Temporal Case-Based Reasoning for Behavioral Selection

1
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Maxim Likhachev and Ronald Arkin
• Mobile Robot Laboratory
• Georgia Tech

2
Broad Picture of the Work
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Part of Multi-Level Learning in Hybrid
  Deliberative/Reactive Mobile Robot Architectural
  Software Systems project at Georgia Tech
• Sponsored by the DARPA MARS program

3
Motivation
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Constant parameterization of robotic behavior
  results in inefficient robot performance
• Manual selection of right parameters is
  difficult and tedious work

4
Motivation (contd)
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Use of Case-Based Reasoning methodology for an
  automatic selection of optimal parameters in
  run-time

5
Evaluated on
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Simulations
• Real robot
• ATRV-JR in outdoor environment
• Nomad 150 in indoor environment

6
Related Work
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• ACBARR, SINS and KINS systems
• use of case-based reasoning and reinforcement
  learning for the optimization of behavioral
  parameters
• contribute to some ideas behind the present
  algorithm
• Automatic optimization of parameters
• genetic programming
• reinforcement learning

7
Behavioral Control and CBR Module
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• CBR Module controls
• Weights for each behavior BiasMove Vector
• Noise Persistence Obstacle Sphere

8
Input Features for Case Selection
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Vector of spatial characteristics of environment
• D - distance to the goal
• lts, rgt - degree of obstruction and distance to
  the most obstructing cluster of obstacles for
  each of K angular regions around the robot

9
Input Features for Case Selection
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Vector of temporal characteristics of environment
• Rs - short term robot movement
• Rl - long term robot movement

10
Computation of Traversability Vector F
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• F
• represents traversability of each region
• approximates obstacle density function around the
  robot
• independent of goal distance
• smoothed over time

11
Input Features Example
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
12
High Level Structure of CBR Module
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
13
Case Example I
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
CLEARGOAL Spatial Vector D (goal distance) 5
density distance
Region 0 s0 0.00 r0 0.00 Region 1 s1
0.00 r1 0.00 Region 2 s2 0.00 r2
0.00 Region 3 s3 0.00 r3 0.00 Temporal
Vector (0 - min, 1 - max) ShortTerm_Motion Rs
1.000 LongTerm_Motion Rl 0.700 Case Output
Parameters MoveToGoal_Gain 2.00 Noise_Gain
0.00 Noise_Persistence
10 Obstacle_Gain 2.00 Obstacle_Sphere
0.50 Bias_Vector_X 0.00 Bias_Vector_Y
0.00 Bias_Vector_Gain 0.00 CaseTime
3.0
14
Case Example II
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
FRONTOBSTRUCTED_SHORTTERM Spatial Vector D (goal
distance) 5 density
distance Region 0 s0 1.00 r0
1.00 Region 1 s1 0.80 r1 1.00 Region 2
s2 0.00 r2 1.00 Region 3 s3 0.80 r3
1.00 Temporal Vector (0 - min, 1 - max)
ShortTerm_Motion Rs 0.000 LongTerm_Motion Rl
0.600 Case Output Parameters MoveToGoal_Gain
0.10 Noise_Gain 0.02 Noise_Persisten
ce 10 Obstacle_Gain
0.80 Obstacle_Sphere 1.50 Bias_Vector_X
-1.00 Bias_Vector_Y
0.70 Bias_Vector_Gain 0.70 CaseTime 2.0
15
Results
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
Simulations
Average travel distance
Mission success rate
ATRV-JR 12 average performance improvement
in time steps (
based on 10 runs for each system in outdoor
environment)
16
Simulations real robot experiments Performance
improvement as a function of obstacle density
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
Simulations
Nomad 150
Based on 10 runs for each system in indoor
environment
17
Real Robot Run with CBR
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
18
Real Robot Run without CBR
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
19
Trajectories of the robot
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection
Robot with CBR module
Robot without CBR module
11 less travel distance
20
Conclusions
Spatio-Temporal Case-Based Reasoning for
Behavioral Selection

• Automatic selection of optimal behavioral
  parameters results in robot performance
  improvement (based on simulations and real robot
  experiments)
• Careful manual selection of behavioral parameters
  is no longer required from a user
• Future Work
• Automatic learning of cases
• identifying when to create a new case
• applying reinforcement learning techniques in
  finding optimal parameters for existing cases
• Integration with other adaptation learning
  methods (e.g., Learning Momentum)