27%20de%20Fevereiro

1
Sensor Fusion Applied to Soccer Robots
Prepared by Pedro Marcelino Oriented by
Prof. Pedro Lima
2
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

3
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

4
Sensor FusionMotivation

• Increased interest in the developing of
  multi-sensor robots
• Limitations in the reconstruction of environments
• Observation errors, bad calibrations or partial
  and incomplete information of the world
• Cooperation to resolve ambiguities
• Robust and consistent description of the world
• Team with a common goal and shared knowledge, so
  it can take the right decisions.

5
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

6
Sensor FusionSensors Caracteristics

• Sensor Complexity
• Observation Error
• Observation Disparity
• Multiples Points of View

7
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

8
Sensor FusionSensor as a Team Member

• Multi-Sensorial System Team of Sensors
• Each sensor is considerer an individual
• Each sensor make local decisions
• Each sensor implements its actions
• The Team coordinate the activity of its members
• Information exchange to resolve conflits and
  validation of observations
• Makes the Team Decision Problema a simple
  Estimation Problem

9
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

10
Sensor FusionSensor Models

• Observation Model
• It is a static description of the sensor
  performance, realting the observation with the
  state of teh environment
• Front Camera Model
• Up Camera Model

C
CL
11
Sensor FusionFront Camera Observation Model
12
Sensor FusionUp Camera Observation Model
13
Sensor FusionSensor Models

• State Model
• Relates the observation of a sensor with a given
  location and its internal state
• Perspective change to a common frame so that the
  observation can be compared

14
Sensor FusionState Model

• Each feature is represented as with a gauss
  distribuition
• Mean
• Variance
• Angle with central axis
• Distance to feature
• New variance results from the perspective
  transformation to a global frame

15
Sensor FusionSensor Models

• Dependency Model
• Describe sthe relation between the observations
  and the actions of each sensor
• Team Utility Function
• Team Decision Fucntion
• Groups Rational Aximos
• Each member makes a decision that maximizes its
  Team Utility Function

16
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

17
Sensor FusionObservation Integration

• Each feature is modeled by a gauss distribution,
  using Bayes Law
• If the Mahalanobis distance is less than 1, then
  there is agreement and the team member will
  cooperate, to estimate the feature position,
  otherwise, there is desagreement and the team
  member observation will not be used

18
Sensor FusionObservation Integration

• Two bayes observers showing agreement

19
Sensor Fusion Observation Integration

• Two bayes observers showing desagreemnet

20
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

21
Sensor FusionImplemented Algoritms Ball
Detection

• Ball detection in front Camera

22
Sensor Fusion Implemented Algoritms Ball
Detection

• Ball detection un Up Camera

23
Sensor FusionCamera Models
24
Sensor FusionSensor Models Diagram
Observation of Sensor 2
Observation of Sensor 1
Observation Model
Observation Model
Sensor Model
State Model
Change of perspective to Global Frame
State Model
Dependency Model
Decision and Integration of Observation
New Fusion Validation Variance Increase with Time
Team Utility Function
Structure that keeps all decisions made by the
team members
25
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

26
Sensor FusionExperimental Results
27
Sensor FusionTopics

• Motivation
• Sensors Caracteristics
• Sensors as Members of a Team
• Sensor Models
• Observation Integration
• Implemented Algoritms
• Experimental Results
• Conclusions

28
Sensor FusionConclusions

• Real time fusion of the world information
• Good estimative of features localization
• Makes system more robust, eliminating sporadic
  errors
• Coerent World decription
• Use of Bayes Teorem to solve the decision problem
• It is a really good method to be used in modern
  robotics, which should be used whenever possible
  to determine the position and orientation of the
  environment features that surrond the robot

29
Sensor FusionFuture Work

• To be developed during the Master
• Sensor Fusion of several robots
• Other players detection
• Team players detection
• Sensor Fusion to determine robot position

30
Sensor FusionSensor Fusion Diagram
World Model
BlackBoard
global.worldmodel.
Local Sensor Fusion Algoritm of Other Robots
Dependency Model
Global Sensor Fusion Algoritm
Local Sensor Fusion Algoritm
Dependency Model
BlackBoard
local.up.
local.front.
local.sonars.
local.odometry.
Observation and State Model
Up Camera
Front Camera
Sonars
Odometry
Sensors
Up Camera
Front Camera
Sonars
Odometry
31
Team Members

• Docentes do IST
• Pedro Lima (coordenação) - DEEC
• Luis Custódio (coordenação) - DEEC
• Carlos Pinto Ferreira (professor associado) -
  DEM
• Alunos de Doutoramento (EEC)
• Miguel Garção
• Alunos Finalistas (TFC)
• Bruno Damas - LEEC
• Pedro Pinheiro - LEIC
• Hugo Costelha - LEEC
• Gonçalo Neto - LEEC
• Cláudio Gil LEIC
• Miguel Arroz LEIC
• Bruno LEIC