How Much Intelligence Do You Need to Play Soccer?

1
How Much Intelligence Do You Need to Play Soccer?

• Soccer and Intelligence?
• Robotic Soccer and the RoboCup Competitions
• CS Freiburg Hardware and Software Architecture
• Cooperative Sensing
• From Perception to Action
• Team Play Dynamic Role Assignment
• Conclusions Outlook

2
Soccer and Intelligence?

• Do you really need any intelligence in order to
  play soccer?
• Isnt it just a reactive game?

3
Soccer and Intelligence!

• Reconstruction of environment from sensor data
• Spatial reasoning
• Being able to select the right motor skill (and
  parameterize it) in order kick the ball into the
  right direction
• Strategic thinking and acting
• Proactive in order to create opportunities
• Reactive in order to exploit opportunities

• Insects do not play soccer

4
Why Robotic Soccer?

• Chess is solved
• Robotic soccer has a number of interesting
  properties (that makes it different from chess)
• uncertainty in sensor interpretation and acting,
  a highly dynamic environment, and the need for
  cooperation
• Challenge for designing integrated systems that
  close the loop
• Solutions have relevance for other areas, in
  particular for multi-robot systems (e.g., teams
  of cleaning robots)

5
The RoboCup Initiative

• Alan Mackworth 1993 Integration of AI,
  Robotics, MAS, Real-Time Reasoning
• RoboCup Kitano et al 97
• Workshops Competitions
• RoboCup leagues
• Simulation league
• F180 (small) league
• Aibo (Sony dog) league
• F2000 league (4 players/team, 4x9 m field)
• Challenge
• Win against human world champion by 2050

Goal against RMIT Raiders (Melbourne) (RoboCup99
)
6
Competitions Science

• Competitions push the development of science
• direct comparisons
• everybody gives the best
• development of new, innovative solutions
• Possible problems
• exploitation of loop holes in the rules
• and concentration on competition
• How to build a successful system
• single conceptual pieces are very important
  (e.g.self-localization, team coordination)
• combination and integration is important
  closing the loop
• Results are communicated in workshops symposia

7
Our Goals, Our Approach

• Demonstration of our self-localization techniques
  using laser scanners
• Exploration of multi-robot systems
• Cooperative acting
• Cooperative sensing
• Attractive area for students
• Classical Artificial Intelligence approach
• explicit model of the world
• deliberation on world model
• . . . finding the right balance between
  reactivity and deliberation

8
CS Freiburg Robot Hardware

• Hardware
• Pioneer 1 robots (from ActivMedia)
• Pioneer 2 controller boards
• Vaio Picturebook (with Linux)
• WaveLan radio-Ethernet
• Kicker custom-made (SICK AG)
• Sensors
• Digital Sony camera with Firewire
• SICK laser scanner
• Internal odometrie
• Kicker sensors (state/ball)

9
Player Architecture
cooperation strategy
action selection
communication
world modeling
action execution
sensors
actuators
10
Sensor-Interpretation

• Inputs
• Laser scanner data (360 range values for 180,
  1cm accuracy, 30 scans/sec)
• Odometrie (translation, rotation since last
  measurement, 10 estimates/sec)
• Vision color data (720x576 pixel, 25 frames/sec)
• Outputs
• Own pose (position and orientation) self
  localization
• Poses and velocities of other players (team mates
  and opponents)
• Ball position and velocity

11
Self-Localization

• Estimation of own pose location and orientation
• Difficult, even if initial position is known
  (odometrie errors accumulate)
• Use sensors to correct pose estimation
• GPS (works only in outdoor environments), beacons
  ...
• Range finding sensors (sonars, laser scanners,
  ...)
• Vision (landmarks or 3D reconstruction)
• Combine odometrie and other sensor measurements
• Kalman filter or Markov localization

12
Our Approach Line Extraction Scan Matching

• Extract lines from scans
• Try all pairings of scan lines with model lines
  and test for geometric realizability
• i.e. find rotation and translation to make it fit
• 2 hypotheses if 3 walls are visible, 4 if 2
  walls. Initial orientation is known
• fast, robust, accurate
• global localization

13
Player Recognition

• Remove all points from the scan that correspond
  to points on the wall
• Cluster remaining points
• Consider point of gravity as middle point of
  player (perhaps with offset)

14
Why is Self-Localization Player Recognition
Important?
Going to the kick-off positions - different
approaches (Game against CMU Hammerheads,
RoboCup 2001)
15
Another Reason The 10 Seconds Rule
We obey the 10 seconds rule and leave the goal
area before 10 seconds have elapsed ... finally
we score a goal against NAIST00 (Japan) (RoboCup
2000)
16
Ball Recognition in Real Time

• Color segmentation (watch for orange blobs)
• previously Newton Lab hardware system
• now Sony camera with Firewire output, vision
  processing on standard notebook, CMVision system
  (CMU) for color segmentation
• now we see the ball up to 5-6 meters
• Size and distance estimation
• using a case-based interpolation method instead
  of Tsai-calibration and analytic estimation -
  more accurate if z-coordinate is constant
• Identification of most plausible blob as the ball
• using size, distance, and duration of observation

17
Multi-Robot Sensor-IntegrationCooperative
Sensing

• All players send their estimates (own position,
  ball position, position of other players)
  together with a time stamp to the global
  sensor-integration module.
• Estimations are combined
• Friend-opponent-distinction our players report
  their own positions
• Global ball position estimation
• Integrate measurements from different robots
  using a Kalman filter
• but ignore hallucinations using a Markov
  localization scheme

18
Kalman Filter

• Prediction step (predict next location where ball
  will be observed)
• Project ball position into the future using a
  constant negative ball acceleration
• Consider a certain projection error

• Update step (when new observation is made)
• Integrate new measurement
• Take into account sensor model
• distance error grows with distance
• angular error is small and constant
• Leads to triangulation (stereo vision with a
  robot group) . . .

19
False Positives
?
Player 2 is hallucinating
20
Phantom Ball EliminationMarkov Localization

• Prediction step (with unknown direction and speed
  - only 2 dimensional probability grid)
• The occupancy probability of a cell z t flows
  into cells z close to z (cond. prob. has normal
  distribution wrt. distance)

• Update step
• Incorporate new ball sighting, use Bayes rule
  (cond. prob. has normal distribution wrt.
  distance)

21
Phantom Balls Development of Probability
Distribution
after 1st measurement (1)
after 2nd measurement (2)
after 3rd measurement (3)
after 4th measurement (1)
after 5th measurement (2)
after 6th measurement (3)
Consider area with highest peak as possible ball
area
At RoboCup 2000, 938 out of 118388 (0.8) ball
observations were ignored because of the Markov
localization filter. At German Open 2001, 0.6
ball observations were ignored.
22
Importance of Accurate Global Ball Position
Estimation
Our goalie cannot see the ball but continues to
defend the goal against Golem (Italy) in the
RoboCup 2000 final ... since he knows the ball
position from his team mates
23
A Positive Example
Minho (Portugal) shoots at our goal from the
other side of the field. Our goalie gets this
information early on and can easily defend
(RoboCup 2001)
24
From Perception to ActionDo the Right Thing!

• Reactive, behavior-based vs. deliberative
  approaches
• Soccer is a fast game and it does not require
  long (on-line) deliberation
• However, there exist higher levels beyond
  situation-action-rule execution
• different actions and an action selection
  mechanism
• cooperation placement and team play
• opponent prediction and counter strategies
• Most teams use hybrid approaches, but some are
  almost completely reactive

25
How do we Structure the Capabilities of a Soccer
Agent?

• Actions State-free behaviors (controllers)
• Action selection
• Select the most appropriate action
• Do this by goal back-chaining (extended behavior
  networks)
• Once an action has been selected, commit to it
  (persistence to avoid oscillations) and quit if
  action will fail necessarily
• Control execution of actions
• Evaluate and adapt continuously every 100 msec
• Strategic considerations are factored out

26
Field-Player Action Repertoire

• With ball
• ShootPos shoot to a position, often close to
  opponent goal
• MoveTrickShoot run into direction of opponent
  goal and in the last possible moment change
  direction
• BumpShoot shoot ball by bumping against it
• TurnBall turn with ball slowly into the
  direction of opponent goal
• DribbleBall run with ball into the direction of
  opponent goal
• Without Ball
• ObserveBall, SearchBall, GotoPosition, WaitPass,
  GetBall, GotoBall . . .

27
What is Important When Implementing Basic Actions?
An attack on the GMD Robots goal, which could
have been carried out more skillfully (RoboCup
1999)
28
Field Player Tricking the Goalie
We score against CE Sharif (Iran) by tricking
the goalie Move in one direction, shoot in the
other direction (MoveTrickShoot) (RoboCup 2000)
29
Field Player Dribbling

• Consider points that are closer to the opponent
  goal and that do not require a sharp turn
• Evaluate the straight lines to these points
  according to
• distance to obstacles
• angle to goal
• angle to current orientation
• Choose best alternative

30
Dribbling An Example
One of the 9 goals against CMU Hammerheads
(USA) (RoboCup 2001)
31
Field Player ShootPos

• Find a good position (close to opponent goal) to
  shoot to
• Consider also rebound shots!
• Choose the shot that
• does not collide with obstacles
• comes as close as possible to the opponent goal
• does require a minimal turn angle

32
Field Player A Rebound Shot
Rebound shot against CoPS Stuttgart (RoboCup
2000)
33
Action Selection Extended Behavior Networks

• Behavior networks Maes 90
• action representation with preconditions and
  consequences
• activation from situation goals (? reactive
  goal-oriented)
• Extended behavior networks Dorer 99
• fuzzy propositions
• activation only from goals
• goal-tracking and no input normalization
  (? decision theoretic
  planning)
• used in magma Freiburg (runner up in the
  simulation league RoboCup 99

34
Part of the Extended Behavior Network
active role
soccer goal
cooperate
ObserveBall
MoveTrick- Shoot
ShootPos (goal)l
WaitPass
ball_present
have_ball
SearchBall
good_p_pos
GetBalll
close_to_ball
have_ball
35
ExampleAction Selection Sequencing
Attack against CMU Hammer-heads (RoboCup 2000
quarter final)
Action Sequence GetBall, TurnBall, DribbleBall,
ShootPos(goal)
36
Acting Cooperatively

• Without coordination, one gets swarm behavior
• Avoidance of interference
• do not attack your own team mates
• do not get into the way of an attacking or
  defending robot
• use competence areas on the field (still backup)
• Task decomposition and task (re-)allocation
• the player which is closest to the ball should go
  to the ball
• if one player cannot do his task, another should
  take over
• Joint execution passing the ball
• Use (dynamic) role assignment

37
Cooperation Dynamic Role Assignment

• Each player has one of 4 roles
• goalie (fixed)
• active player with ball or close
• supporter other half of field
• strategic player defender
• Placement each role has a preferred location,
  which depends on the situation
• ball position, position of team mates and
  opponents
• defensive situation or attack

active Role
strategic role
supporter role
38
Dynamic Role Assignment

• Each player computes the utility for each role
  and sends it around (same as in ART Italy)
• Utility depends on distance to preferred location
  (large distance has low utility) and on the role
  (active gt strategic gt supporter)
• Each player tries to maximize the group utility
• under the assumption that all team members do
  that
• Roles are reassigned only when two players agree
• this does not exclude that there are occurrences
  when two players have the same role (because
  there is no synchronization)
• Note that opinion about global position can
  differ (even with global world model)

39
Team Play Role Switch
Tight defense against CE Sharif
(Teheran) (RoboCup 2000 semi final)
40
Role Switch by Communication
green strategic light blue supporter white
active
41
Another Example for Role Switching
Defense against Artisti Veneti (Italy) (RoboCup
2001) The roles active and strategic player are
switched a couple of times
42
Joint Execution A Pass . . . that was
Unsuccessful
A pass in the semi-final against the Italian ART
Italy team (RoboCup 1999). This was based on
standard plan if it is not possible to score
directly, wait until supporter arrives, then make
the pass
43
Passing the Ball as Emergent Behavior
A slightly unconventional pass in a game against
Robosix (RoboCup 2001)
44
Reactive vs. Deliberative RoboCup 2001 Final
CS Freiburg against Osaka Trackies (Japan) CS
Freiburg is most of the time slower and has to
defend ... but it is more robust. Finally, when 2
Trackies had been removed . . .
45
CS Freiburg Performance

• Competition Results
• Winner of RoboCup98, RoboCup 2000 and RoboCup
  2001
• Winner of German Vision-RoboCup98 99 and
  German Open 2001, runner up in 2002
• 2 draws 3 lost games in 52 official games
• Goal Rates (goals/minute)

46
Conclusion Outlook

• The kind of intelligence you need to play soccer
  is quite different from the one you need to play
  chess
• Robotic soccer is an attractive and interesting
  research challenge, in particular for the area of
  multi-robot systems
• Current future research issues
• Learning adapting the skills to new
  environments and capabilities
• Addressing a different problem in the domain a
  robot referee
• Address other multi-agent problems such as
  RoboCup-Rescue
• Transfer to a new domain table soccer

47
Table Soccer

• . . . is interesting because computer systems may
  be able to play this game against humans (we do
  not intend to wait until 2050)

• Two humans against one computer unfair?

48
Acknowledgements

• CS Freiburg Team 2001 (www.cs-freiburg.de)
• Markus Dietl, Florian Diesch, Steffen Gutmann
  (Sony), Alexander Kleiner, Boris Szerbakowski
  (SICK), Thilo Weigel, Patrick Stiegeler
• Previous team members
• Burkhard Dümler, Wolfgang Hatzack, Immanuel
  Herrmann, Kornel Marko, Klaus Müller, Livia
  Predoiu, Christian Reetz, Frank Rittinger,
  Maximilian Thiel, Augustinus Topor
• Institutions
• SICK AG (Hardware and manpower)
• DFG, MFG, und Univ. Freiburg (financial support)
• Sony, Siemens