Multiagent Probabilistic Smart Terrain

1
Multiagent Probabilistic Smart Terrain

• Dr. John R. Sullins
• Youngstown State University

2
Multi-Agent Search in Games

• Player in someroom on thislevel
• Multiple guardssearching forplayer
• Some roomsmore likelythan others

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
2
3
Multi-Agent Search in Games

• Guards mustdivide up rooms inplausible way
• Focus on most likely rooms
• While makingsure all roomssearched

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
3
4
Outline

• Goals of multi-agent probabilistic search
• Background Probabilistic smart terrain
• Estimating global expected distances to targets
  that meet goals of group
• Matching agents to targets
• Demonstration on examples

5
Multi-Agent Smart Terrain

• Assumptions
• Teams of NPCs with same goal (such as find
  player)
• One NPC finds target that meets goal ?entire
  team succeeds
• Must be fast solution
• No time for complex negotiations among characters
• Plausible behavior from POV of player sufficient

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
5
6
Plausibility Benchmarks

• Cooperative behavior
• Room 1 closer to G1 and more probable than room 2
• G1 should still move to room 2
• G2 can cover room 1
• Both rooms searched quickly

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
6
7
Plausibility Benchmarks

• Probability as a factor
• Room 1 much more probable than room 2
• G1 should move directly to room 1
• Player overwhelmingly likely to be found there
• Main purpose Find goal, not search all targets

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
7
8
Plausibility Benchmarks

• Divide and conquer
• G1 closer to both rooms and could explore both
• G2 should still move to the rooms also even
  though closer to neither
• If G1 moves to one room, G2 can quickly cover the
  other

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
8
9
Outline

• Goals of multi-agent probabilistic search
• Background Probabilistic smart terrain
• Estimating global expected distances to targets
  that meet goals of group
• Matching agents to targets
• Demonstration on examples

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
9
10
Smart Terrain

• Target meets goals ? transmits signal
• Signal moves around objects, weakens with
  distance
• Character follows signal to target

11
Limits of Smart Terrain

• Normal smart terrain not appropriate for all
  situations
• Guard search example
• Player transmits signal
• Guards follow directly to player
• Obvious cheat!
• Guards should have to search for player based on
  probabilities

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
11
12
Probabilistic Smart Terrain

• Targets broadcast signal of form I meet goal
• I may meet goal with
  probability P

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
12
13
Expected Distances

• Expected number of tiles character must travel
  from tile x to target that meets goal
• dmax
  Dist(x) S ? (1 pi )
  d0 di lt d

Probability no target within d tiles of x meets
goal (assumption of conditional independence)
Summed over all distances
up to some maximum dmax (otherwise sum could be
infinite)
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
13
14
Expected Distances

• Compute Dist(x) for adjacent tiles x
• Move to adjacent tile with lowest Dist(x)

p 0.7 distance 6
p 0.6 distance 8
15
Outline

• Goals of multi-agent probabilistic search
• Background Probabilistic smart terrain
• Estimating global expected distances to targets
  that meet goals of group
• Matching agents to targets
• Demonstration on examples

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
15
16
Global Expected Distances

• Each agent Aj estimates moves until goal found by
  some agent (not necessarily itself)
• Distances to targets Aj is moving towards
• Distances of other agents to targets Aj is
  moving away from

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
16
17
Finding Closest Targets

• Step 1 Each agent Aj determines set of targets
  Tj that it is closer to than any
  other agent

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
17
18
Partitioning by Direction

• Step 2 For each possible next tile for Aj,
  determine which targets Tj direction ? Tj
  would be closer in that direction
• Tj left
• Tj right
• Tj up
• Tj down

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
18
19
Global Expected Distances

• Step 3 Compute global expected distance for each
  possible next tile x based on
  targets ti ? Tj
• dmax Dist(x) S ? (1
  pi ) d0 di lt d
  
• ti ? Tj direction ? di distance(x, ti ) 1
• ti ? Tj direction ? di min(distance(Ak, ti ))
  j ? k(distance
  to closest other uncommitted agent)

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
19
20
Global Expected Distances

• Tj left
• Tj right
• Tj up
• Tj down

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
20
21
Global Expected Distances

• Example Expected global distance for A1 moving
  left
• Neither room reached in lt 4 moves
• A1 reaches room 2 (probability 0.4) in 4 moves
• A2 reaches room 1 (probability 0.5) in 5 moves

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
21
22
Moving Agents

• Each agent computes global expected distances for
  surrounding tiles
• Each agent then moves to tile with the lowest
  global expected distance

A1 moves left
A1 moves right
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
22
23
Global Expected Distances

• Probability of targets is also an important factor

A1 covers R2, A2 covers R1
A1 covers R1, A2 covers R2
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
23
24
Outline

• Goals of multi-agent probabilistic search
• Background Probabilistic smart terrain
• Estimating global expected distances to targets
  that meet goals of group
• Matching agents to targets
• Demonstration on examples

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
24
25
Assigning Agents to Targets

• Closest targets in direction of tile x with
  minimum Dist(x) now assigned to that agent
• May not be considered by any other agent this move

R2 assigned to A1
R2 cannot be considered by A2
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
25
26
Committing Agents to Targets

• Agent now committed to those targets
• That agent will not be used by other agents to
  compute their Dist(x)

A1 committed to R2 and will not move to R1
Cannot be used by A2 to determine global expected
distance to R1
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
26
27
Subsumed Agents

• Some agents may not be closest to any targets
• Agents subsumed by other agents
• Initially, no move chosen
• Are reconsidered after other agents choose
  directions

A2 closest to neither
A1 closer to both
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
27
28
Subsumed Agents

• Targets agents moving away from are released to
  subsumed agents
• ti ?Tj , ti ?Tjx where Dist(x) is minimum
• A1 chooses to move towards R2
• R1 released for consideration by other agents
• A2 now uses R1 to move left

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
28
29
Overall Algorithm Structure

• for each move
• while (unassigned targets uncommitted agents)
  
• find closest uncommitted agent to each unassigned
  target (agents with no targets are subsumed)
• for (each non-subsumed agent A)
• A computes Dist(x) based only on other
  uncommitted agents
• A committed to move in direction with minimum
  Dist(x)
• As targets in that direction assigned to A
• Targets not in that direction released for next
  cycle of loop

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
29
30
Subsumed Agents

• Agents may still be subsumed if all targets
  assigned to other agents
• Loop ends without all agents being assigned
  targets
• A1 best move is down
• Both R1 and R2 closer in that direction
• Both R1 and R2 assigned to A1
• No remaining targets for A2

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
30
31
Subsumed Agents

• Base subsumed agent move on all targets
  regardless of what other agents are doing
• Use original single-agent probabilistic smart
  terrain formula
• Gives agent appearance of doing something

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
31
32
Subsumed Agents

• Agents may eventually not be subsumed
• Agent moves to area with multiple targets
• Will move towards one target and away from others
• Those other targets now available to subsumed
  agents in area

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
32
33
Outline

• Goals of multi-agent probabilistic search
• Background Probabilistic smart terrain
• Estimating global expected distances to targets
  that meet goals of group
• Matching agents to targets
• Demonstration on examples

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
33
34
Performance on Initial Example

• Player may be in one of 7 rooms, 3 with
  treasure
• 3 guards searching for player
• Probability player in a given treasure room
  0.2
• Probability player in a non-treasure room 0.1

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
34
35
Dynamic Targets

• Target tiles placed one step inside each room
• Gives guards appearance of looking in a room
• Probabilities change when guard reaches tile
• Player not in room ? probability set to 0
• Guard now influenced by other rooms

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
35
36
Performance on Initial Example
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
36
37
Initial Example Modified

• Gold room moved to upper left
• Guard 2 moves to jewel room instead
• Guard 1 path also altered as result

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
37
38
Ongoing Work

• Testing with actual players
• Implement algorithm as game (Unreal engine)
• Goal steal treasure while avoiding guards
• Player can see guard movement, guards use
  algorithm to search for player
• Do NPC guard actions appear plausible to players?

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
38
39
Multiagent Probabilistic Smart Terrain

• Dr. John R. Sullins
• Youngstown State University

40
Player Found in Large Example

• Player tile probability set to 1
• Other target probabilities set to 0
• All guards will now converge on player

John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
40