Unmanned Underwater Vehicles

1
(No Transcript)
2
Pursuit Evasion Games (PEGs) for Multiple UUVs

• UUVs have the potential to provide an effective
  defense against submersible threats to military
  and civilian assets
• The strategies and protocols for their operation
  are at least as big a challenge as the design and
  construction of the vehicles themselves
• This project address the strategies and
  coordination protocols necessary to enable this
  technology
• Defense against enemy subs is the number one FNC
  that is called for in every briefing since 2000.

3
Bear UUV Being Built byLt Tulio Celano III, USN
8 ft X 10 in., 400 lbs. displacement, upto 100
ft. depth, Top speed 12 knots, effective
cruising speed 5 knots, endurance at 5 knots is
45 hours, Modular design
4
BEAR 1-UUV, Nov 2004
Showing Modules with Mast Up
Celano Machine Shop
Ballast Pump and Ballast Module
5
Prior PEG Experience

• PEG experience in
• Unmanned aerial and ground vehicle (UAVs and
  UGVs)
• Two and three dimensions.
• Symmetrical and asymmetrical games
• Proven tool Nonlinear Model Predictive
  Trajectory Control (NMPTC)
• Explicitly addresses nonlinear systems with
  constraints on operation and performance
• A cost minimization problem in the presence of
  state and input constraints
• Control resulting in the minimum cost is
  determined over a model predicted horizon
• Previously demonstrated in rotary-wing and
  fixed-wing UAVs

6
NMPTC Cost Function Definition

• Cost function is defined by

7
NMPTC Cost function illustration
8
NMPTC Cost function minimization

• A gradient decent method is used to minimize the
  cost function
• Initialization with previous result reduces the
  number of iterations required
• Usually 3-4 iterations are required
• The number of iterations in limited to prevent
  overruns in real-time
• In rapidly changing situation this can result in
  suboptimal solution
• Sudden changes may take several time steps
• However, this is alright since the situation is
  changing and unpredictable

9
Previous UC Berkeley PEG Experiments

• Berkeley Aerobot Project (BEAR)
• Goal to build a coordinated, intelligent network
  with multiple heterogeneous agents
• 11 Rotorcraft-based unmanned aerial vehicles
  (UAVs)
• 5 Unmanned ground vehicles (UGVs)
• Shipdeck simulator (landing platform)
• Stochastic Pursuit-Evasion Games (PEG)
• Self-localization
• Target detection
• Map building
• Pursuit policy
• Trajectory generation
• Control / Action

10
Previously PEGs with 4 UGVs and 1 UAV

• Sub-problems for Pursuit Evasion Games
• Sensing
• Navigation sensors -gt Self-localization
• Detection of objects of interest
• Framework for communication and data flow
• Map building of environments and evaders
• How to incorporate sensed data into agents
  belief states
• probability distribution over the state space of
  the world(I.e. possible configuration of
  locations of agents and obstacles)
• How to update belief states
• Strategy planning
• Computation of pursuit policy
• mapping from the belief state to the action space
• Control / Action

11
PEG Experiment with UAV/UGVs

• PEG with four UGVs
• Global-Max pursuit policy
• Simulated camera view
• (radius 7.5m with 50degree conic view)
• Pursuer0.3m/s Evader0.5m/s MAX

12
Evaluation of Policies for different visibility
Capture time of greedy and glo-max for the
different region of visibility of pursuers 3
Pursuers with trapezoidal or omni-directional
view Randomly moving evader

• Global max policy performs better than greedy,
  since the greedy policy selects movements based
  only on local considerations.
• Both policies perform better with the trapezoidal
  view, since the camera rotates fast enough to
  compensate the narrow field of view.

13
Evaders Speed vs. Intelligence
Capture time for different speeds and levels of
intelligence of the evader 3 Pursuers with
trapezoidal view global maximum policy Max
speed of pursuers 0.3 m/s

• Having a more intelligent evader increases the
  capture time
• Harder to capture an intelligent evader at a
  higher speed
• The capture time of a fast random evader is
  shorter than that of a slower random evader, when
  the speed of evader is only slightly higher than
  that of pursuers.

14
SEC Capstone Demo Fixed-Wing PEGs

• Capstone Demonstrations were proposed to
  highlight and test the technologies developed in
  the SEC program
• One was a fixed-wing UAV flight test
• 6 participant technology developers (TDs)
• Honeywell, Northrop Grumman, U Minnesota, MIT,
  Stanford, and UCB/U Colorado/CalTech
• System Integrator was Boeing
• OCP would be software framework
• Autonomous T-33 trainer as UAV surrogate
• Piloted F-15 as wingman/opponent
• 13 month schedule May 03 June 04
• UCB Contribution Fixed-Wing PEGs

15
Demo UCB PEG Scenario

• 20 60 min. games confirm NMPTC feasibility at
  real-time
• Evader goal get to final waypoint or avoid
  evader
• Pursuer goal target evader
• Pursuer and evader restricted to same performance
  limits
• Scenarios
• UAV as evader
• UAV can become pursuer

OCP Experiment Controller Snapshot T-33 Evader
(yellow) F-15 Pursuer (blue)
Target cone definition (?10,d3 nm) Left F15
not behind UAV, middle F15 not pointed at UAV,
right F15 behind AND pointed at UAV
16
Flight Test 1 (UAV as evader)
17
Flight Test 2 (UAV as evader/pursuer)
18
UUV PEGs Multiplayer Games

• In littoral waters the pursuit evasion game
  consists of an enemy submarine attempting to
  cross a line of UUVs which are protecting an
  asset
• The enemy submarine has a speed advantage over
  the blue force UUVs
• UUVs play a role in between a sensor web and a
  group of pursuers
• Research aimed at determining new approaches to
  teaming for multi-player games. Current
  literature focuses exclusively on either Nash or
  Stackleberg solutions

19
Multiplayer PEG Challenges

• The research challenge includes extending the
  strategies to
• Large multi-player teams
• Asymmetric platform characteristics
• Limited communications
• High level of uncertainly

20
UUV PEG Approach

• The UUV PEG involves two distinct phases
• Detection phase
• Maximize chances of detection constrained by
• Area to cover
• Number of UUVs available
• Possible evader strategies
• Capabilities of UUVs and evader (sensors and
  noise signatures)
• Response phase
• Maximize chances of catching the pursuer
  constrained by
• Capabilities of UUVs and evader (speed,
  manueverability and communications)
• Number of UUVs available
• These two phases also depend on each other as
  both must succeed
• How to share resources to maximize overall chance
  of success
• How to overlap the strategies detectors are
  responders as well

21
Multiplayer PEGs Proposed Solution

• A close analogy is football or other team games
• Multi player
• Initial (global) strategies well defined
• Limited (local) coordination after the snap
• What can we learn?
• How can we apply this?
• How far does the analogy go?

22
Multiplayer PEGs

• Preseason (Off-line precomputed strategy)
• Play book
• Evaluate strategies and configurations that will
  maximize chance of success based on best estimate
  of other teams tactics
• Practice and preseason games
• Test playbook and find problems
• Game time (On-line adaptive strategy)
• Choose play based on best knowledge and
  experience
• Line up (in best detection configuration, not
  necessarily static)
• Execute the play
• Active and reactive actions (respond to detected
  evader)
• Local communication
• Adapt to evolving behavior
• Learn from experience, repeat as necessary
  (Learning by Doing)

23
Pre-game Strategies for Detection

• Maximize the chance of detecting the evader
• Tradeoffs
• Movement of the pursuer
• Moving quickly covers more area, but
• This makes it easier for evader to see the
  pursuer and avoid the pursuer
• Sensors
• Using passive sonar reduces the range of
  detection
• Using active sonar reveals the pursuers location
• Number of pursuers in detector role
• Increases chance of detection

24
Basic principle Defense in depth

• May not be the optimal with limited resources,
• for instance if there are not enough UUVs to
  ensure detection of the evader by the front line.

25
Options Zone Defense

• Would this leave seams for an evader to exploit
  if they have superior sensors, for instance?
• Is communication necessary to make such a zone
  defense effective?
• Is there an alternative?

26
Options Channeling the evader

• A coordinated, heterogenous detection strategy.
• For instance, some pursuers could use a very
  active strategy that exposed them to intentional
  detection by the evader, with the intension of
  channeling the evader towards other more
  passive pursuers.

27
Pursuer Strategies Capture

• Speed disadvantage means that simply optimizing
  the detection probability is not sufficient
• Reachability of UUVs must be known
• Communication and coordination will be necessary
  to overcome speed disadvantage

28
Defining the ProblemBasic UUV PEG Scenarios

• Scenario 1
• Single evader infiltration
• Objectives
• Red pass through game area undetected
• Blue detect red team only
• Scenario 2
• Single evader attack
• Objectives
• Red get within weapons range of some objective
• Blue prevent red attack on objective
• Capabilities
• Red limited number of torpedoes available to
  attack target or blue team UUVs
• Red suicide attacks only, must get within
  effective range
• Scenario 3
• Multiple evader attack
• Objectives Capabilities
• Same as Scenario 2

29
UUV Characteristics in General

• Performance
• Speed and acceleration
• Maximum rate of turn
• Maximum rate of ascent/descent (not symmetric in
  general)
• Maximum depth
• Sensors
• Effective range in passive detection mode
• Effective range in active detection mode
• Deployable sonar buoys
• Communications
• Effective communication range (variable in
  general)

30
UUV Characteristics, cont.

• Method of attack, including
• Self detonation, effective range and perhaps
  effectiveness as a function of range
• Missile (i.e. torpedo) capabilities
• Counter measures, including
• Sonar buoys
• Noise canisters
• Noise signature as a function of
• Speed
• Acceleration
• Rate of turn
• Rate of ascent/descent
• Sensor mode
• Communication mode
• And others

31
The First Problem Definition

• Based on Scenario 2
• Single evader attack, many defenders
• Objectives
• Red get within weapons range of some objective
• Blue prevent red attack on objective
• Capabilities
• Red team
• Limited number of torpedoes available to attack
  target or blue team UUVs
• 3 times speed advantage over the Blue (pursuer)
  team
• Blue team
• Suicide attacks only, must get within effective
  range
• 3 times manuever (turn rate ) advantage over the
  Red team
• Limited communication range
• Passive and active sensors available

32
The First Problem Definition, cont

• Characteristics
• Speed
• 3X advantage to Red Team
• Maneuver advantage
• 3X to Blue Team
• Detection a function only of
• speed,
• communication use
• Distance
• Sonar
• Active passive available
• Communications available for each team
• range a function of power
• detectability also a function of power

33
Detection Strategy Comparison
34
Detection Monte Carlo results
Line abreast
Staggered

• Goals
• Statistical model as a function of configuration,
  spacing, etc.
• Test strategies

(Recall detection function)
35
Capture Strategies based on reachability
(Mitchells Level Set Toolbox)
From Airplane example
Still even, turn rate reduced to 1/3
Pursuer speed reduced to 1/3
Pursuer speed reduced to 1/3, turn rate increased
by 3

36
Combined the Strategies Advances in Game Theory

• These PEGs fit Game Theory descriptions as
• mixed strategy
• simultaneous move
• multiplayer
• coordinated games
• games with incomplete information
• Specific tactics can be evaluated to find the
  equilibria and optimal strategies in certain
  situations, e.g.
• the best search patterns within a single zone for
  one UUV, or for two adjacent zones/UUVs
• Statistical methods or Monte Carlo methods could
  be used to determine the changes of success for
  each player

37
Communication Between UUVs

• The issue of the short range of underwater
  communications with multiple players is very
  similar to the problem of ad hoc wireless
  networks of motes devices
• This experience is directly adaptation to the
  multiple, coordinated UUV scenarios and used to
  disseminate information within the team
  regarding
• Detection of an evader
• Likely detection by the other team
• Coordination instructions
• Strategic commands
• etc.
• This is integral into the simulation environment

38
Future Research

• We are building a group of UUVs at the USNA in
  Annapolis. These are also being shared with NUWC,
  Newport
• We will develop a theory of multi-player pursuit
  evasion games with off-line strategies (using
  robust optimization, the level set toolbox),
  on-line modifications of the strategy using Model
  Predictive Control, and an outer-loop of learning
• Expect to have simulation results by June 05,
  experiments for single UUV by June 05, multiple
  UUV experiments by June 06

39
UCAR Transition NMPC in a Complex 3-D Environment
Potential function
NMPC
40
UCAR-Obstacle Sensing

• Map-based approach
• - Obstacle map is measured and stored in the
  computer
• - Upon request, the nearest obstacle coordinate
  is passed to the MPC unit
• - Sensing is always perfect, thus reducing any
  risks due to sensing failure or any unpredicted
  control behavior

41
Obstacle Sensing using Laser Scanner
Scanner Control Computer
Tilt Mount
Position Command

• - Encoder
• Servo
• Micro controller

Encoder reading
- PIII 700MHz PC104 module
Measured range data
Ground Station
Measured range data

• 361meas/scan

Minimum range data

• Real-time 3D Visualization

Vehicle state
Light weight 2D Laser Scanner
Flight Computer

• Real-time optimization

Reference trajectory

• GPSINS
• PIII 700MHz
• PC104 module

Navigation data Minimum range data
MPC Engine
42
November 2004 Flight Tests
43
Multi-Player Games The Play of the Big Game 1982