Serious Games for Robotics and Command

1
Serious Gamesfor Robotics and Command Control

• Jay Crossler
• Senior Technical Advisor,The MITRE Corporation
• For more info, Google me

2
Serious Games
Serious Games Initiative is focused on the uses
of games in exploring management and leadership
challenges facing the public sector use of games
in education, training, health, and public
policy Seriousgames.org
6.5 million registered users

• MMOG - Massively multiplayer online games
• 1000s of players simultaneously connecting
• (supported by 1000s of self-transforming machine
  elves)
• Sharing actions and state millions of objects
• In persistent worlds

Ultima Online 1997 Origin Systems
3
Ludology social science analysis of games
4
Higher Education is supporting Ludology
And GMU is starting in 2008 an Undergrad degree
in Computer Science for Computer Game Design!
5
Overcoming Negative Social Stigma
Excerpt from Raph Kosters Theory of Fun book
6
Using Games forCommand and Control
7
Game Engines
Cost ranges from open source (CrystalSpace) to
100K (Unreal Engine)

• 3D graphics tools
• Physics engine
• Audio
• Animation
• Character AI

Visual3D Architect .NET Screenshot RealmWare
Corporation

• Typically tailored to a particular kind of game
• first person shooter (FPS)
• massively multiplayer online role-playing game
  (MMORPG)
• A few game engines include facial animation
  toolkits (e.g. Source Engine)

8
Game Engine List

• Engines
• Torque Low cost set of engines (2D, 3D,
  3DShaders), large dev community
• 3D Game Studio Hundreds of games, C-script,
  many libraries of pre-made games
• OGRE Scene-oriented, 3D engine, open source,
  Basic Physics
• Crystal Space Used for Modeling and Simulation,
  Physics engine, True 6DOF
• Many others at http//www.devmaster.net/engines/
• Terrain Tools
• L3DT Plugable Terrain Generation engine, low
  cost, importing into Torque
• Terragen 2 Amazing photorealistic terrains and
  terrain imagery More real than real
• Mod tools
• Return to Castle Wofenstein / Enemy Territory -
  Based upon an older version of the Quake engine. 
  
• Quake III - One of the most heavily modified
  games ever.  id has announced they will make the
  game code open source. 
• Counter Strike - A great starting point for
  tactical law enforcement sims and FPS
• Counter Strike Source - A rebuild of the
  original but to use the Source engine. Despite
  using the Source engine, you can "dial down" the
  graphics and effects to make this run on a more
  modest PC 

9
Game Engine List, cont.

• More Mod tools
• Call of Duty - COD2 added a smarter AI.  Tool
  support is really just for the map editor and not
  really for the program logic.
• Unreal Run-time - This is free and can be
  downloaded from the Unreal Dev site.  Very well
  documented - UT/editor documentation can be
  applied pretty easily as well.
• Medal of Honor franchise - Based on the older
  Quake engine so modest resource requirements.
• Halo (PC) Works on a very large number of
  machines and not just to the hardware extreme. 
  You can only make levels for an add-in
  multiplayer environment.
• Empire Earth II - An RTS favorite. Much more
  modest graphics and physics than Empire Earth
  III. Has a map tool but no way to re-write the
  application logic. 
• Rise of Nations - Comes with a map editor and a
  script editor for editing the logic of the game. 
  The scripting environment is specific to this
  platform but uses an event-driven paradigm that
  is widely used elsewhere - conceptual you get
  some mileage out of that investment.  You can
  create scenarios that are modern - good for
  simulations.
• Warhammer 40,000 -  The game as a very good RTS
  engine in it and Relic clearly knows its stuff in
  this genre.  The Relic developer site looks
  informative and has lots of tutorial and
  map-making information. 
• NeverWinter Nights - Used for several studies or
  experiments discussed in journal articles from
  the IEEE, ACM, etc. If you buy the DVD version
  you get lots of product for your money. Tools are
  easy enough for school kids to use.

10
Current Microsoft SDK Beliefs

• Visually stunning titles
• Half Life 2, Halo 3, Far Cry, Doom 3
• Largest worlds
• Most detailed levels of realism
• Not just graphics AI, physics, animation
• Longest development times
• Extensions
• Maya
• Max
• Photoshop
• Windows Shell
• Xbox 360/DirectX 10 Integration
• Increasing geometric detail
• 2K ? 5K ? 15Ktriangles/character
• More silhouette
• More shadow detail

Offload common functions to the GPU
Variable Transparency Mapping
Tessellation Displacement
Unsorted Transparency
11
Torque and Microsoft XNA
12
Were using Torque Gamine Engine

• Selected Torque tool
• Low cost for development/distribution
• Widely distributed with active user community
• Extensible feature set
• Robust
• Rich tool set
• Sold by Garage Games
• Platforms supported
• Windows
• Macintosh
• Linux
• License costs
• 500 - 1500 per software development seat
• No royalties or licensing costs for production
  software
• Ease of use short learning curve for software
  engineers
• 2 weeks to become productive, expertise within
  2-3 mos.

13
Things weve built
Proof of concept game

• UAV Surveillance

Job of Honor Recruiting Game
UAV RoE Prototype
14
Torque Features

• Torque provides complete visual engine and tools
• Torque Core Engine
• Near real time performance on PCs achieved
  through OpenGL and DirectX
• Math library with vectors and matrices
• Vehicle physics and spatial database
• Music driver
• Torque scripts for customization of
  missions/scenarios
• C like syntax
• Local and remote debugging
• Networking capabilities
• Complete source code provided for the engine and
  all tools
• Editors provided for objects, terrain, scenarios,
  etc
• OpenAL for sound

15
UAV Proof of Concept

• Objectives
• Leverage game tool technology to help
  organizations accomplish their mission
• Develop capability to deliver rich, dynamic
  visual information to users with less schedule
  and cost
• Provide simulation capability
• Demonstrate concepts using a visual and
  interactive approach
• Show effectiveness of real-time, interactive
  approach over canned rendering
• Reduce project risk
• Evaluate COTS tool and demonstrate its
  applicability
• Status
• Proof-of-concept done
• Reused/repurposed for Graphics Expo and TOCTWD
  game

16
UAV Rules of Engagement

• Objectives
• Leverage Game Engine technology to rapidly
  develop a UAV situational awareness and alerting
  prototype
• Incorporate real-time Cursor-on-Target data
• Leverage inherent technology (collision
  detection, realistic physical models, real-time
  2D/3D views)
• Status
• Proof-of-concept done
• CoT location/track/route messages supported
• Collision detection/avoidance being added
• Geo-referencing and NGA-like maps being added

17
Games are not currently

• Designed to replace specialized high fidelity
  simulations
• Able to replace all the time/effort that has been
  put into existing systems
• Interoperable with other games
• Pre-populated with models of real military assets
• Great at supporting NGA maps, coordinate systems

18
Job of Honor

• Objectives
• Improve communication with candidate new college
  hires
• Attract highest quality candidates
• Increase the pool of talented new college hires
  that apply to MITRE for employment
• Including under represented candidates
• Facilitate meeting or exceeding FY 2006 AC2
  hiring goals
• Status
• Phase 1 (short sample game) developed
• Campus modeled, two buildings interiors complete,
  game-play features in place
• Informative/educational dialog complete

19
Using Games whereKinetics meets Information
20
MITRE Meteor 2005 DARPA Grand Challenge Finalist
21
Meteor Layout
Vehicle Positioning Obstacle Sensors Road
Sensors Proprioceptive sensors Computing Vehicle
Control
GPS Antennas
Lights Siren
Downlooking Lasers
GPS INS IMU
Compass
Steering Actuator
Brake Actuator
Throttle Actuator
Roll Pitch Yaw
Shift Actuator
Gimbaled Laser
Computers Computer Racks Cooling System
Power System
Vertical Lasers
Emergency Stop
Horizontal Lasers
22
Vehicle Modifications

• 2004 Ford Sporttrak
• Retrofitted for drive by wire
• Electronic Mobility Controls Corporation (EMC)
• specializes in retrofits for handicapped drivers
• 6 Superlift Kit
• Off road tires
• Super swampers
• Heavy Duty Alternator

23
Software Organization
Admiral (console)
on/ off
go /stop
load
fuel, temp,volt, pitch,roll, vibration
WatchOfficers (datafusion) ObstacleDetection
VehicleLocationand Speed StatusWarnings Progr
essto Goal
Logger /Telemetry db
Ansible
Proprioception
Captain
waypointdatabase
Cursor on Target messages with encoded Loc Img
obstacle features
Visual
location,speed,orientation
go / stop
REST website
obstacle features
Range
nextwaypoint
Helmsman
waypointparser
desired speed
Navigator
obstacle features
Proximity
IMG DB
desired direction
Road Obstacle features
Location /Speed
location / speed
Geo Reports
current speed, orientation, caution / emergency
indicators
TerrainDatabase
Orientation
orientation
Google Maps
onboard
offboard
AJAX REST calls

• Agent-based architecture, Location transparency
• Any agent can run on any machine
• REST-based web interface gives agile interface to
  summary data

JavaScript browserVideo player
24
Online viewer
25
Online Viewer, 2
26
Online Viewer, 3
27
Online Viewer, 4
28
Online Viewer, 4
29
Takeaways

• Game engines offer low-cost (but low-fidelity)
  simulation alternatives
• Torque Gaming Engine is cheap enough and powerful
  enough for our first-effort requirements
• SOA methods help simplify designs
• Lots of decoupling
• Use lightest-weight standards possible
• Robotics are expensive to test and operate
• SA of what robotics operations are currently
  not integrated into C2 systems
• Game systems give a cheap alternative to test out
  various non-kinetic subsystems
• GMU is starting undergrad program in Game Design