Colyseus: A Distributed Architecture for Online Multiplayer Games

1
Colyseus A Distributed Architecture for Online
Multiplayer Games

• Ashwin Bharambe, Jeffrey Pang, and Srinivasan
  Seshan
• Carnegie Mellon University

2
Online First-Person Shooters

• Multiplayer architecture for multiplayer
  first-person shoot games.
• low latency required
• weakly consistent state tolerable
• spatial contiguity allows pre-fetching
• Traditionally this is done with a client-server
  model
• leads to bottlenecks
• can only sustain user numbers in the dozens

3
Colyseus

• Single-copy replication model
• maintains consistency by serializing updates
• mirrors existing server model
• cuts latency (reading from local copy), but
  sacrifices consistency in game-wide state
• DHT lookup for object queries
• both random and range-query DHTs implemented
• range-query DHT works well since object queries
  will always be in contiguous spatial regions
• DHT query delay overcome by anticipating which
  objects will be needed soon and pre-loading them

4
Replica Manager

• Follows Tunable Availability and Consistency
  Tradeoffs (TACT) model
• depending on specific game characteristics,
  developers can select either more availability or
  consistency - consistency lowers availability
  (increases lag), availability lowers conistency.
• synchronizes replicas to primary
• changes are delta-encoded, sent to primary, then
  distributed serially to all replicas
• when a node becomes interested in a replica (i.e.
  the player is near that object), it registers
  with the primary and receives updates directly
  (decoupled discovery/synchronization)
• creates new replicas
• deletes replicas that are no longer needed
• fast moving objects (missiles) use a special case
  attachment so that they are automatically sent to
  nodes that request the object they are attached
  to (the person who fired the missile)

5
Range-Queriable DHT

• both standard random DHT and rangeDHT implemented
  with Mercury
• adjacent nodes responsible for adjacent keys
• player x,y coordinates used as key, then game can
  request other objects that are near the player
  from adjacent nodes
• predictions based on current player motion can be
  used to pre-load upcoming objects
• with a known average DHT lookup time, the
  prediction can be tweaked so that objects finish
  loading about when they are needed

6
DHT comparison
7
Evaluation

• Experimental Setup
• Emulab used to simulate virtual servers
• no link capacity constraint, but ties end-to-end
  latency to measured samples
• artificially dilate time to counter slow virtual
  servers
• model game based on density statistics for a
  quake III map ( Zipf distribution)
• Modified Quake II to work with Colyseus
• Mercury rangeDHT
• variable size bounding box corresponding to
  visible objects for a character used as
  area-of-interest
• client/server messaging remains intact so
  unmodified engines could connect to any of the
  p2p nodes as if it were a server
• Custom map w/ bots for workload
• Emulab testbed

8
Results
9
Discussion

• Colyseus enables multiplayer first-person-shooter
  games to handle hundreds of players, instead of
  dozens
• since FPS games have very high demands for low
  latency and consistency, extending this
  architecture to other game types, like role
  playing games, is very feasible
• adaptation of commercial Quake II shows that this
  is feasible for other games in a production
  environment
• this method opens up many possibilities for
  cheating (a node could be modified to request
  objects that it shouldnt see, for instance),
  but more work could be done to address those
  threats.

10
Related Work

• Real-time strategy (command conquer)
• parallel simulation often used, since consistency
  is very important
• often limited to less than 10 players
• Online role playing (world of warcraft,
  second-life)
• cell based
• centralized server or server cluster
• Distributed Virtual Reality Environments
• similar goals to Colyseus, but specific
• not catering to common game applications

11
Issues

• why not just evaluate actual gameplay? (custom
  map, all bots seems a little suspect)
• synchronization decoupling seems to introduce a
  bottleneck. Its still better than a server model
  since a primary replica might be the only one on
  a node, but the node still has to handle all
  traffic for that replica
• how are node failures (player logs off) handled?
• how do you handle updates on objects in a view
  that is already inconsistent, especially since
  the node cannot know if its view is consistent.