Solving Games Without Determinization

1
Solving Games Without Determinization

• Nir Piterman
• École Polytechnique Fédéral de Lausanne (EPFL)
• Switzerland
• Joint work with Thomas A. Henzinger

2
Nondeterminizing NondeterministicAutomata

• Nir Piterman
• École Polytechnique Fédéral de Lausanne (EPFL)
• Switzerland
• Joint work with Thomas A. Henzinger

3
What?

• Get a nondeterministic automaton with n states.
• Construct a nondeterministic automaton with 2nn2n
  states.
• Why?

4
Plan of Talk

• Verification.
• Automata on Infinite Words.
• Synthesis.
• Design Synthesis in Action.
• Our solution.

5
Verification

• The normal process of development
• Write specifications (informally).
• Develop design.
• Test.
• Check that the system satisfies the specification.

6
Reactive Systems

• We are interested in systems that behave rather
  than compute (CPU, Operating system).
• Main complexity is in maintaining communication
  with a user / another program / the environment.
• The system has to be ready for every possible
  input.
• The system maintains behavior forever.

7
What is Behavior?

• The sequence of states the system passes along a
  computation.
• Nondeterministic systems / many possible inputs
  produce many possible behaviors.
• For reactive systems the behavior is infinite.

8
Automata Theoretic Approach to Verification

• Use automata to reason about systems and
  specifications.
• Questions like satisfiability and model checking
  reduce to emptiness of automata.
• Separates logical and algorithmic aspects of
  problems.

9
Automata on Infinite Words

• Introduced by Büchi, McNaughton, Elgot,
  Trakhtenbrot, Rabin, in the 60s.
• Basically take the same machine run it on
  infinite words.
• In infinite runs there is no last state. Use the
  set of recurring states.
• Büchi acceptance the set of recurring states
  intersects the set of accepting states.

10
Examples
q1
q0
11
Examples
q1
q0
12
Applications

• Satisfiability of S1S Buc62 and linear time
  logics.
• A linear time formula characterizes sets of
  sequences.
• Construct an automaton that accepts the set of
  models of the formula.
• Is the language of the automaton empty?

13
Applications

• Linear-time model checking VW94.
• A linear time formula characterizes sets of
  sequences.
• Construct an automaton that accepts all
  non-models of the formula.
• Consider the intersection of the automaton and
  the system.
• Is the intersection empty?

14
Verification

• The normal process of development
• Write specifications (informally).
• Develop design.
• Test.
• Check that the system satisfies the
  specification.
• We need a formal way to write specifications
  temporal logic.

15
Specifications

• We formally write specifications using temporal
  logic.
• We use automata on infinite words as an
  intermediate tool to reason about specifications.

16
Synthesis

• Cant we automatically produce the system from
  the specification?
• Produce systems that are ensured to work
  correctly.

17
Churchs Problem

• In 1965 Church posed this problem as
• Given a circuit interface and a behavioral
• specification, determine
• Does there exist an automaton (circuit) that
  realizes the specification?
• 2. Construct an implementing circuit.

18
Solutions

• Rabin develops the theory of automata on infinite
  trees Rab69.
• Büchi and Landweber propose a reduction to
  infinite duration games BL69.
• These are the main two solutions up till today.

19
Synthesis as a Game

• System controls internal variables. Environment
  controls input.
• Moves of system must match all possible future
  moves of environment.
• System plays against environment.
• System tries to satisfy specification.
• Environment tries to falsify specification.
• Success of system determined by the outcome of
  interaction.

20
Game Graphs

• We represent games as directed graphs.
• GhV,V0,V1,E,v0i
• The vertices are partitioned to those of player 0
  (system) and player 1 (environment).
• A play starts with a pebble on v0.
• If the pebble is on v2V0, player 0 chooses an
  outgoing edge and transfers the pebble.
• If the pebble is on v2V1, player 1 chooses the
  successor.

21
(No Transcript)
22
Winning Condition

• An infinite play is an infinite sequence of
  states.
• Winning conditions
• Recurrence / persistence in terms of states of
  the game.
• Linear temporal logic or automata on infinite
  words over states of the game.
• Does there exist a winning strategy?
• Use the automaton to follow the play and
  determine the winner?

23
Use Automaton

• Add one pebble on the automaton.
• Move the pebble on the automaton according to the
  move in the game.
• Decide acceptance according to the automaton.

Environment
System
Game
Automaton
24
Simple Game
Visit finitely many 0s
Environment
System
1
0
1
25
Nondeterminism is bad
Environment
System
26
Whats the Problem?

• The opponent chooses between (infinitely) many
  different paths.
• A guess should match all possible paths.
• Deterministic automata dont guess!

27
Determinization

• Need stronger acceptance conditions Lan69.
• Starting with NBW with n states
• DRW with 22n states McN66.
• DRW with (12)nn2n states and 2n index Saf88.
• DPW with n2n2 states and 2n index Pit06.
• Lower bound nO(n) Mic88,Yan06

28
Back to Games

• Games
• The opponent chooses between many different
  paths.
• A deterministic automaton enables monitoring the
  goal of the game.
• Games with LTL/NBW goals
• Convert LTL to NBW, convert NBW to DPW.
• Create product of game and DPW.
• Reasoning about general games reduces to
  reasoning about parity games.

29
The End?!

• Not really

30
In Practice

• Determinization is extremely complex.

31
Safras Construction

• Have a tree of subset constructions.
• Whenever a node (subset) visits F, create a new
  son with the states in F.
• If a node is removed flash red light.
• If a node equals its sons flash green light.
• The Rabin condition has a pair for every node.
  Node flashes red bad. Node flashes green
  good.

32
Deterministic State

• Ordered tree.
• Nodes are elements in 1,,n.
• Every node is labeled by a subset of the states.
• Every node is colored green, red, or white.
• Unused names are colored red.

33
Deterministic Transition

• The transition of d is the result of the
  following
• transformations.
• Replace node label by labels of successors
  (subset construction).
• Spawn new sons with accepting states.
• Move states to best nodes.
• Remove empty nodes.
• Nodes that equal their sons colored green.

34
What about your variant?

• Recently, improvement of Safra
• Safra NBW(n) ! DRW(12nn2n,n)
• Variant NBW(n) ! DPW(n2n2,2n)
• But still trees, and everything else.

35
Or abcdefghij
36
In Practice

• Determinization is extremely complex.
• First implementation in CIAA05.

37
OmegaDet STW05
38
In Practice

• Determinization is extremely complex.
• First implementation in CIAA05.
• No way to implement symbolically.
• All or nothing.
• Resort to other solutions.

39
Practical Solution 1

• Restrict attention to a subset of LTL.
• Safety / reachability linear time
  RW89,AMPS98.
• Recurrence / persistance quadratic time
  AMPS98.
• Boolean combinations of safety / reachability
  AT04.
• Generalized Reactivity(1) cubic time PPS06.

40
Practical Solution 2 JGB05,HRS05

• Heuristics that use the NBW.
• Works? Good.
• Does not work?

41
Nondeterminism

• Nondeterministic automata cannot be used for game
  monitoring.
• Or can they?
• They just have to be built correctly

42
Good for Games Automata

• Automata that can be controlled in a step-wise
  fashion.
• Defined via a game on the structure of the
  automaton.
• Can be used for game monitoring.

Environment
System
Game
Automaton
43
Definition

• Define the monitor game played on the structure
  of the automaton
• Start from the initial state.
• Opponent chooses a letter.
• We choose successor.
• We win if
• The resulting word is not in the language
• The resulting run is accepting
• An automaton is GFG if we win from initial state.

44




1
1
1
1
1
1
1

1
1
1
1
1
1
1



1
1
0
45
0
0,1
2
1
3
0,1
1
1
46
Use for Game Monitoring

• Given a GFG we combine the game with the GFG.
• Player 0 chooses how to advance the GFG.

Environment
System
Game
Automaton
47
Where do I get one?

• Prove that an automaton is good for games if it
  fair-simulates another good for games.
• Deterministic automata are trivially good for
  games. So start from the deterministic automaton.
  
• We show how to construct one.

48
Construct a GFG Automaton

• Replace the tree structure by nondeterminism.
• Follow nondeterministically n subsets of states.
• Ensure that all the runs followed by some subset
  visit accepting states infinitely often.
• Wrong guess? Change your mind!
• Intuition
• - first set is the subset construction.
• - other n-1 sets follow subsets of first set.

49
Construct a GFG

• Lets start with details on determinization.

50
Determinization in Detail
Subset Construction

• There are infinitely many runs that reach an
  accepting state a finite number of times.
• Somehow these runs have to be separated.

51
Determinization Construction

• Have a tree of subset constructions.
• Whenever a node (subset) visits F, create a new
  son with the states in F.
• If a node is removed flash red light.
• If a node equals its sons flash green light.
• The parity condition follows the minimal node
  that flashed red/green infinitely often.

52
(No Transcript)
53
What is a state

• A tree.
• Nodes are elements in 1,,n.
• Every node is labeled by a subset of the states.
• G21,...,n1 - the least node colored green.
• R21,,n1 the least node that got erased.

54
Transition

• Replace label by the set of successors (subset
  construction).
• Create youngest son with subset of accepting
  states.
• Move double states to older brothers.
• If node equal to union of sons, remove sons and
  color green.
• Remove empty nodes.
• Compact names.

55
1
2
b
subset construction
remove empty nodes
1
1
0,3
0,3
2
c
subset construction
spawn sons
move to older sons
Handle full nodes
1
1
1
1
2
2
2
2
4
4
4
4,1
4
a
3
3
subset construction
move to older sons
Handle full nodes
spawn sons
1
1
1
1
0,1,3
0,1,3
0,1,3
0,1,3
3
2
2
2
4
4
4
2
4
3
3
3
1
3
5
5
56
From OmegaDet STW05
1
1
0
0
1
0
1
0
57
Safra from a nodes point of view

• I follow some states.
• Some of them may disappear.
• If all visit acceptance set, I raise a green
  flag.
• If all disappear I die.
• After I die, I can be revived with a new set.

58
Our ConstructionA State

• Up to n subsets of the states of the NBW.
• Every state in a subset is either marked or
  unmarked.
• If a subset is empty all subsets above it are
  empty.

59
Our ConstructionA Transition

• Replace every set with a subset of the possible
  successors.
• Successors of marked states are marked accepting
  states are marked.
• If all are marked, remove marking.
• An empty set can load a subset of the first set.

60
Advantages

• Very simple construction.
• Amenable to symbolic implementation.
• Natural incremental structure leading to complete
  solution.

61
A Range of Constructions

• We can get closer / further from the
  deterministic automaton.
• The number of states goes between n2n and n3n.
• It all depends on the symbolic implementation

62
Incremental Construction

• We dont always need n sets.
• An automaton with i1 sets monitors fully more
  games than an automaton with i sets.
• It depends on the game itself.
• It is not related (directly) to memory.

63
Summary

• Replace deterministic automata by
  nondeterministic automata.
• Definition of GFG automata.
• Construction of GFG automata.
• Simple, amenable to symbolic implementation.
• Incremental structure leading to the full
  solution.
• Initial enumerative implementation.
• Lower bound.

64
Safraless Decision Procedures KV05

• Emptiness of alternating parity tree automata by
  rank computation.
• Requires determinization for the upper bound.
• Reduces to Büchi games instead of parity.
• Complexity may be quadratically worse.
• Strategy may be exponentially worse.
• Enables solution of games with LTL winning
  conditions. Does not apply for NBW winning
  conditions. Does not apply to infinite structures.

65
Future Work

• Implementation.
• Reuse work done in increments.
• Understand better the incremental structure.
• Automata for the complement language.
• Lower bound on the index.

66
Going Both Ways

• It would be nice to find both winning and losing
  states fast.
• Starting from LTL it is easy.

• Starting from NBW?
• Build GFG for N.
• Build KV ranks for N.

• Build NBW N? for ?.
• Build NBW N ? for ?.
• Combine the game incrementally with GFG for N?.
• Combine the game incrementally with GFG for N ?.

67
Thank You