Reverification of Adaptive Agents Plans

1
Re-verification of Adaptive Agents Plans

• Diana Gordon
• Naval Research Laboratory

2
How This Project Fits in the MURI

• Model checking issues
• Real-time
• Probabilistic
• Abstraction
• Adaptive systems

3
Motivation

• We build agents/systems that become smarter and
  more efficient.
• Machine learning assumption More knowledge -gt
  potential to be more constructive.
• But what guarantees do we have that our systems
  wont also be more destructive?

4
A Solution Re-Verification of Adaptive Agents
Plans
O F F L I N E
PLAN
Verification
O N L I N E
PLAN
Adaptation
NEW SITUATION
Rapid Re-verification
REVISED PLAN
5
Objective Rapid Re-verification

• Re-verification from scratch.
• Time-inefficient. If m actions for each of n
  agents, time complexity is O(m ).
• Restrict learning using a priori results.
• Safe machine learning
• S P gt L(S) P or S P gt L(S)
  L(P)
• Safety guarantee with no run-time cost!
• Incremental re-verification.
• Useful when general a priori results are negative
  or difficult to obtain.
• Time efficiency gained by localizing and reuse.

n
-(
-)
-)
6

Some Examples Where Restriction vs
Incremental is Useful

• Could restrict the learning
• Planetary rovers that quickly adapt to unforseen
  events but stay within mission constraints.
• Require rich repertoire of learning methods
  require incremental
• Anti-viruses that learn but dont act like
  viruses.
• Intrusion detection systems with countermeasures
  that improve but dont clobber resources of
  friendly users.

7
Prior Related Research

• Results relevant to a priori Kurshan (1994)
• Incremental re-verification
• Sokolsky and Smolka (1994)
• Weld and Etzioni (1994)
• Reps and Teitelbaum (1989)

8
Assumptions

• Automaton (reactive) plans for agents (S)
• A Boolean algebra is the language of the
  transition conditions
• Temporal logic (TL) properties (P)
• (Re-)verification Model-checking (S P)
• Learning methods abstraction A and
  generalization G.

9
Abstraction vs Generalization
10
Abstraction
(A no_MAVS) (B no_MAVs)
(C go_A)
WAIT, WAIT, GO_A
(A no_MAVs) (B MAVs_wait) (C go_A)
(A MAVs_wait) (B MAVs_wait) (C go_A)
GO, WAIT, GO_B
WAIT, WAIT, GO_B
((A MAVs_go) (B MAVs_wait) (C go_B))
(A no_MAVs) (B MAVs_wait) (C go_B)
WAIT, GO, GO_B
11
Generalization
(A no_MAVS) (B no_MAVs)
(C go_A)
WAIT, WAIT, GO_A
(A MAVs_wait) (B MAVs_wait) (C go_A)
(A no_MAVs) (B MAVs_wait) (C go_A)
GO, WAIT, GO_B
WAIT, WAIT, GO_B
((A MAVs_go) (B MAVs_wait) (C go_B))
((A no_MAVs) (B MAVs_wait) (C go_B))
(A no_MAVs) (B MAVs_wait) (C go_B)
WAIT, GO, GO_B
12
Abstraction and Model Checking

• Abstraction A for system S and property P.
• For abstractions that reduce complexity of
  model-checking
• Want A to be sound
• A(S) A(P) S P
• For safe machine learning abstractions
• Want A to be complete
• S P A(S) A(P)

13
New Results for Abstraction

• Popular abstractions (projection and
  partitioning) are a priori guaranteed to be
  safe (novel application of Kurshans 1994
  results), but only if the property is abstracted
  also.
• Identified situations in which its ok to
  abstract a property.

14
New Results for Generalization

• Generalization is not always a priori safe.
• Novel algorithms for incremental re-verification
  of generalization of automaton transition
  conditions. To maximize efficiency, tailored to
  property types
• Always/Never properties
• Sometimes properties
• Proofs of correctness for algorithms.
• Time complexity results.

15
Conclusions

• With our novel methods, agents can
• Adapt to new situations.
• Adapt safely and quickly (for rapid response
  time).

16
Future Work

• Continue research on a priori results.
• Develop a theoretical foundation for incremental
  re-verification of adaptive agents plans.
• Plan repair for unsatisfied properties.
• Use counterexamples from failed re-verification
  to guide choice of better learning method for
  plan repair.
• Extend to stochastic automata and probabilistic
  (re-)verification.