Better etalk preliminary notes

1
Better e-talk(preliminary notes)

• tim_at_menzies.us
• bobbrown_at_uow.edu.au
• aditya_at_uow.edu.au

2
Sound bites

• STIR!
• When busy, use an agent
• The future is easy
• Learn what matters after sampling what might
  be
• Q what might be?
• A LURCH
• Everything is an FSM
• Q what really matters
• A TAR
• E-talk ltLURCH,TARgt
• Lots to do

1a 2a
PSU
WVU
JPL
2b 4
7
UOW
1a 3 5 6
3
The STIR philosophy
technology3

• STIR
• Throw all assertions into the pot
• Seek islands of consistency in the mix
• Formally abduction
• Consistent islands worlds of belief
• NP-hard
• Try abduction stochastic search
• ATMS parallel worlds generation
• STIR Serial world generation
• Reset
• fast generation of any world
• Go to 1

• Give the worlds to a data miner
• E.g. the TAR3 treatment learner
• find the key issues that most differentiate the
  worlds
• Alternative approach MENU
• Intricate solution knowledge (very 1980s)
• Must program domain AND solution knowledge
• Is STIR better than MENU?

technology1
technology2
4
Background

• Lets talk better

5
Motivation the world is a busy place

• How to handle peaks?
• Amazon.com
• 2001 XMAS holiday sales
• 41 of annual revenue
• Must automate

6
When busy, send an agent

• Personal agents
• Im in a meeting
• Go buy the kids some presents
• Business agents
• Tricycles
• Stocks low in USA
• Sticks high in Brazil
• Game boys
• Stocks high everywhere
• Goals
• Now
• nudge consumers from tricycles to Game Boys
• Ship stock brazil to USA
• Soon
• Nudge consumers to tricycles

7
E-business agents arguing
happiness
problems
solutions
monitor
find
find
assess
select
8
Agents reasoning about current beliefs and
future events
9
Negotiation agentsfor software engineering

• Increasing use by NASA
• Rapid development of mission concepts
• E.g.
• JPLs TeamX
• GSFCs IMDC

• How to audit TeamX or IMDC sessions?
• How ensure that, in the heat of the moment,
  critical features of a mission are not
  over-looked?
• Or worse, accidentally over-written by other
  decisions?

10
AgentSpeak (not the language, just a language)
A. S. Rao. AgentSpeak(L) BDI agents speak out in
a logical computable language. In W. Van de Velde
and J. Perram (eds), Proc. Seventh Workshop on
Modelling Autonomous Agents in a Multi-Agent
World (MAAMAW?96), Eindhoven, The Netherlands,
number 1038 in LNAI, pages 42?55.
Springer-Verlag, 1996. Bridging the gap between
BDI theory and practice
Agent ag1 auction(N) true lt-
place_bid(N,6). Agent ag2 myself(ag2). bid(ag2,
4). ally(ag3). auction(N) myself(I) ally(A)
not(alliance(A,I)) lt- ?bid(I,B)
place_bid(N,B). auction(N) alliance(A,I) lt-
place_bid(N,0). alliance(A,I) myself(I)
ally(A) lt- ?bid(I,B) .send(A,tell,bid(I,B))
.send(A,tell,alliance(A,I)).
Just bid 6 for each auction
bid 4 unless in league with ag3 (then bid 0)
.primitive e.g. .send(A,tell,bid(I,B) !todo
e.g. !auction(N) ?test e.g. bid(I,B) additions
-deletions todo context lt- action
11
Discourse is hard?

• Discourse strategies
• Threats
• or else!
• Promises of future reward
• free set of steak knives
• Appeals to past reward
• you liked it last time!
• Appear to precedent as counterexample
• darling, no one would be seen dead in that coat
• Appeal to prevailing practice
• our college was home to 7 past presidents
• Appeal to self-interest
• you get rich quick
• Etc.

• The future problem is a problem

• Elite
• need specialists to code
• Brittle
• Often need re-programming
• And such re-programming hard/slow
• Batch
• Interaction? Insight?
• Costly
• Since elite and brittle and batch

12
Technology 1

• Stochastic search
• thrashing around

13
Nondeterministic algorithms

• Can be simplest/fasters algorithm available
  Motwani95.
• E.g. skip lists simpler than AVL trees Pugh90
• E.g. Randomized min-cut faster than network flow
  Motwani95.
• E.g. TAR3 (data miner) learns much smaller models
  than standard data miners.

• Can success when determinism fails
• E.g. Can solve planning problems that defeat best
  deterministic methods Kautz96.
• E.g. LURCH (temporal simulator)
• Finds many of the faults found by SPIN/SMV
• runs faster (secs/mins, not hours)
• less memory (1-10MBs not 10-100MBs)
• Much simpler (!!) than model checkers
• Can jump out of cul de sacs
• No order effects
• E.g. Quick-Sort slowest on input already sorted
  in descending order.

14
e.g. stochastic n-queens
15
Technology 2

• The LURCH temporal simulator
• STIRing FSMs

16
LURCH exampleMutual Exclusion
define FALSE 0 define TRUE 1 enum A, B
turn A int a TRUE, b TRUE void before()
turn A a b TRUE a1 -
a TRUE a2 a2 - turn
B a3 a3 (!b) -
acs a3 (turn A) - acs acs -
a FALSE a5. b1 - b
TRUE b2 b2 - turn A
b3 b3 (!a) - bcs b3
(turn B) - bcs bcs -
b FALSE b5. ok. acs,bcs - _fault.

• Theory ltmachinegt
• Machine lttransition, stategt
• Transition ltnow, in, out, nextgt
• State ltX _X X. Xgt
• X progress cycle
• _X faulty state
• X. legal resting state
• X other
• States mutually exclusive
• One state is the default state
• Global state
• One state assignmentto every machine

• Machine 1 2
• Two processes
• competing for same resource.
• Machine 3
• Property model are 12 both in critical sections
  simultaneously (a fault).

17
LURCH exampleMutual Exclusion (2)

• An error is introduced by deleting the input
  condition for the transition marked in red

a1 - a TRUE a2 a2 -
turn B a3 a3 (!b) -
acs a3 (turn A) -
acs acs - a FALSE a5.
Machine A no longerchecks whether B hasclaimed
the sharedresource before enter-ing its
critical section.
a1 - a TRUE a2 a2 -
turn B a3 a3 - -
acs a3 (turn A) -
acs acs - a FALSE a5.
18
Inside LURCH

• Too deep default500
• Too long (secs) no default
• Too many repeats default 500
• On saturation
• each global state hashed to a 32-bit unsigned int
• lt X new global states seen in this repeat
• default 0.01
• Too many collisions
• same global state reached over all different
  repeats
• requires inter-iteration memory
• default250

• Inputsinitializations
• While not stop do
• time
• for m ? machines do
• for t ? m.transitions do
• if t.now m.now and t.in
• then begin
• ok(t.out)
• mt.next and t.out
• skip to next machine
• Inputs global state
• globalStatePathtimeglobal state
• done

• Search in random order
• Use one transition/machine
• Block if t.out inconsistent

19
Flight Guidance System

• Flight Guidance System
• Rockwell Collins Inc.
• Flight Control System
• Pitch and Roll Commands
• Complex mode logic
• Method
• Convert to LURCH, SPIN
• Seed errors in using historical records

Jimin Gao UMN
Mats HeimdahlUMN
20
LURCH runtimesLURCH low memory
21
LURCH stability
N-queens
Holzmanns leader election model (with
errors Injected)
Dining philosophers 1) Normal looping 2) No
loops (never get hungry again)
Got to find the shortest path (hard for SPIN)
Holzmann certified Promela
22
Technology 3

• The TAR3 treatment learner
• what is found by the random search?

23
TAR3 Reasoning aboutsoftware plans
24
TAR3 Patterns of risk,requirements, mitigations
requirements
risks
mitigations
25
TAR3 vs Simulated Annealing
with Dr. Martin Feather JPL, NASA

• JPL cost/benefit model
• 99 risk mitigation actions 2991030
• Run, find sweet spots, apply, re-run
• SA, TAR3 similar end results

• GAs, SA
• selected 99 actions
• TAR3
• selected 30 actions

26
Discourse strategies

• An application for
• technologies 1,2,3?

27
Discourse is easy?

• Discourse knowledge
• arguments, disputes, contracts, monitors
• Use domain theory to find pathways to
• goals(X0)-
• goals(X0)
• goals(X1)-
• goals(X1)
• Use discourse strategies to
• trade-off path-ways
• generate contracts
• Contract ltdeliverable, monitor, repairgt
• If monitor then repair
• Use temporal logic to monitor the contracts

• Domain knowledge
• Game boys, tricycles
• X0
• us
• X1
• them
• goals(X)
• things X wants
• goals(X)-
• what stops goals(X)

28
Discourse and paths
them1
D
E
us0
A
mine
F
B
yours
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
goal(a,1)
goal(c,1)
L
current
29
Discourse and paths
them1
D
E
us0
A
mine
F
B
yours
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
goal(a,1)
goal(c,1)
L
current
request
Us to them Please go to G
30
Discourse and paths
them1
D
E
us0
A
mine
F
B
yours
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
goal(a,1)
goal(c,1)
L
current
request
Us to them Please go to G
threat
Us to them Or I will go to c and threaten
goal(a)
31
Discourse and paths
them1
D
E
us0
A
mine
F
B
yours
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
goal(a,1)
goal(c,1)
L
current
request
Us to them Please go to G
threat
Us to them Or I will go to c and threaten
goal(a)
alert
Us to them Hey! You did not go to G. Im going
to C!!
32
Discourse and paths
them1
D
E
us0
A
mine
F
B
yours
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
mine
goal(a,1)
yours
goal(c,1)
L
current
request
Us to them Please go to G
threat
Us to them Or I will go to c and threaten
goal(a)
alert
Us to them Hey! You did not go to G. Im going
to C!!
promise
Us to them If you get G, Ill get goal(b) for
you
33
Discourse and paths
them1
D
E
us0

• Discourse strategies
• Threats
• Promises of future reward
• Appeal to self-interest
• Appeals to past reward
• Appear to precedent as
• counterexample
• Appeal to prevailing practice
• Etc.
• And efficiency (maybe)

A
F
B
J
G
H
goal(a,_)-

C
I
K
goal(b,0)
M
goal(a,1)
goal(c,1)
L
current
request
Us to them Please go to G
threat
Us to them Or I will go to c and threaten
goal(a)
alert
Us to them Hey! You did not go to G. Im going
to C!!
promise
Us to them If you get G, Ill get goal(b) for
you
34
The future is not a problem

• Q How to find all possible future paths
• A Dont find all.
• Let LURCH find most
• Q How to condense future paths into a succinct
  strategy?
• A TAR3
• What to do after what-ifs
• Q How to, anytime, get a strategy
• A TAR4 (incremental, anytime treatment learning)
• Q How to monitor future actions?
• A At runtime, add another FSM to LURCH
• Needs LURCH2
• E-talk ltLURCH2,TAR4gt ?

35
Lots to do

• Can it work?
• Temporal encodings (Menzies)
• AgentSpeak to FSMs (Vo)
• New tools (Menzies Owen)
• LURCH3 (dynamic FSMs)
• TAR4 (online, anytime treatment learning)
• Does it work at all? (Vo)
• Reproduce standard agent examples
• Does it work fast? (Owen)
• Dull if slow
• Connection to other methods? (Chang, Ghose)
• Formal analysis
• What cant it do? (Brown)
• Review of other discourse strategies
• Does it scale? (All)
• Big case study
• TeamX at JPL

36
Sound bites

• STIR!
• When busy, use an agent
• The future is easy
• Learn what matters after sampling what might
  be
• Q what might be?
• A LURCH
• Everything is an FSM
• Q what really matters
• A TAR
• E-talk ltLURCH,TARgt
• Lots to do

1a 2a
PSU
WVU
JPL
2b 4
7
UOW
1a 3 5 6
37
Questions? Comments?
38

Source IJCAI-01 tutorial Gomes, Hogg, Walsh,
Zhang
20 40 50