Arguments From Experience: The PADUA Protocol

1
Arguments From Experience The PADUA Protocol

• Maya Wardeh, Trevor Bench-Capon and Frans Coenen
• Department of Computer Science
• University of Liverpool

2
A Strange Beast

• It is warm blooded, has hair, lays eggs, does not
  suckle its young. Is it a mammal?
• Albert It can be a mammal because it lays eggs
• Bruce Ive seen mammals that lay eggs. And it
  does have hair
• Albert But mammals with hair also suckle their
  young
• Bruce All hairy, warm blooded animals are
  mammals
• Albert So I suppose we must say it is
• Albert and Bruce argue on the basis of mammals
  they have seen
• Their experiences have been very different
• There may be no right answer yet!

3
PADUA

• PADUA Protocol for Argumentation Dialogue Using
  Association Rules
• A dialogue game to argue about classification
• Arguments are taken directly from a data base of
  past examples using data mining techniques

4
Arguing from Experience

• Most dialogue systems are based on belief bases
• Participants use facts and rules to construct
  their arguments
• PADUA uses examples directly
• Participants have a data base containing
  collections of instances representing their past
  experience
• Resembles case based rather than rule based
  reasoning

5
Experiences May Differ

• Participants will have seen different samples.
• Geographical all swans are white in the Northern
  hemisphere
• Exceptions may be only rarely encountered
  insufficient support in some DBs
• Sample may be abnormal in law, only hard cases
  seen at highest level of appeal

6
Advantages

• No knowledge representation bottleneck
• No advance commitment to a theory
• Can deal with gaps and conflicts
• Pools experience

7
Dialogue Moves

• Participants point to features of the current
  case which are reasons why it should (or should
  not) be classified in a certain way
• Participants respond by citing other features
  which provide reasons to challenge the
  classification

8
Moves Based On Belief Bases

• Claim P P is the head of some rule
• Why P Seeks the body of rule for which P is head
• Concede P agrees that P is true
• Retract P denies that P is true
• P since S A rule with P head and S body

9
Persuasion in Belief Base Systems

• A Has a Rule with P as head, but one literal Q in
  the body is unknown B shows that Q is true.
• B gives A a rule with P as head and body S. A
  already believes S
• A is shown to have an inconsistency retraction
  enables P to be shown

10
Moves in Case Based Reasoning

• Citing a case
• A past case which shares features with the
  current case and had the desired outcome
• Distinguishing a case
• features in the past case missing from the
  current case
• Features in the current case missing from the
  past case
• Counter Example
• A past case which shares features but had a
  different outcome
• Arguments from Experience have many similarities

11
Moves in Argument from Experience

• Citing reasons
• Features in the current case which are typically
  associated with the desired classification, C
• Distinguishing
• An additional feature which typically identifies
  an exception
• A feature which Cs typically have but which is
  not present
• A feature which increases confidence in the
  classification
• Counter Example
• Features in the current case which are typically
  associated with a different classification, not
  C

12
PADUA Protocol - Basics
PADUA Scenario
Instance Case
Class C1
Class C2 (?C1)
A1 P suggests Q
A1
A2
A2 P suggests Q
13
PADUA Protocol - Basics
PADUA Scenario
PADUA Moves
1
1 Propose Rule
P is a reason for C
2 Distinguish
3 unwanted consequences
6
2
It need not be S
P and q is a reason for not C
4 Counter Rule
5 Increase Confidence
6 Withdraw unwanted consequences
3
Cs are not Q
5
it would be more a C if it were R
PADUA Protocol
P is a reason for not C
4
14
Strategies

• The protocol offers a lot of scope for choice
• Which move to make?
• Introduce a new association or refine an existing
  one?
• Which association to propose?
• Best or just a good one? In terms of confidence
  or support?

15
Strategies

• We have different strategies according to
• Aim establish a rule or critique opponents
  (build versus critique)
• Persistence concede when reasonable or only when
  no argument left (agreeable versus disagreeable)
• Different strategies give rise to different
  flavours of dialogue
• Build Disagreeable more like persuasion
• Critique Agreeable more like deliberation

16
Experiments

• We have experimented with a number of Data Sets
• Poisonous Fungi
• US Senators voting records (ESQUARU 2007)
• Welfare Benefits (COMMA 2008)

17
Welfare benefits

• Large numbers of cases
• Lay adjudication many (often inexperienced)
  adjudicators
• A high (often 20-30) error rate is typical
• Particular clerks and offices may make systematic
  misinterpretations
• PADUA can act as a moderation meeting, allowing
  debate over classifications drawn from different
  adjudication sources

18
Conditions for Benefit

• Age condition Age appropriate to retirement is
  interpreted as pensionable age 60 for women and
  65 for men.
• Income conditionAvailable income is
  interpreted as net disposable income, rather than
  gross income, and means that housing costs should
  exceed one fifth of candidates available income
  to qualify for the benefit.
• Capital condition Capital is inadequate is
  interpreted as below the threshold for another
  benefit.
• Residence condition Resident in this country
  is interpreted as having a UK address.
• Residence exception Service to the Nation is
  interpreted as a member of the armed forces.
• Contribution condition Established connection
  with the UK labour force" is interpreted as
  having paid contributions in 3 of the last 5
  years.

19
Results

• Given two databases, each containing a
  significant proportion of wrongly decided cases
  based on different systematic errors, correct
  classifications can be reached
• While this is true for errors concerning most
  attributes, success is markedly less when
  mistakes relate to the contribution condition

20
Intermediate Predicates

• These are legal concepts, which must be satisfied
  for the law to apply
• Need to be defined in terms of observable facts
• Some can be unfolded into observable facts
• Others need to applied on the balance of
  consideration of a number of factors
• These last present particular problems
• E.g. the contribution condition in the example
• Also true for other machine learning and data
  mining systems (e.g. Mozina et al)

21
Nesting Dialogues

• If we are aware of intermediate predicates which
  do not unfold appropriately into sufficient
  conditions, we can nest a dialogue to decide this
  issue within the main dialogue
• This handles the contribution problem
• Confirms other work in AI and Law in which issue
  based classification is more accurate than
  holistic approaches

22
Summary

• PADUA offers a novel kind of persuasion dialogue,
  based on examples rather than a belief base. The
  result has more in common with case based than
  rule based reasoning
• It avoids the need for knowledge representation
  effort
• The databases are not shared, enabling
  distinctive features of particular DBs to be
  identified and maintaining some level of privacy
• Where issues can be identified and resolved in
  preliminary dialogues, accuracy can be improved

23
The Talk Is Finished