Stimulating Preference Expression using Suggestions

1
Stimulating Preference Expression using
Suggestions
AAAI Fall Symposium Mixed Initiative Interaction

• Paolo Viappiani1 (speaker), Boi Faltings1
• Vincent Schickel Zuber1, Pearl Pu2
• Artificial Intelligence Lab (LIA)
• Human Computer Interaction Group (HCI)
• École Polytechnique Fédérale Lausanne
• Switzerland

2
Agenda

• Example Critiquing

2. Stimulate Preference Expression
3. Pareto Suggestion Strategies
4. Evaluation
3

• How do we usually search for products on the web?

4
Sorry there is no solution!
5
(No Transcript)
6
Far too expensive!
Go Back
7
index.html ltHTMLgt ltFORM ACTION..gt
lt/FORMgt ..lt/HTMLgt
ltHTMLgt ltTABLE ..gt lt/TABLEgt ..lt/HTMLgt
query
database

• REPONSE
• sorry there are no results ? have to start
  from scratch
• results not satisfactory ? back-button
• no checks that the user has really stated all
  that he wanted!

8
Questions form-filling

• Users feel obliged to fill in many of the
  preferences
• Stating preferences without having seen any
  example is inaccurate
• Users dont know what the attributes mean
• Users state means objectives

9
Means-objectives

• True objective cheap accommodation
• System asks for preferred type
• Means objective I prefer a shared-apartment
  Keeney, Value Focused Thinking
• Means objectives lead to inaccurate decisions
  (only 25 accurate)

10
Example Critiquing

• Mixed initiative system
• Shows examples of complete solutions
• Invites users to construct and revise their
  preferences through critiques
• cooperation between user and the system

11
Example Critiquing
Initial preferences
User revises the preference model by critiquing
examples
System shows K solutions
User picks the final choice
12
Example Isy-travel 2000
13
What to show?

• Candidates the best K given a utility model for
  current preferences
• Suggestions solutions that stimulate the user
  in stating his (true) preferences

Maximize a combination of all preference orders
Our job!
14
Suggest Extreme options

• Linden et al. 1997
• Often unreasonable
• Cheapest flight
• but leave at 1a.m.
• Closest apartment to centre
• but no parking
• Cheapest student accommodation 0
• but have to work as au pair take care the
  family children
• Too many to choose from two for each attribute!

15

• User states his preferences only if he expects
  that it will have some impact on the solutions
• Suggestions should be options that could become
  optimal when an additional preference is stated

Optimality definition depends on preference
modeling ? Pareto Optimality Dominance Relation
16
Dominance relation

• A dominates B if A not worse than B wrt to all
  preferences and A strictly better than B for at
  least one preference

• A dominates B (B dominated by A) if A better wrt
  all preferences
• Pareto-optimal are the non dominated options
• Example
• A1suburbs, downtown
• A2apartment, house
• A3expensive, cheap
• There is no HOUSE available in DOWNTOWN

17

• Downtown gt Suburbs
• SuburbsHouseCheap dominated by
• DowntownAptExpensive, DowntownAptCheap
• Cheap gt Expensive
• SuburbsHouseCheap dominated by
• DowntownAptCheap
• House gt Apartment
• SuburbsHouseCheap no longer dominated
• we have another Pareto-optimal option
• Dominated options can become Pareto-optimal when
  adding preferences

18
There are no houses downtown!
Downtown,House,Cheap
DowntownHouseExpensive
DowntownAptCheap
SuburbsHouseCheap
SuburbsAptCheap
SuburbsHouseExpensive
DowntAptExpensive
SuburbsApartmentExpensive
Pareto Optimal
configuration not possible
19
Assumption about the User

• The user will state his hidden preference on
  attribute i
• if
• The system shows an option that will become
  Pareto Optimal if that preference is stated

20
Suggestions strategies
Extrema
Probabilistic
Counting
Attribute
To become optimal an option there should exist an
attribute on which it is different/extreme than
all dominating options.
Pick options showing extreme values.
how many options dominate the current one?
Maximize the probability that dominance will be
broken (through heuristic measure).
21

• You are looking for an accommodation
• cheaper than 700chf
• Not too far from the university (10 minutes car
  distance)

22
Preferences PRICElt700, DIST_UNIVlt10
SUGGESTIONS
RANDOM
EXTREME
23
Preferences PRICElt700, DIST_UNIVlt10
SUGGESTIONS
COUNTING STRATEGY
ATTRIBUTE STRATEGY
PROBABILISTIC STRATEGY
24
(No Transcript)
25
User Test

• 54 users
• Interface C
• only shows candidates
• Interface CS
• shows candidates suggestions

26
Between groups

• Group A shown interface C
• Group B shown interface CS
• tStudent, BgtgtA
• 99 confidence

27
Within group A

• group A
• use C
• then, use CS

group A using C CS ? group B using CS
28
Work in progress

• Online user study monitoring real user looking
  for an accommodation
• New supervised user study measuring real accuracy

29
Measuring Decision Accuracy
Step 1
Step 2
Step 3
EC interface with 6 optimal
EC interface with 3 optimal 3 suggestions
Full list
choice
choice
choice
?
30
User study on Decision Accuracy

• 60 subjects
• Apartment search
• Supervised test

31
Conclusions

• Standard approaches (form-filling) lead users to
  state wrong preferences
• Mixed-initiative systems help people construct
  the preference model
• Dramatically increases decision accuracy
• Suggestions are important

32
Attribute filter motivation
Preferences on price (to minimize), on M2 (to
maximize)

• S2 and S3 are both dominated by S1
• If we add new preference
• on Location ? if North is preferred S2 will be
  Pareto Optimal
• on Transport ? if Tramway is preferred to Bus
  then S2 will be P.O.
• S3 will always be dominated!!

33
Dominance relation and Pareto optimality
Penalty table, 2 preferences
s4
P2
9
s5
6
s3
s1

• S1 and S2 are Pareto optimal
• S3 is dominated by S1 and S2
• S4 is dominated by S1
• S5 is dominated by S1, S2, S3.

3
s2
3
6
9
P1
34
A new preference is added

• New column with penalties
• S4 becomes Pareto optimal even if the new penalty
  (0.6) is worse than for S3 (0.5) and S5 (0.4)

The counting filter predict that S4 has better
chances to become P.O. when a new preference is
added.
35
Attribute filter motivation
Preferences on price (to minimize), on M2 (to
maximize)

• S2 and S3 are both dominated by S1
• If we add new preference
• on Location ? if North is preferred S2 will be
  Pareto Optimal
• on Transport ? if Tramway is preferred to Bus
  then S2 will be P.O.
• S3 will always be dominated!!

36
SUGGESTIONS
COUNTING FILTER
ATTRIBUTE FILTER
PROBABILISTIC FILTER
37
Impact of number of preferences
100 sims, 9 preferences, 9 attributes average
fraction of preferences discovered
38
Impact of number of attributes
100 sims, 6 preferences, 6/9/12 attributes of
correctly discovered preferences
39
Probabilistic filter

• Directly estimate probability of becoming P.O.
• The bigger the difference on a specific
  attribute, the more likely the penalties will be
  different

penalty
penalty
1
1
domain
domain