prof. dr. Lambert Schomaker

1
KI2 8
Heterogeneous-Information Integration

• prof. dr. Lambert Schomaker

Kunstmatige Intelligentie / RuG
2
Heterogeneous-information integration

• aka
• multi-sensor fusion
• multi-expert combination
• multi-agent collaboration
• The improved use of multiple information sources
  which are of different unit and scale

3
Heterogeneous-information integration

• Examples
• terrorist weapon classification
• friend or foe
• forensic evidence collection
• finding oil sources
• pattern classification by multiple experts
• audio-visual speech recognition

4
different units

• Celsius
• microgram
• Volt
• Ampere
• Lumen
• probability
• pseudo-probability
• integer count

5
different scale

• ratio scale
• interval scale
• ordinal scale (1st 2nd 3rd 4th 5th 6th )
• nominal scale
• yes/no
• green red purple
• good bad ugly
• true/false

A
B
6
Architecture, example
Expert 1, NN
Expert 2, Rule-based
real world
Expert 3, Bayesian
COMBINE
Measurement i
DECISION
Measurement j
agent k
agent l
agent m
7
How to combine heterogeneous information?

• trained parameter-estimation methods
• context-free methods

8
Trained, parametric combination methods

• Use a trainable function approximator
• mean field (linear, weights)
• multi-layer perceptron (NN)
• polynomial
• Bayes!
• cumbersome train individual components, train
  the combination
• if a new module or expert is added, the system
  must be completely retrained!
• independent training sets are needed for the
  single functions and for the combination function

9
Context-free combination methods

• majority voting
• plurality voting
• product rule
• sum rule
• rank combination schemes

10
Voting

• A candidate ci is a person, object or proposal,
  and C is the set of all possible candidates, and
• Ce is the set of candidates taking place in a
  particular election
• A voter is a function vj Ce ? R, in words, each
  candidate partaking in the election obtains a
  real- valued confidence of vj in ci

11
Election

• An election is a tuple (Ce,Ve) where Ce ? C
• and Ve ? V, such that
• ?vj?Ve vj Ce ? R
• yielding Ve orderings of the candidates, in R

12
Voting system criteria

• Condorcet winner will win from all candidates if
  elections were held in a pairwise fashion.
• A Condorcet loser could exist too
• Consistency if ci is a winner for voters Vk and
  for voters Vm, then ci should also be the winner
  if the election is based on Vk ? Vm

13
More voting-system criteria

• Monotonicity if votes become available, this
  should not affect the existing valuation
• (humans often react non-monotonously in a
• sequential voting procedure). Also, voting
  procedures which eliminate candidates one by one
  are non monotonous.
• Pareto optimality the voting system
• choses cx over cy if all voters choose cx over
  cy

14
Example majority vote in unreliable but
independent experts
15
Special case Borda rank combination

• Each of N voters ranks M candidates
• The assumption is that an optimal ranking exists
• Individual voters utilize an unknown evaluation
  function
• vj Ce ? R where j1,N, e1,M
• Evaluations are sorted, such that the best
  evaluation ranks 1, etc. up to M, worst

16
Example Evaluation scores 0-100
Beer Voter A Voter B Voter C
Heineken 45 30 99.1
Grolsch 42 31 70.
Hertog Jan 30.2 12 31.2
Duvel 10.4 5 40.8
Koninck 80 40 90.9
17
Example Ranks
Beer Voter A Voter B Voter C
Heineken 2nd 3rd 1st
Grolsch 3rd 2nd 3rd
Hertog Jan 4th 4th 5th
Duvel 5th 5th 4th
Koninck 1st 1st 2nd
18
Example Ranks
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 ?
Grolsch 3 2 3 ?
Hertog Jan 4 4 5 ?
Duvel 5 5 4 ?
Koninck 1 1 2 ?
19
How to combine rankings?

• Several models are possible
• standard Borda take the average (best guess)
• also
• median rank (disregard outlying ranks)
• mode of ranks (plurality of ranks)
• min of ranks (optimistic)
• max of ranks (pessimistic)

20
standard Borda mean rank
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 2 ? 2nd
Grolsch 3 2 3 2.67 ? 3rd
Hertog Jan 4 4 5 4.33 ? 4th
Duvel 5 5 4 4.67 ? 5th
Koninck 1 1 2 1.33 ?1st
21
modal rank
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 2
Grolsch 3 2 3 3
Hertog Jan 4 4 5 4
Duvel 5 5 4 5
Koninck 1 1 2 1
22
min rank
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 1
Grolsch 3 2 3 2
Hertog Jan 4 4 5 4
Duvel 5 5 4 4
Koninck 1 1 2 1
23
min rank
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 1
Grolsch 3 2 3 2
Hertog Jan 4 4 5 4
Duvel 5 5 4 4
Koninck 1 1 2 1
How to solve ties?
24
max rank
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 3
Grolsch 3 2 3 3
Hertog Jan 4 4 5 5
Duvel 5 5 4 5
Koninck 1 1 2 2
25
How to solve ties in the combined Borda ranking?

• Random choice of candidates
• If the validity of the voters judgment is known
  take the rank of the best voter
• But then we digress towards knowledge-based and
  probabilistic schemes

26
Example non-stochastic tie solving Voter C is
known to be superior to A, B
Beer Voter A Voter B Voter C Combined
Heineken 2 3 1 1
Grolsch 3 2 3 2
Hertog Jan 4 4 5 4?5
Duvel 5 5 4 4?4
Koninck 1 1 2 1
27
How to choose for a combination method?

• mean? mode? median? min? max?
• Empirical tests are needed, mostly
• The type of question to be answered is important
• Example sportsperson of the year contest

28
How to choose for a combination method?

• The type of question to be answered is important
• Example sportsperson of the year contest

• Not the average rank over N sports for M
  sportspersons
• but the minimum rank (best played sport) is
  indicative