Progress on the Structure-Mapping Architecture for Learning

1
Progress on the Structure-Mapping Architecture
forLearning

• Dedre Gentner
• Kenneth D. Forbus
• Northwestern University

2
Symbolic modeling crucial for understanding
cognition

• Heavy use of conceptual knowledge is a signature
  phenomena of human cognition
• People understand, make, compare, and learn from
  complex arguments
• People learn conceptual knowledge from reading
  texts, and apply what they have learned to new
  situations
• People reason and learn by analogy, applying
  precedents and prior experience to solve complex
  problems
• People use symbolic systems (e.g., language,
  maps, diagrams)
• Symbolic models remain the best way to explore
  many conceptual knowledge issues

3
Overview

• Structure-Mapping Architecture
• Accelerating learning via analogical encoding
• Brief review
• Tacit analogical inference
• Analogy on the sly
• Similarity-based qualitative simulation
• Transfer and outreach activities

4
Structure-Mapping Theory (Gentner, 1983)

• Analogy and similarity involve
• correspondences between structured descriptions
• candidate inferences fill in missing structure in
  target
• Constraints
• Identicality Match identical relations,
  attributes, functions. Map non-identical
  functions when suggested by higher-order matches
• 11 mappings Each item can be matched with at
  most one other
• Systematicity Prefer mappings involving systems
  of relations, esp. including higher-order
  relations

Candidate Inference completes common structure
Inferenceis selective. Not all base knowledge is
imported
5
Functional Overview
Similarity-based retrieval of relevant examples
and knowledge
Analyzing similarities and differences, reasoning
from experience, applying relational knowledge
Potentially relevant precedents
SEQL
SME
MAC/FAC
Incrementally constructs generalizations,
producing human-like relational abstractions
within similar number of examples.
Long-term memory
6

• Psychological Studies
• Case-comparison method
• Previous work Transfer
• New work Learning of principles
• 2. Unaware analogical inference
• Previous work Unaware inference
• New work Attitude congeniality unaware
  inferences
• New work Unaware alignment-based decision
  making

7

• Analogy
• Core process in higher-order cognition
• A general learning mechanism by which complex
  knowledge can be acquired
• e.g., causal structures explanatory principles
  
• Unique to humans (or nearly so)

Similarity Species-general
Analogy Species-restricted
8
Analogical Encoding in Learning
Standard analogical learning

• Analogy can promote learning
• Induces structural alignment
• Generates candidate inferences

Inferences
FamiliarSituation
NewSituation

• But, memory retrieval of potential analogs is
  unreliable
• Inert knowledge Learned material often
• fails to transfer to new situations

Analogical encoding

• Solution Analogical encoding
• Use comparison during learning to
• - highlight the common relational
  system
• - promote relational abstraction
  transfer

NewSituation
RelationalSchema
New Situation
Compare
NewSituation
9
Case Comparison Method in Learning to
Negotiate  Studies of MBAs learning negotiation
strategies Students study two analogous cases
prior to negotiating Loewenstein, Thompson
Gentner, 1999 Thompson, Gentner Loewenstein,
2000 Gentner, Loewenstein Thompson, 2003
case 1
case 2
Separate Cases Condition Read each case, write
principle and give advice.
Comparison Condition Compare the two cases and
write the commonalities
Simulated Negotiation

On a new analogous case
10
Negotiation transfer performance across three
studies Proportion using strategy exemplified in
the cases
.8
.7
.58
.6
.5
.4
Prop. Forming Contingent Contracts
.24
.3
.19
.2
.1
0
Separate Cases N83
Compare N81
No Cases N42
11
Better schemas ? Better transfer
.8
.7
Compare
.6
.5
Prop. Forming Contingent Contracts
.4
Separate Cases
.3
.2
.1
0
0
0.5
1.0
1.5
2.0
Dyadic Schema Rating
So, what happens if we just give them the
principle?
12
Aligning case and principle improves ability to
use principle in transfer
.7
Separate Compare Principle Principle Plus
case and Case
.6
Case 1________________ Case 2 ___________ ______
_____
.44
.5
.4
Prop. Forming Contingent Contracts
.3
.19
Test Face-to-face negotiation
.2
.1
0
Separate Cases N26 dyads
Compare N27 dyads
Error bars assume binomial with prop.19
(baseline)
13
Comparison promotes transfer even when the
principle is given - Why?

• Principles utilize abstract relational language
• Relational languageverbs, prepositions,
  relational nounsis contextually mutable ?
  interpretation difficulties
• e.g., force in physics / force in commonsense
  language
• Assembling a complex relational structure is
  errorful
• So, beginning learners dont understand
  principles when presented solo
• Case provides a firm relational structure that is
  correct but overly specific
• learning is context-bound strongly situated
• So unlikely to transfer
• Comparing a principle and a case
• grounds the principle in a firm structure
• invites abstracting the specific relations in the
  case

14
Learning Negotiation Principles- Experiment 1

• Training
• participants read two passages
• a negotiation principle (Contingent Contract)
• an analogous case
• Separate condition Participants consider each
  passage separately.
• Compare condition Participants consider how the
  case and principle are alike.
• Two orders case?ppl and ppl?case
• All participants answer the question
• "How could this be informative for negotiating?"
• 20-minute delay
• Test Recall task subjects write out the
  principle they learned

15
Principle Contingent Contract

• A contingent contract is a contract to do or not
  to do something depending on whether or not some
  future event occurs. At least two kinds of
  situations exist in which contingent agreements
  add potential for joint gains when disagreeing
  over probabilities and when both parties try to
  influence an uncertain outcome. When the
  uncertain event itself is of interest, there are
  familiar economic contingent contracts with
  betting based on the probability of
  differences. Parties are dealing with uncertain
  quantities and actually or apparently differ in
  their assessment, and here contingent
  arrangements offer gains. When the parties feel
  capable of influencing an uncertain event, making
  the negotiated outcome dependent on its
  resolution may be a good idea. In both cases of
  course, contingent arrangements based on
  underlying differences are not a panacea.
  Crafting them effectively can be a high art. And
  once the outcome of the uncertain event is known,
  one party may have won and the other lost.
  Whether the outcome will then be considered fair,
  wise, or even sustainable is an important
  question to be planned for in advance.

16
Training Case

• Two fairly poor brothers, Ben and Jerry, had
  just inherited a working farm whose main crop has
  a volatile price. Ben wanted to sell rights to
  the farms output under a long-term contract for
  a fixed amount rather than depend upon shares of
  an uncertain revenue stream. In short, Ben was
  risk-averse. Jerry, on the other hand, was
  confident that the next season would be
  spectacular and revenues would be high. In
  short, Jerry was risk-seeking. The two argued
  for days and nights. Ben wanted to sell
  immediately because he believed the price of the
  crop would fall Jerry wanted to keep the farm
  because he believed the price of the crop would
  increase.
• Finally, Jerry proposed a possible agreement to
  his brother They would keep the farm for
  another year. If the price of the crop fell
  below a certain price (as Ben thought it would),
  then they would sell the farm and Ben would get
  50 of the farms current value, adjusted for
  inflation Jerry would get the rest. However,
  if the price of the crop were to rise (as Jerry
  thought it would), Jerry would buy Ben out for
  50 of the farms current value, adjusted for
  inflation, and would get to keep all of the
  additional profits for himself. Jerry was
  delighted when his brother told him he could
  agree to this arrangement, thereby avoiding
  further conflict.

17
Recall Scores (Max. 8)
Gentner Colhoun
Mean Recall Score
3.4
2.5
(26) (26)
Two blind raters Agreement 94
t(50) 2.10, p 0.041
18

• Quotes from the Compare Group
• P18 "Contingent contract Principle if there is
  an uncertain event occurring in the future which
  two parties disagree on, the outcome of this
  event becomes the determining factor in the
  outcome of the negotiation."
• P30 "The contingency contract is created as an
  agreement to do/not do something in the future in
  the event of a situation. As the future is
  unknown, the CC is created on the probability
  that something will occur"

• Quotes from the Separate Group
• P50 "It is important to consider how much you
  will lose or win when betting on an uncertain
  event. Negotiating in this situation is more
  complicated than just predicting the outcome."
  (this was the entire answer)
• P51 "We read about the two poor brothers on the
  farm. One was risk-seeking and the other was
  risk-seeking, so they couldn't decide on whether
  or not to sell the farm" (no mention of the
  principle)

19
Read Principle Case(20 mins)Immediate
Recall Test case Asian MerchantN14 7 sep, 7
comp(4 days)Long term Recall New test
caseN14 7 sep, 7 comp
Delayed Recall
20
Immediate Recall Scores Both Orders
Mean Recall Score
4.4
3.1
(11) (12)
T(23) -2.44, p 0.023
21
Delayed Recall Scores Both Order
Mean Recall Score
4.3
3.1
(11) (12)
T(21) -1.91, p 0.07
Combines two groups with slightly different
procedures
22
Conclusions Comparison group gt Separate
group Case-first groups gt Principle-first groups

• Comparing case and principle greatly benefits
  comprehension of principle
• The case provides firm relational structure
• and a clear (though overly specific)
  interpretation of the relational terms
• Comparing case with principle prompts
  re- representation and abstraction of the
  relational structure

23
Practical Implications

• Case-based training is heavily used in
  professional schools (business, medicine, law)
  intensive analyses of single cases
• Our results suggest that learning could be
  greatly increased by changing to a
  comparison-based instructional strategy
• Based on our findings, some institutions are
  revising their instructional methods
• Medical School of McMaster University
• Developing a new curriculum relying heavily on
  comparison-based instruction
• Harvard Business School
• Exploring comparison-based method
• CMU discussions with Marsha Lovett

24
Unaware Analogical Inference
Analogy as generally conceived
Current Studies

• Conscious
• Discerning
• Deliberate
• Effortful

• Non-aware
• Oblivious
• Non-deliberate
• Accidental

Suggestive evidence Blanchette Dunbar, 2002
Moreau, Markman Lehman, 2001
25

• New thrust Study of unwitting analogy
• Can analogical inferences occur without awareness
  of
• making the inferences?
• Can analogical inferences occur without awareness
  of
• the analogy itself?
• Can the highlighting effect of analogical
  alignment
• Influence future decision-making?

26

• Analogical insertion effect
• believing that the analogical inference from
• B?T actually occurred in T
• Evidence for analogical insertion
• Blanchette Dunbar 1999
• Analogy Anti-marijuana laws are like
  Prohibition
• Participants misrecognized parallel inferences
  as having occurred
• in marijuana passage
• But, these pro-marijuana inferences were likely
  to be
• congenial to college students
• Will analogical insertion occur if the inference
  is not
• so congenial?

27
Attitudes towards gayness assessed (Mass testing)
3-4 weeks (unrelated context)
Read paragraph Is it OK to be gay
15-min filled delay
Old-New recognition test
Rate soundness of analogy
Perrott, Gentner Bodenhausen, 2005
28
Perrott, Gentner Bodenhausen, 2005
Proportion old responses

Condition(2) X Item type(4) F(3, 228) 4.97, p
.002, MSE .048
29

• Results within analogy group
• Attitudes towards gays within predicted the rated
  soundness of the analogy
• But
• Likelihood of analogical insertion was not
  predicted by rated soundness of the analogy
• Even more surprisingly,
• Likelihood of analogical insertion was not
  predicted by attitude towards gays No attitude
  congeniality effect
• Attitudes measured on 15-item questionnaire ?
  composite scale from
• 1 (very negative) to 7 (very positive). Range
  1.8-6.8 (M 4.7)
• Cutoffs for lower and upper quartiles 3.3 and
  5.8

30
Can analogical insertion occur without awareness
of the analogy

• Participants read a series of passages
• Told that they would be asked questions about
• content of passages
• We observed extent to which analogous
• passages early in the set influenced the
  
• interpretation of later passages

• No goal other than comprehension
• Inferences support understanding the input

• Question can analogical inferences occur
  without awareness?

31
Day Gentner 2003, in prep
Current Studies

• Participants read a series of passages
• Some early passages are relationally similar to
  later
• passages
• Will participants use structure-mapping in
• interpreting the later passages?

• TEST
• Participants answer TF questions about passages
• Dependent measure Answering True to questions
  that are inferences from earlier analogous
  passages.

32
Experiment 1
Two versions of each base passage

• If participants use analogical inference from the
  earlier similar base passage,
• they will understand the target differently,
  depending on which base version they got.

Target has some ambiguous portions
33
Example Source Passages
34
Expt. 1 Results More false recognitions for
base-consistent statements
100
73
Percentage yes responses
50
25
0
Base-consistent
Base-inconsistent
Using base consistency as a within-subjects
factor
Day Gentner, 2003
t (19) 4.79, p lt .001
35
E1
Results Analogical insertion

• Ps interpreted the ambiguous portion of the
  target in a manner consistent with structurally
  matching information in the base.
• The same target passage was interpreted
  differently, as a function of which base Ps had
  read
• Evidence suggests that analogical inference
  influences the interpretation of new material
• Not due to deliberate strategies
• 90 noticed similarities between passages
• But, 80 said all passages were understandable
  on their own.
• Not due to simple priming further study showed
  inferences are specific to the structural role of
  the inserted information

36
Experiment 3
Is the analogical insertion effect occurring
during online comprehension of target, or is it a
later memory error?
37
Experiment 3 Self-paced Reading Task
George's absence from the service was
conspicuous, especially since he had been seen
around his uncle's estate prior to his death, and
the police soon found out about his flight to
Rio.
If Ps insert the seeded inference into the
target story, they will take longer to read the
key test sentence when it is inconsistent with
that inference
38
Experiment 3 Self-paced Reading Task
George's absence from the service was
conspicuous, especially since he had been seen
around his uncle's estate prior to his death, and
the police soon found out about his flight to
Rio.
If Ps insert the seeded inference into the
target story, they will take longer to read the
key test sentence when it is inconsistent with
that inference
F (1,19) 6.81, p lt .05
39
Tacit analogical inferences
Day Gentner 2003, in prep

• People interpolated analogical inferences from a
  prior similar passage
• due to shared representational structure, not
  simply to general priming
• Implication Structure-mapping can operate in
  nonaware, non-deliberative processing
• But what about large number of analogy studies
  that show failure to transfer ?

Current studies

• Vary delay 20 minute vs. 4 days later
• Vary surface similarity between the passages
• Future work Progressive alignment effect? Does
  an obvious alignment potentiate more analogical
  creep?

40
Day Bartels (2005)
Unaware effects of analogy Decision-making
Structure mapping theory proposes that comparison
involves the alignment of representational
structures (Gentner, 1983 Gentner Markman,
1997)
This implies two kinds of differences
alignable differences different values on same
predicate or dimension related to common
structure non-alignable differences none of
the above
Alignable differences are weighted more heavily
in perceived similarity (Markman Gentner,
1996) difference detection (Gentner Markman,
1994) recall (Markman Gentner,
1997) preference (e.g., Roehm Sternthal, 2001)
Hypotheses Alignment along a dimension renders
that dimension more salient in immediate
use Repeated alignment use renders the
dimension more salient in future encodings
41
Method Ps choose among portable digital video
players

• First, participants gave preference ratings for
  models that
• varied on only one alignable dimension

Model B
Model A
Firewire and USB connectivity Battery
life Voice recorder Hard drive
capacity Built-in FM radio Wireless projection
range Support for WMV and MP2 formats Screen
size Weight
Yes 4 hr No 7 Gb Yes 12 ft No 2.5 in 10 oz
Yes 4 hr No 4 Gb Yes 12 ft No 2.5 in 10 oz
Strongly prefer Model A
Strongly prefer Model B
42
Method

• First, participants gave preference ratings for
  models that
• varied on only one alignable dimension

Model B
Model A
Firewire and USB connectivity Battery
life Voice recorder Hard drive
capacity Built-in FM radio Wireless projection
range Support for WMV and MP2 formats Screen
size Weight
Yes 4 hr No 7 Gb Yes 12 ft No 2.5 in 10 oz
Yes 4 hr No 4 Gb Yes 12 ft No 2.5 in 10 oz
Strongly prefer Model A
Strongly prefer Model B
43
Method
2. Eventually, they make judgments between
models varying on two dimensions, each favoring
a different alternative
Model B
Model A
Firewire and USB connectivity Battery
life Voice recorder Hard drive
capacity Built-in FM radio Wireless projection
range Support for WMV and MP2 formats Screen
size Weight
Yes 4 hr No 10 Gb Yes 12 ft No 1.5 in 10 oz
Yes 4 hr No 7 Gb Yes 12 ft No 2.5 in 10 oz
Strongly prefer Model A
Strongly prefer Model B
44
Experiments
Experiment 1

• Are more recently used dimensions weighted more
• in future decisions?
• That is, does aligning a dimension make it more
  salient for some period of time?

Experiment 2

• Are dimension that have been used more
  frequently weighted
• more in future decisions?
• That is, does repeated alignment along a
  dimension
• render that dimension more salient in future
  encodings?

45
Types of item series
1 v. 2 back
1 back
Diagnostic dimension
Diagnostic dimension
A - -
B - -
C - - -
D - - - -
E - - - -
A -
B - -
C - -
D - - -
E - - -
2 v. 3 back
1 v. 3 back
Diagnostic dimension
Diagnostic dimension
A - - -
B - - -
C - - - -
D - - - -
E - - - - -
A - - -
B - - -
C - - - -
D - - - -
E - - - - -
46
Day Bartels (2005)
Results
Experiment 1

• Each response was coded as a value between 0
  and 1
• .5 would be chance averages closer to 1
  indicate a
• preference for the more recently diagnostic
  dimension

Average response was .62 (p lt .001) 18 out of
20 participant had average ratings greater than .5
Participants weighted a dimension more if it had
been used in a more recent decision
47
Day Bartels (2005)
Results
Experiment 2

• Found correlation between preference ratings
  and number
• of prior uses of a dimension for each
  participant
• Individual correlations transformed into
  Fishers Z for use
• in analysis

Average transformed correlation was .20 (p lt .01)
Participants weight a dimension more if it had
been used more frequently in prior decisions
48
Day Bartels (2005)
Conclusions

• Finding an alignable difference along a
  dimension makes that dimension more salient for a
  period of time
• more recently aligned dimensions play a larger
  role in future decisions

• Repeated alignment of a dimension increases its
  salience in future encodings
• higher numbers of repetitions ? greater
  dimension weights in decisions

• These effects of comparison may go unnoticed,
  but may have pervasive effects on the mental
  landscape

49
Resistance is futile

• Analogical insertioninterpolation of inferences
  into the target situationcan occur
• when an analogy is given explicitly
• when an alignable analog has been presented
  recently
• Online comparisons increase the salience of
  aligned dimensions for future encodings
• Hypothesis Continual subtle learning occurs
  via structural matching and inference
• Fits with MAC/FAC assumption of continual
  unbidden retrieval
• Challenges Future work
• How recent?
• How similar and in what ways?
• Effects of intervening items?

50
How do people do common sense reasoning?

• Todays methods of qualitative reasoning are very
  useful
• Many successful applications in engineering,
  education, supporting scientific reasoning
• Are they also good models of how people common
  sense reasoning?
• Yes, but similarity plays major role in reasoning
• Important question for cognitive science
• Central to understanding mental models

51
The standard Qualitative Reasoning community
answer
Scenario model
Situation description input
?
?
?
?
Qualitative Simulator
Model Builder
Qualitative simulation
F G H
i
F ? G ? H
F ? G H
F G ? H
1st principles Domain Theory
i
i
F ? G ? H
F ? G ? H
F G H
52
First-principles qualitative simulation
Useful properties
Problematic properties

• Handles incomplete and inexact data
• Supports simple inferences
• Explicit representation of causal theories
• To prevent melting, remove kettle from stove
• Representation of ambiguity
• We easily imagine multiple alternatives in daily
  reasoning

• Exclusive use of 1st-principles domain theory
• inconsistent with psychological evidence of
  strong role for experience-based reasoning
• Exponential behavior
• inconsistent with rapidity flexibility of human
  reasoning
• Generates more complex predictions than people
  report
• logically possible, but physically implausible

53
Working hypotheses about human common sense
reasoning and learning(Forbus Gentner, 1997)

• Common sense Combination of analogical
  reasoning from experience and first-principles
  reasoning
• Within-domain analogies provide robustness, rapid
  predictions
• Human learning requires accumulating lots of
  concrete examples
• Structured, relational descriptions essential
  feature vectors inadequate
• First-principles reasoning emerges slowly as
  generalizations from examples
• Human learning tends to be conservative
• But human learning also tends to be faster than
  pure statistical learning
• Qualitative representations are central
• Appropriate level of understanding for
  communication, action, and generalization

54
An alternative Hybrid qualitative simulation

• Most predictions, explanations generated via
  within-domain analogies
• Provides rapidity and robustness in common cases
• Multiple retrieved behaviors leads to multiple
  predictions.
• Logically possible behaviors that are rarely
  observed arent predicted.
• 1st principles reasoning relatively rare
• 1st principles domain theories fragmentary,
  partial
• Some 1st principles knowledge created by
  generalization over examples
• Much of it taught via language
• We built a similarity-based qualitative simulator
  to explore this approach

55
A Prototype SQS System
Situation
Rerep Engine
MAC/FAC
Candidate Behaviors Projector
Predictions
ExperienceLibrary
SEQL
56
Experience Library Contents

• Current sources
• Classic QR examples
• Generated envisonments using Gizmo Mk2
• Feedback systems
• Generated descriptions of behavior by hand
• Test of whether system can operate without a
  complete 1st-principles domain theory
• Each case consists of a qualitative state
• Individuals, ordinal relations, model fragments
• Concrete information about entities (stand-in for
  perceptual properties)
• Description of transitions to other states

57
Example Two Containers Liquid Flow
58
Input Scenario
?(AmountOf Water Liquid Beaker) ?(AmountOf Water
Liquid Vial) ?(Pressure Wb) ?(Pressure Wv) (gt
(Pressure Wb) (pressure Wv))
?(AmountOf Water Liquid Beaker) ?(AmountOf Water
Liquid Vial) ?(Pressure Wb) ?(Pressure Wv) (
(Pressure Wb) (pressure Wv))
Behavior Prediction
59
Example Heat Flow
Input Scenario
60
Example Discrete action feedback system
61
Mappings for Feedback Example
62
Stored Feedback System Behavior
Retrieved Behavior
63
Mapped Feedback System Behavior
Retrieved Behavior
Predicted Behavior
64
Example Proportional action control system

• Amount of correction applied is proportional to
  the error signal
• SQS prototype with current library makes
  incorrect prediction
• Retrieves discrete-action controller behavior
• Currently has no means of detecting
  inconsistencies
• Possible solutions
• Include some first-principles reasoning for
  reality checks
• When failure detected, add new behavior to
  Experience Library to improve future performance

65
Current Issue Combining Behaviors
Two mappings, how to combine?
66
Pastiche Mappings

• Retrieve behaviors for unexplained parts of
  system
• Combine by re-evaluating closed-world assumptions

Perform influence resolution to combine
influences across cases
67
Next steps Hybrid qualitative simulation

• Significantly expand Experience Library
• Plan Use EA NLU system to describe qualitative
  states in QRG Controlled English
• Test skolem resolution strategies
• Identify hypothesized entities with unmapped
  current situation entities when possible.
• Formulate criteria for using multiple remindings
• When to generate alternate predicted behaviors?
• Develop more selective rerepresentation
  strategies
• Currently performed exhaustively
• Explore learning strategies
• Store rerepresented results and new behaviors
• Use SEQL to construct generalizations

68
Geometric Analogy Problems

• Evans classic 1968 work ANALOGY
• Miller Analogies Test geometric problems
• Non-trivial human intelligence task
• Goal of our simulation
• Show that general-purpose simulations can handle
  this task
• Another source of data for tuning visual
  representations in our sketching system

69
Sketching the Geometric Analogy Problems
70
Finding the Answer Evans

• A is to B as C is to 1, 2, 3, 4 or 5?
• Compute all transformations AgB, Cg1, Cg2,
• Search for best match between transformation for
  AgB with all of the transformations for Cg1, Cg2,
  

71
Finding the Answer Our simulation

• A is to B as C is to 1, 2, 3, 4 or 5?

Differences compared at second level
Two-stage structure mapping
72
Results(Based on Evans answer key)
73
Problem case 12
74
Summary Geometric Analogies Simulation

• SME qualitative spatial representations provide
  a basis for solving geometric analogy problems
• Two-stage structure mapping provides an elegant
  model for this task
• Explicit transformation rules unnecessary
• Applicable to other analogy tasks?

75
Future Directions
76
THE END