Representation, Development and Disintegration of Conceptual Knowledge: A ParallelDistributed Proces

1
Representation, Development and Disintegration of
Conceptual KnowledgeA Parallel-Distributed
Processing Approach

• James L. McClelland
• Department of Psychology andCenter for Mind,
  Brain, and ComputationStanford University

2
Parallel Distributed Processing Approach to
Semantic Cognition

• Representation is a pattern of activation
  distributed over neurons within and across brain
  areas.
• Bidirectional propagation of activation underlies
  the ability to bring these representations to
  mind from given inputs.
• The knowledge underlying propagation of
  activation is in the connections.

3
A Principle of Learning and Representation

• Learning and representation are sensitive to
  coherent covariation of properties across
  experiences.

4
What is Coherent Covariation?

• The tendency of properties of objects to co-occur
  in clusters.
• e.g.
• Has wings
• Can fly
• Is light
• Or
• Has roots
• Has rigid cell walls
• Can grow tall

5
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development of
  conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of semantic disintegration
  seen in semantic dementia.

6
Some Phenomena in Development

• Progressive differentiation of concepts
• Overgeneralization
• Illusory correlations

7
(No Transcript)
8
The Rumelhart Model
9
The Training Data
All propositions true of items at the bottom
levelof the tree, e.g. Robin can grow, move,
fly
10
Target output for robin can input
11
Forward Propagation of Activation
12
Back Propagation of Error (d)
aj
wij
ai
di Sdkwki
wki
dk (tk-ak)
Error-correcting learning At the output
layer Dwki edkai At the prior layer Dwij
edjaj
13
(No Transcript)
14
(No Transcript)
15
Early Later LaterStill
Experie nce
16
(No Transcript)
17
What Drives Progressive Differentiation?

• Waves of differentiation reflect coherent
  covariation of properties across items.
• Patterns of coherent covariation are reflected in
  the principal components of the property
  covariance matrix.
• Figure shows attribute loadings on the first
  three principal components
• 1. Plants vs. animals
• 2. Birds vs. fish
• 3. Trees vs. flowers
• Same color features covary in
  component
• Diff color anti-covarying
  features

18
Properties Coherent Incoherent
CoherenceTraining Patterns
is can has is can has
Items
No labels are provided Each item and each
property occurs with equal frequency
19
Effect of Coherence on Representation
20
Overgeneralization of Frequent Names to Similar
Objects
goat
tree
dog
21
(No Transcript)
22
Illusory Correlations

• Rochel Gelman found that children think that all
  animals have feet.
• Even animals that look like small furry balls and
  dont seem to have any feet at all.
• A tendency to over-generalize properties typical
  of a superordinate category at an intermediate
  point in development is characteristic of the PDP
  network.

23
A typical property thata particular object
lacks e.g., pine has leaves
An infrequent, atypical property
24
Sensitivity to Coherence Requires Convergence
A
A
25
Another key property of the model

• Sensitivity to coherent covariation can be
  domain- and property-type specific, and such
  sensitivity is acquired as differentiation
  occurs.
• Obviates the need for initial domain-specific
  biases to account for domain-specific patterns of
  generalization and inference.

26
Differential Importance (Marcario, 1991)

• 3-4 yr old children see a puppet and are told he
  likes to eat, or play with, a certain object
  (e.g., top object at right)
• Children then must choose another one that will
  be the same kind of thing to eat or that will
  be the same kind of thing to play with.
• In the first case they tend to choose the object
  with the same color.
• In the second case they will tend to choose the
  object with the same shape.

27
Adjustments to Training Environment

• Among the plants
• All trees are large
• All flowers are small
• Either can be bright or dull
• Among the animals
• All birds are bright
• All fish are dull
• Either can be small or large
• In other words
• Size covaries with properties that differentiate
  different types of plants
• Brightness covaries with properties that
  differentiate different types of animals

28
Testing Feature Importance

• After partial learning, model is shown eight test
  objects
• Four Animals
• All have skin
• All combinations of bright/dull and large/small
• Four Plants
• All have roots
• All combinations of bright/dull and large/small
• Representations are generated by
  usingback-propagation to representation.
• Representations are then compared to see which
  animals are treated as most similar, and which
  plants are treated as most similar.

29
(No Transcript)
30
(No Transcript)
31
Similarities of Obtained Representations
Brightness is relevant for Animals
Size is relevant for Plants
32
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development of
  conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of semantic disintegration
  seen in semantic dementia.

33
Disintegration of Conceptual Knowledge in
Semantic Dementia

• Progressive loss of specific knowledge of
  concepts, including their names, with
  preservation of general information
• Overgeneralization of frequent names
• Illusory correlations

34
Picture namingand drawing in Sem. Demantia
35
(No Transcript)
36
Grounding the Model in What we Know About The
Organization of Semantic Knowledge in The Brain

• There is now evidence for specialized areas
  subserving many different kinds of semantic
  information.
• Semantic dementia results from progressive
  bilateral disintegration of the anterior temporal
  cortex.
• Rapid acquisition of new knowledge depends on
  medial temporal lobes, leaving long-term semantic
  knowledge intact.

language
37
Proposed Architecture for the Organization of
Semantic Memory
name
action
motion
Temporal pole
color
form
valance
38
Rogers et al (2005) model of semantic dementia

• Gradually learns through exposure to input
  patterns derived from norming studies.
• Representations in the temporal pole are acquired
  through the course of learning.
• After learning, the network can activate each
  other type of information from name or visual
  input.
• Representations undergo progressive
  differentiation as learning progresses.
• Damage to units within the temporal pole leads to
  the pattern of deficits seen in semantic dementia.

39
Errors in Naming for As a Function of Severity
Simulation Results
Patient Data
Severity of Dementia
Fraction of Neurons Destroyed
40
Simulation of Delayed Copying

• Visual input is presented, then removed.
• After several time steps, pattern is compared to
  the pattern that was presented initially.
• Omissions and intrusions are scored for typicality

41
IFs camel
DCs swan
Simulation results
42
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development of
  conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of semantic disintegration
  seen in semantic dementia.

43
(No Transcript)
44
Sensitivity to Coherence Requires Convergence
A
A