Perceptual Symbol Systems

1
Perceptual Symbol Systems

• L. W. Barsalou (1999), Brain and Behavior
  Science, 22, 577-660.

An overview of Barsalous Perceptual Symbol
System theory
James Sulzen June 6, 2001
Psychology 264 Gordon Bower Spring 2001
2
Terminology

• P-state - Perceptual state
• A combination of multimodal percepts constituting
  an experience or some aspect of perception of the
  real world.
• P-sym (or p-symbol) - Perceptual symbol
• Recalled or constructed subset of p-states which
  symbolically stands for a referent of some sort
• PSS - Perceptual Symbol System
• System of symbols and processes which operate on
  them to produce cognitive processes.
• Frame
• Simulation

3
Amodal Symbol Systems

• Amodal systems transduce p-states into amodal
  equivalents.

4
Amodal vs. Modal Issues

• Evidence Little/no direct evidence for amodal
  systems
• Neuroscience Psychology Much evidence for
  modal processing
• Awkwardness Certain computations are very
  amodally awkward (i.e. spatio-temporal)
• Transduction and symbol grounding Amodal
  systems are essentially arbitrary.
  (Grounding/associating amodal representations
  back to perceptual ones just ultimately rates to
  make the amodal systems redundant.)
• Too much power Amodal systems ultimately are too
  powerful They can explain anything. Modal
  systems have a priori limitations (modal ones do
  not). Modal systems are open to falsifiability
  and provocative hypothesis

5
PSS Core Properties

• P-syms - Stand in as referents
• Frames - These combine p-syms frames with
  relationships
• Simulators - dynamically bind other elements
• Language - Can stand for or drive other elements

6
PSS Core Properties

• P-syms
• Have a neural representation / substrate
• Are schematic representations of p-states (see
  Fig. 1)
• Arise from repeated exposure neuronal
  recruitment / exclusion
• Are inherently multimodal (including
  proprioception and introspection)
• Selective attention operates to extract p-sym
  attributes
• Frames
• Organize p-syms (i.e., define primitive
  relationships - up/down, in/out)
• Combine together
• Predicates
• Attribute-value bindings
• Constraints
• Recursion

7
Multimodal Organization of Knowledge
Sensory registers
V- vision G - Gustatory H - Haptic K -
Kinesthetic proprioceptive O -
Olfactory A - Auditory L - Language S -
Spatial E - Emotional Other systems certainly
exist
- Each modality can be thought of as a
representational system - Each provides certain
affordances
8
Frames
Establishing an initial frame for car after
processing a first instance.
Evolution after processing a second instance.
Creating a simulation of second instance from
frame in B
9
Simulators

• Simulators
• This is an area requiring more development in
  the theory
• Frames a set of p-syms, frames, simulators to
  create higher-level structures
• Temporally dynamic integration of other elements
• Can be dynamically constructed, modified,
  componentized, and executed
• Comparable to mental models, schema, concepts,
  etc.
• Simulation
• This is the execution of a simulator
• Are always sketchy and incomplete - are never
  veridical
• Idealization occurs (i.e., Gestaltist principles
  apply)
• Example Categorization - if a category
  simulator can produce a satisfactory simulation
  of a perceived entity, then the entity belongs in
  the category (p. 587)
• Derived properties of simulators
• Productivity Can be combinatorially and
  recursively combined
• Propositions Can be bound to individual entities
  framing?
• Variability Implement variable embodiment
• Abstraction Can combine physical and
  introspective events to represent abstractions

10
Simulator Productivity
Object categories
Spatial relationships
Combinatorialproductivity
Recursiveproductivity
11
Prop- ositions
Proposition representation (the balloon is above
the cloud)
Complex hierarchicalproposition (balloon above
cloud)
Alternative proposition (balloon below cloud)
simulators
simulations
Perceived situations
12
Language

• Can be linked any other element (p-sym, frame,
  simulator)
• Can be used to construct and control simulators
• Language is probably the amodal basis of
  cognition and memory?

13
PSS Symbol Manipulation

• Barsalou essentially shows that p-syms, frames,
  simulators, etc can
• Be composed, associated, combined, subtracted,
  and so on
• They constitute a symbol manipulation system
• He does not demonstrate a formal correspondence
  between his PSS and an amodal symbol system(such
  as a semantic knowledge network)
• Such a demonstration would constitute a form of
  completeness proof
• Language as a representational system and can
  serve the role of an amodal symbol system

14
Abstractions

• Metaphor (anger gt liquid exploding out of a
  container)
• Not adequate in of itself to represent all
  abstractions
• Abstraction are constructed from three main
  elements
• Framing Are framed against the background of a
  simulation
• Selectivity Selective attention highlights the
  core content of the concept in the context of the
  simulation
• Introspective symbols (i-syms) These are central
  to representing abstract concepts (i-syms are the
  internal sense of an experience or internal
  perception)
• Methodology for identifying abstractions
• Find a frame that characterizes the abstraction
• Identify seeming p-syms and i-syms
• Identify the focal element that characterizes the
  abstraction

15
Truth Falsity
Mapping succeedstruth

• Simulated event seq. frames the concept
• The abstraction is only a focal part of the
  simulation (i.e., the outcome that construes the
  concept)
• Introspective symbols are central to the
  construing of the meaning of the concept
• After many mappings, a simulator develops for
  truth

Mapping failsfalsity

After many mappings abstractionsimulatorbecomes
established
16
Anger

• Anger involves
• Appraisal of an initiating event, and that the
  agents goal is blocked by the event
• Intense affective states
• Behavioral responses (i.e., disapproval, revenge,
  and redirecting goals)
• Core component Blocked goal
• A goal is a simulated state that the agent
  desires to achieve
• A blocked goal is a failed mapping when it is
  expected to successfully map
• Similar to falsity
• A simulated situation fails to map to a perceived
  situation
• Except that affective states and behavioral
  responses are also associated with anger
• Lie
• A statement induces a simulation purported to be
  true that is actually not(i.e., simulation is
  negative in liars simulation, but false in the
  deceiveds)

(P. 602)
17
Disjunction
Attempted reconstruction
Partial recall
Original event
18
Transformations
19
Ad hocCategories