Morphology and Picture Relatedness

1
Morphology and Picture Relatedness

• Koji Miwa, Sally Rice, Gary Libben
• University of Alberta
• ACOL, October 21, 2006

2
Outline

• INTRODUCTION
• - Shared Constituency Phenomenon
• - Baayen Boroditsky (in press) study
• EXPERIMENT 1
• - Picture Comparison Task
• EXPERIMENT 2 3
• - Picture Comparison Task with Picture Naming
• - Word Image Comparison Task
• SUMMARY

3
Introduction Shared Constituency Phenomenon

• Translated words have different nuances of
  meaning
  
• e.g. the concept fireworks could be realized
  in different ways in different languages.
  
• fireworks (English)
• le feu dartifice performance-fire (French)
• hanabi flower-fire (Japanese)
• yànhuo flame-fire (Chinese)

4
Introduction Shared Constituency Phenomenon
ENGLISH
stilt
bamboo
fire
flower
bouquet
horse
bundle
work
stripe
zebra
JAPANESE
stilt
take bamboo
hana flower
uma horse
bouquet
hi fire
shima stripe
zebra
taba bundle
shi-goto work
5
Introduction Shared Constituency Phenomenon

• Baayen and Boroditsky (in press) study in a
  nutshell
  
• Stimuli Line-drawn images which are named by
  Dutch multimorphemic words and are related by
  shared constituency in Dutch (e.g. vrachtwagen
  kinderwagen)
• Participants 3 groups - native speakers of
  English, native speakers of Dutch, and bilinguals
  of English and Dutch.
  
• Procedure Presented pairs of 2 images
  simultaneously and asked the participants to rate
  the similarity on a 9-point scale.
  
• Results Morphological relatedness have influence
  on perceived picture similarity.

6
Introduction Shared Constituency Phenomenon

• Baayen and Boroditsky (in press) study in a
  nutshell
  
• Provides another line of support to Linguistic
  Relativity Hypothesis (LRH) by approaching the
  issue empirically (c.f. Lucy, 1997).
  
• Helps to study morphology in the lexicon from a
  more semantic perspective without the use of
  written stimuli (c.f. orthographical shape
  effects in lexical decision tasks, Beech
  Mayall, 2005).
• The present study applies the approach to
  Japanese and English morphology with some
  modifications in methodology.
  
• e.g. More ecological validity (use of picture
  stimuli)
  

7
Introduction Shared Constituency Phenomenon

• Different languages have different shared
  constituency patterning
  

Research question Is there shared constituency
effects on picture relatedness ratings?
Hypothesis 1 There is a shared constituency
effects on picture relatedness ratings.
Hypothesis 2 Head constituent have more
influence on the ratings than modifier
constituents
Other factors? Hypothesis 2 degree of word
activation in the mind influences the picture
relatedness ratings

8
Experiment 1 Picture Comparison Task

• Stimuli
• 40 pairs of target stimuli pictures which share
  a constituent in the same position in their
  Japanese translation equivalent, but not in
  English.
• 20 pairs of them share a constituent in the
  modifier position.
  
• The other 20 pairs share a constituent in the
  head position.
  
• 20 pairs of control stimuli morphologically
  unrelated.

9
Experiment 1 Picture Comparison Task

• Participants
• 20 native speakers of Japanese (experimental
  group)
  
• 20 native speakers of English (control group)

• Procedure
• Participants received audio instructions in
  Japanese and English
  
• Two pictures are presented on a computer screen
  simultaneously by the PsyScope.
  
• The orders of presentation were changed randomly
  for each participant.
  
• The participants then judged the similarity
  between the two presented pictures on a 9-point
  scale (1different, 9similar).

10
Experiment 1 Picture Comparison Task
3
2
1

chikyu ?? the Earth
chizu ?? map
11
Experiment 1 Picture Comparison Task

• Results
• JPN ratings ENGL ratings under experimental
  conditions.
  
• Different relatedness ratings on control stimuli

Task effect? Inherent difference?
12
Experiment 1 Picture Comparison Task

• Results
• Japanese speakers higher ratings under control
  condition is
  
• not due to a task effect.

13
Experiment 1 Picture Comparison Task

• Results
• The two groups ratings are higher under
  experimental conditions.
  
• Different relatedness ratings on control stimuli

Task effect? Inherent difference?
Two-factor ANOVA with repeated measures
F(2,76) 0.077, P 0.93
14
Experiment 1 Picture Comparison Task
Experiment 1 Picture Comparison
Experiment 2 Picture Comparison with Picture Na
ming
Experiment 3 Word Image Comparison
15
Experiment 2 3 Follow-up Experiments

• Stimuli
• The same as in Experiment 1

• Participants
• 20 native speakers of Japanese (experimental
  group)
  
• 20 native speakers of English (control group)

• Procedure
• Picture comparison with picture naming
• Participants named pairs of pictures and then
  rated the similarity.
  
• 2. Word Image comparison
• Pairs of word stimuli were presented, and
  participants rated
  
• the similarity of the objects expressed by words.

16
Experiment 2 3 Follow-up Experiments
Results
Experiment 3
Experiment 1
Experiment 2
Why is the baseline of comparison unstable ?
17
Experiment 2 3 Follow-up Experiments
Results Relatedness ratings on control stimuli in
Experiment 2
Japanese speakers ratings on control stimuli
declined over time.
18
Experiment 2 3 Follow-up Experiments
Results Relatedness ratings on control stimuli in
Experiment 3
Tendency was carried over from the preceding
Experiment 2
19
Experiment 2 3 Follow-up Experiments
Results
Experiment 3
Experiment 1
Experiment 2
F(2,76) 0.077 P 0.93
F(2,76) 11.375 P 0.0001
F(2,76) 3.289 P 0.05
20
Summary

• Contrary to Baayen and Boroditsky (in press)
  study, shared constituency phenomenon did not
  influence the target groups picture relatedness
  ratings significantly under normal picture
  comparison tasks.
• Picture perception does not lead to automatic
  word activation in the lexicon (or very weak
  activation).
  
• Picture perception could influence language
  performance (Ikeda, 1994), but not vice versa.
  Language influence on picture relatedness is
  conditional only when language is activated.
• Influences of shared constituency on picture
  relatedness ratings are correlated with the
  degree of target word activation.
  

21
References Acknowledgement

• Baayen, R. H., Boroditsky, L. (in press).
  Lexical effects on picture comparison. Elsevier
  Science.
  
• Beech, J. R., Mayall, K. A. (2005). The word
  shape hypothesis re-examined Evidence for an
  external feature advantage in visual word
  recognition. Journal of Research in Reading, 28,
  302-319.
• Cohen, J. D., MacWhinney, B., Flatt, M.,
  Provost, J. (1993). Psyscope A interactive
  graphic system for designing and controlling
  experiments in the psychology
• Ikeda, S. (1994). A study on the process of
  translating English and Japanese words An
  examination of interference and facilitation from
  pictures. The Japanese Journal of Psychology, 65,
  121-129.
• Lucy, J. A. (1997). Linguistic relativity. Annual
  Review of Anthropology, 26, 291-312.
  
• Tamaoka, K. Hatsuzuka, M. (1998). The effects
  of morphological semantics on the processing of
  Japanese two-kanji compound words. Reading and
  Writing An interdisciplinary Journal, 10,
  293-322.
• Special thanks to my academic supervisors, Dr.
  Gary Libben and Dr. Sally Rice
  
• for their kind and enduring assistance. My thanks
  extends to Dr. Bernard Linsky,
  
• who supported me to conduct the follow-up study
  through the Roger Smith
  
• Undergraduate Researchers Award. I appreciate all
  the technical and academic
  
• assistances from the Centre for Comparative
  Psycholinguistics, the University of
  
• Alberta. Finally, I would like to thank Dr. John
  Hogan for giving me this opportunity.