Brain activations related to hearing and seeing an action

1
Brain activations related to hearing and seeing
an action
Jonas T. Kaplan Marco Iacoboni
UCLA Ahmanson-Lovelace Brain Mapping Center
Results Consistent with previous research,
seeing an action video produced greater signal
changes in premotor cortex, the intraparietal
region, and the inferior frontal gyrus compared
with seeing the control video.  Hearing an
action sound compared with hearing the control
sound yielded greater signal changes in the
inferior and middle frontal gyri, and in the
superior temporal gyrus
Background  Action observation has been shown to
activate the premotor cortex in monkeys and in
humans, suggesting that premotor cortex may
support an action representation that abstracts
across self and other1. Mirror neurons have
been identified in monkey premotor cortex that
respond not only to the sight of an action, but
to the sound of the action2,3.  Premotor cortex
has multimodal responsivity4 and may participate
in an integrative, abstract action representation
that combines across the senses as well as across
actors. To investigate multimodal integration
in action perception, we used fMRI and presented
people with an action stimulus and a non-action
stimulus in a visual only condition (V), auditory
only condition (A), or audio and visual together
(AV).
Action sounds minus control sounds
Stimuli Two video clips were used, one action
clip and one control clip. The action clip showed
someone tearing a piece of paper. The control
clip was an abstract animation with the same
color scheme. The natural sound of the paper
tearing action was recorded with the video. The
control clip made a computer-generated tone in
concert with the movement of the square.
Only one region was activated both by sound
alone and by sight alone, the posterior superior
temporal sulcus (STS) bilaterally. The STS
responded to both senses during both the action
clip and the control clip.
action clip
One site in the left ventral premotor cortex
was activated by the action AV condition, but
not by action A or action V alone. This region
was not activated by the control clip.
control clip
Method 10 subjects were scanned. Each subject
completed three functional scans. During each
scan period, the 8-second clips were presented
interleaved with 12 second rest periods. Each
clip was presented with the video component alone
(V), the audio component alone (A), or audio and
video together (AV). Each stimulus type
appeared 3 times in each scan period. Subjects
were simply asked to watch and listen. Functional
images were acquired with the following
parameters TR2s, TE25ms, flip angle90ª, 36
slices each 3mm thick with 1 mm gap, in-plane
resolution 3mm x 3mm. Data analysis was done with
FSL.
Discussion  Action sounds activate prefrontal
cortex. This region is involved in recognizing
environmental sounds and may also contribute to
recognizing actions by sound.  STS is involved
in cross-sensory integration, but is not specific
to actions. The left ventral premotor cortex
may be involved in multisensory integration of
action-related information. The left
lateralization of this site is consistent with
TMS data5 showing left-hemisphere responsiveness
to action sounds and also with a possible
connection between language and action
recognition.

• References
• Rizzolatti Craighero. 2004. Annu Rev Neurosci
  27 169-192.
• 2. Kohler et al. 2002. Science 297 848-848.
• 3. Keysers et al. 2003. Exp Brain Res 153
  628-636.
• 4. Shubotz von Cramon. 2003. NeuroImage 20
  Suppl 1 S120-131.
• 5. Aziz-zadeh et al. 2004. Eur J Nerosci 19
  2609-2612.

email jonask_at_ucla.edu