Lecture 08: Semiotic Media Theory

1
Lecture 08 Semiotic Media Theory
IS 246Multimedia Information
Prof. Marc Davis UC Berkeley SIMS Monday and
Wednesday 330 pm 500 pm Fall
2003 http//www.sims.berkeley.edu/academics/cours
es/is246/f04/
2
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

3
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

4
Kuleshov Effect

• Kuleshov Effect
• Neutral Face ? Soup (Pensive Face)
• Neutral Face ? Dead Woman (Sad Face)
• Neutral Face ? Child playing with toy bear
  (Happy Face)
• How do you describe the face?
• Video has a dual semantics
• Sequence-independent stable semantics of shots
• Sequence-dependent variable semantics of shots

5
Isenhour on Context and Order

• A1B A2C ? A1 NOT EQUAL A2
• Shot context affects shot meaning
• The shot before affects the shot after
• BA1 CA2 ? A1 NOT EQUAL A2
• Shot context affects shot meaning
• The shot after affects the shot before
• AB NOT EQUAL BA
• Short order effects shot meaning
• (AB)C NOT EQUAL A(BC)

6
Burchs Transitions

• Temporal transitions
• Continuous
• Discontinuous
• Temporal ellipsis
• Measurable time ellipsis
• Indefinite time ellipsis
• Temporal reversal (flashback, overlapping cut)
• Measurable time reversal
• Indefinite time reversal

7
Burchs Transitions

• Spatial transitions
• Continuous
• Discontinuous
• Proximal
• Radically discontinuous

8
Barthes Action Sequences

• Consecutive
• Temporal succession
• Consequential
• Causal succession
• Volitive
• Action results from an act of will
• Reactive
• Causal succession based on stimulus-response
• Durative
• Indicating the beginning, ending, or duration of
  an action
• Equipollent
• Necessarily paired actions (e.g., asking a
  question and answering a question)

9
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

10
Semiotic Media Theory

• Application of semiotic methods of analysis to
  media, especially cinema
• Traditionally used for structural and functional
  analysis
• In computational media (especially automatic film
  editing), used also to define the primitives,
  means of combination, and means of abstraction
  for cinematic synthesis

11
Semiotic Media Theory

• Key questions for semiotics of cinema
• How does cinema mean?
• What are the significant structures of cinematic
  form?
• What are the various hierarchical levels of these
  structures?
• How do they compare to linguistic structures?
• Does cinema have any unique semiotic properties?

12
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

13
The Sign
Signified
Signifier
14
The Linguistic Sign
dog
dog
15
The Video Sign
dog
16
Arbitrariness of the Video Sign

• Theories of video denotation
• Iconic (i.e., onomatopoetic)
• Video is a mechanical replication of what it
  represents
• Arbitrary
• Video constructs an arbitrary relationship
  between signifier and signified
• Motivated
• The relationship between the signifier and
  signified is motivated, but by what?
• A natural analogy between video and the world?
• By the conventions of cinematic language?

17
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

18
Articulation

• Articulation
• Any form of semiotic organization which produces
  distinct combinable units
• Double articulation in natural language
• First articulation
• Morphemes smallest formal units of significance
  (e.g., cow)
• Constructed out of phonemes
• Second articulation
• Phonemes sound units which in and of themselves
  lack significance (e.g., c ow)

19
Commutation

• Etymologically change together
• The substitution of one signifier for another
  produces a change of the signified
• Example in phonemes to morphemes
• Different pronunciations of the ow in cow
  will still be understood as cow
• But we distinguish cow caw quay coo cal

20
Cinematic Articulations

• Metz
• Cinema has no double articulation because its
  smallest units (shots) are significant
• Based on Bazinian view of cinema as reproduction
  of reality
• Eco
• Cinema has three levels of articulation which
  include sub-shot units
• Similar to Eisensteinian view of cinema as
  construction of representations

21
Iconic Figures / Semes / Signs
Iconic Figures (deduced from perceptive codes)
constituting a paradigm from which units are
selected to form
Iconic Signs
Iconic Semes
Photograms
22
Ecos Photographic Articulations

• Iconic semes
• Example a dark-haired man stands here wearing a
  patterned shirt
• Iconic signs
• Example human nose, human eye, shirt, etc.
• Iconic figures
• Example angles, light contrasts, curves, etc.

23
Ecos Cinematic Articulations

• Kinesic semes (kinemorphs)
• Example Im saying yes to the person on the
  right
• Kinesic signs (kines)
• Example Nod head yes
• Kinesic figures
• Example move head to right, move head up, move
  head down

24
Kinesic Figures / Kines / Kinemorphs
Iconic Figures
Kines
Kinemorphs
Diachrony
Kinesic Figures
Iconic Signs
Iconic Semes
Photo- grams
Synchrony
25
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

26
Metzs Grand Syntagmatique
27
Metzs Grand Syntagmatique

• Autonomous Shot (single shot)
• Single-Shot Sequence (complete unto itself)
• Inserts (differentiated from shot context)
• Non-diegetic insert
• A single shot which presents objects exterior to
  the story world
• Displaced diegetic insert
• Diegetic images temporally and/or spatially out
  of context
• Subjective insert
• Memories, fears, dreams, etc. of character
• Explanatory insert
• Single shots which clarify diegetic events

28
Metzs Grand Syntagmatique

• Achronological Syntagmas
• Parallel Syntagma (alternating)
• Two alternating motifs without clear spatial or
  temporal relationship
• Bracket Syntagma (non-alternating)
• Brief scenes without temporal sequence but often
  organized around a concept
• Chronological Syntagmas
• Descriptive Syntagma (non-narrative)
• Objects shown to create spatial contiguity to
  situate action
• Narrative Syntagmas (narrative)

29
Metzs Grand Syntagmatique

• Narrative Syntagmas
• Alternate (Narrative) Syntagma
• Narrative crosscutting showing temporal
  simultaneity (parallel action)
• Linear (Narrative) Syntagma
• Scene (continuous)
• Spatial contiguity and temporal continuity across
  a series of shots
• Sequences (elliptical)
• Episodic Sequence
• Symbolic summary of chronological progression
  usually to compress time (montage sequence)
• Ordinary Sequence

30
Brian OConnor on Metz
31
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

32
Discussion Questions (Barthes)

• Jaiwant Virk on Barthes
• If The principle of narrative art it is a
  matter of producing a discourse which best
  satisfies the demand for completeness, then how
  does one define the boundaries of the action
  sequence? How much cultural history does one pack
  to prevent the horror of the vacuum? Or should
  the sequence be completely abandoned to a
  subjective analysis of the reader/viewer?

33
Discussion Questions (Barthes)

• Jaiwant Virk on Barthes
• What are the possible factors that can be
  considered to define/label an action sequence for
  media metadata purpose (motion, etc.)?

34
Discussion Questions (Eco)

• Yongwook Jeong on Eco
• According to Eco, because all signs are arbitrary
  and conventional, we have to learn how to
  interpret them. This made me have a question
  about whether it is possible to have more than
  one languages of cinema. Does this make it
  possible to distinguish one kind of films from
  another?

35
Discussion Questions (Eco)

• Yongwook Jeong on Eco
• Regarding the cinematic code as the only code
  carrying a triple articulation, Eco seems to make
  iconic codes a basis for his cinematic language.
  Can we apply his arguments to today's films?

36
Discussion Questions (Metz)

• Ryan Shaw on Metz
• Metz believes that in cinema, unlike in spoken
  language, even the smallest identifiable units
  are complete signs, having a signifier (the
  image) and a signified (the thing which the image
  is depicting). Moreover, unlike in linguistic
  signs, the relationship between the signifier and
  the signified in these signs is motivated by
  perceptual similarities ("the image of a dog is
  like the dog").
• What implications does Metz's view have for the
  task of programming a computer to understand a
  film? If the smallest analyzable units of cinema
  depend on human perception for their meaning,
  must we solve the problems of computational
  perception before such a task can be undertaken?
  Or might the perceptual motivations of cinematic
  signs be encoded in metadata in such a way that a
  computer could construct an understanding of a
  film, even without the ability to perceive the
  motivations itself?

37
Discussion Questions (Metz)

• Ryan Shaw on Metz
• Metz struggles with the application of linguistic
  models to the question of how films are
  understood. His comparisons of linguistics and
  cinema seem to point out more differences than
  similarities, yet he persists in his effort due
  to his belief that the methods of linguistics...
  provide the semiotics of cinema with a constant
  and precious aid in establishing units that...
  are liable over time... to become progressively
  refined.
• Is this belief justified? Is the semiotics of
  cinema making progress in refining these units?
  Or might it be more useful to stop trying to
  treat cinema as a language and investigate
  other methods (psychological or neurological,
  perhaps)?

38
Todays Agenda

• Review of Last Time
• Editing II
• Semiotic Media Theory
• The Video Sign
• Cinematic Articulations
• Syntagmatic Structures
• Discussion Questions
• Action Items for Next Time

39
Metz on Shot/Word Distinction

• Shots are infinite in number, contrary to words,
  but like statements, which can be formulated in
  verbal language.
• Shots are the creations of the film-maker, unlike
  words (which pre-exist in lexicons), but similar
  to statements (which are in principle the
  invention of the speaker).
• The shot presents the receiver with a quantity of
  undefined information, contrary to the word. From
  this point of view, the shot is not even
  equivalent to the sentence. Rather, it is like
  the complex statement of undefined length (how is
  one to describe a film shot completely by means
  of natural language?).
• The shot is an actualized unit, a unit of
  discourse, an assertion, unlike the word (which
  is a purely virtual lexical unit), but like the
  statement, which always refers to reality or a
  reality (even when it is interrogative or
  jussive). The image of a house does not signify
  house, but Here is a house the image
  contains a sort of index of actualization, by the
  mere fact that it occurs in a film.
• Only to a small extent does a shot assume its
  meaning in paradigmatic contrast to the other
  shots that might have occurred at the same point
  along the filmic chain (since the other possible
  shots are infinite in number), whereas the word
  is always part of at least one more or less
  organized semantic field. (Metz 1974 115-116).

40
Administrivia

• You MUST have a SIMS computer account to do
  Assignment 1

41
Readings for Next Time

• Wednesday 09/29
• Eisenstein, S.M. Film Form Essays in Film
  Theory. Harcourt Brace Jovanovich, Publishers,
  San Diego, 1949 pp. 45-63.