Understanding Visual Media

1
Understanding Visual Media
IS146 Foundations of New Media

• Prof. Marc Davis, Prof. Peter Lyman, and danah
  boyd
• UC Berkeley SIMS
• Tuesday and Thursday 200 pm 330 pm
• Spring 2005
• http//www.sims.berkeley.edu/academics/courses/is1
  46/s05/

2
Lecture Overview

• Review of Last Time
• History and Technology of Digital Imaging
• Today
• Understanding Visual Media
• Preview of Next Time
• Case Study Cameraphone

3
Lecture Overview

• Review of Last Time
• History and Technology of Digital Imaging
• Today
• Understanding Visual Media
• Preview of Next Time
• Case Study Cameraphone

4
Image reading

• Visual communication is always coded
• It seems transparent only because we know the
  code already, at least passively but without
  knowing what it is we know, without having the
  means for talking about what it is we do when we
  read an image.
• Our culture is moving from textual to visual
• Until now, language, especially written
  language, was the most highly valued, the most
  frequently analyzed, the most prescriptively
  taught and the most meticulously policed code in
  our society.
• Visual literacy is not taught and needs to be
• If schools are to equip students adequately for
  the new semiotic order, if they are not to
  produce people unable to use the 'new writing'
  actively and effectively, then the old boundaries
  between 'writing' on the one hand, traditionally
  the form of literacy without which people cannot
  adequately function as citizens, and, on the
  other hand, the 'visual arts', a marginal subject
  for the specially gifted, and 'technical
  drawing', a technical subject with limited and
  specialized application, should be redrawn.

5
Image technology

• development of technology (e.g. camera, film
  stock, etc.)
• development of presentation techniques (e.g.
  illumination)

• development of technology (e.g. camera,
  compression, manipulation)
• development of presentation techniques (e.g.
  automatic collage)

• development of technology (e.g. colour, brushes,
  canvas, etc.)
• development of presentation techniques (e.g.
  central perspective)

Painting VERMEER VAN DELFT, c. 1665,Oil on
canvas, 46,5 x 40 cm, Mauritshuis, The Hague
Digital image (1950)
Photo (1830)
6
The camera
Analog camera produces a continuous image,
simulating the capture of the eye.
Negative (reverse reality)
Actual image (various manipulation processes)
Analog technology is expensive and complicated to
handle.
7
The digital image
The image is a matrix of values, representing the
colour, texture, dimension, etc.
8
The digital image - size
1 Bit -gt black and white 8 Bit -gt
greyscale 16 Bit -gt 64.000 colours (High
Colour) 24 Bit -gt 16 Mil. colours (True Colour)
9
The digital image - manipulation
Pixel or vector

• Image format
• Resolution
• Mode
• File type

• Image enhancement
• resize
• colour enhancement
• artistic filters
• establishing planes
• any other sort of manipulation

ALWAYS ask what do I intend to do with the image
now AND in the future?
10
Lecture Overview

• Review of Last Time
• History and Technology of Digital Imaging
• Today
• Understanding Visual Media
• Preview of Next Time
• Case Study Cameraphone

11
Understanding Comics

• Scott McCloud - comic artist
• Explains visual communication through comics
• Explains comics through visual communication
• "If you've ever felt bad about wasting your life
  reading comics, then check out Scott McCloud's
  classic book immediately. You might still feel
  you've wasted your life, but you'll know why, and
  you'll be proud. - Matt Groening, creator of
  The Simpsons.

12
What Are Comics?

• Juxtaposed pictorial and other images in
  deliberate sequence, intended to convey
  information and/or to produce an aesthetic
  response in the viewer. (p. 9)
• How do comics differ from
• Photographs?
• Movies?
• Hieroglyphics?
• Emoticons?

13
Why Understand Comics?
14
Old Comics Trajans Column
15
Old Comics Mayan Codex Nuttall
16
Old Comics Tortures of St. Erasmus
17
Scott McClouds Big Triangle
Picture Plane
Reality
Language
In Chapter Two of Scott McClouds 1993 book Understanding Comics, he devises a map of visual iconography (i.e., pictures, words, symbols) that takes the shape of a triangle.
18
Scott McClouds Big Triangle
Picture Plane
Reality
Language
The lower left corner is visual resemblance
(e.g., photography and realistic painting).
19
Scott McClouds Big Triangle
Picture Plane
Reality
Language
The lower right includes the products of what he
calls iconic abstraction (e.g., cartooning).
20
Scott McClouds Big Triangle
Picture Plane
Reality
Language
And at the top are the denizens of the picture
plane (pure abstraction) which cease to make
reference to any visual phenomena other than
themselves.
21
Scott McClouds Big Triangle
Picture Plane
Reality
Language
The move from realism to cartoons along the
bottom edge is a move away from resemblance that
still retains meaning, so words, the next
logical step in the progression, are included at
far right, thereby enclosing anything in comics
visual vocabulary between the three points.
22
Scott McClouds Big Triangle
Picture Plane
Reality
Language
McCloud found that The Big Triangle as it came
to be known, was an interesting tool for thinking
about comics art...
23
Scott McClouds Big Triangle
Picture Plane
Reality
Language
...and for visual art and language in general!
24
Cartoons and Viewer Identification
25
Closure From Parts To The Whole
26
Closure Bridging Time and Space
27
Closure in Comics
28
Closure The Gutter in Comics
29
Closure Moment-To-Moment
30
Closure Action-To-Action
31
Closure Subject-To-Subject
32
Closure Scene-To-Scene
33
Closure Aspect-To-Aspect
34
Closure Non-Sequitur
35
Questions for Today

• How do we interpret images and sequences of
  images?
• How do we read different visual representations
  of the world (especially different levels of
  realism and abstraction) differently?
• How does what is left out affect how we
  understand images and sequences of images?

36
Questions for Today

• What are some of the differences between how text
  and images function in comics?
• What would be lost/gained in moving between
  images and text?

37
Questions for Today

• How could we represent images and sequences of
  images in order to make them programmable?
• What could computation do to affect how we
  produce, manipulate, reuse, and understand images
  and sequences of images?

38
Lecture Overview

• Review of Last Time
• History and Technology of Digital Imaging
• Today
• Understanding Visual Media
• Preview of Next Time
• Case Study Cameraphone

39
Readings for Next Time

• Marc Davis, Simon King, Nathan Good, and Risto
  Sarvas. From Context to Content Leveraging
  Context to Infer Media Metadata. in Proceedings
  of 12th Annual ACM International Conference on
  Multimedia (MM2004). New York ACM Press, p.
  188-195, 2004.
• Discussion Questions
• Nancy A. Van House, Marc Davis, Morgan Ames,
  Megan Finn, and Vijay Viswanathan. The Uses of
  Personal Networked Digital Imaging An Empirical
  Study of Cameraphone Photos and Sharing. in
  Extended Abstracts of the ACM Conference on
  Computer-Human Interaction (CHI2005). New York
  ACM Press, Forthcoming 2005.
• Discussion Questions