Interactive Music using a MultiTouch Cellular Automata

1
Interactive Music using aMulti-Touch Cellular
Automata
LANSDOWN CENTRE for Electronic Arts

• Robin Fencott
• www.RobinFencott.com 
• Robin_at_RobinFencott.com

2
(No Transcript)
3
Multi-Touch Screens

• A Multi-Touch Screen is a touch screen that can
  register one or more points of simultaneous
  physical contact.
• Intuitive and direct. No physical controls (e.g.
  stylus, puck).
• Multi-Touch screens are excellent tools for the
  manipulation of visual information.
• Building a Multi-Touch Screen is a well
  documented process.
• An active field of research in academic,
  commercial and hobby/DIY domains.

4
Multi-Touch Screens

• This project uses the Frustrated Total Internal
  Reflection approach (FTIR).
• A sheet of acrylic is illuminated along its
  edges by infra-red LEDs. The light is trapped by
  Total Internal Reflection.
• When a fingertip touches the acrylic the trapped
  light can escape.
• A camera can see this light as a blob. A blob
  tracking program can monitor the position of
  touch points over time.
• We can use a standard video projector to display
  an image back onto the touch surface.

5
Version 1.0

• Built from wood and scrap materials.
• Obsolete or salvaged equipment.
• Electrical components are standard.

6
Project Goals

• Intuitive and accessible.
• Encourage creative and playful engagement.
• Abstract, both visually and sonically.
• Musical, but not an instrument, tool or
  environment.

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Cellular Automata

• Cellular Automata (CA) are systems where time and
  space are measured in discrete steps.
• The values of a CA are stored in cells arranged
  in one or more dimensions (e.g. a two-dimensional
  grid).
• As time progresses the cells are simultaneously
  updated according to a pre-defined set of rules.
• For example in Conways Life each cell is
  either dead or alive. The value of the cell is
  determined by the values of its neighbours.
• CA can exhibit emergent behaviour - a simple set
  of rules can produce a complex outcome.

15
Interactive Cellular Automata

• Adjacent cells begin to interact with each other
  creating localised activity around the
  participants points of touch.
• The Cellular Automata gives the work an active
  and responsive feel, rather than being passive.
• Criteria for the CA
• Must not erupt or seem totally random.
• Activity is localised.
• Cells must decay over time and eventually die
  out.

16
Mapping

• Direct and (hopefully) perceivable. This is in
  line with the objective of approachability.
• Each cell is given a colour. Initially all cells
  are orange or pink, however greens and yellows
  are introduced over time.
• Each cell is given a musical note, which is not
  related to the xy position.
• Instead, notes are picked at random from a small
  musical scale. This means a non-musician can
  still make nice music.

17
Mapping

• Different colour groups get different synthetic
  timbres and individual cells get slight
  variations on these timbres.
• Spatial position is aligned with the horizontal
  screen location.
• Gliding gestures cause pitch bending.

18
Sessions

• The grid is randomised at the start of the
  program.
• The software notes when the screen is in use and
  gradually adds variations, e.g. replacing cells,
  introducing new sonic and visual colours etc.
• Once the system has been inactive for a short
  time the system is reset to a new random state.
• The temporal thresholds are also randomised with
  each session.

19
Audience Reception

• Participants grasp the nature of the interaction
  and appreciate the correlation between colours
  and timbres.
• The work appeals to people of all ages.
• Participants often spend at least a minute
  interacting with the work.
• People are inquisitive and want to know how it
  works.
• People usually approach the piece as a toy,
  although occasionally I am expected to play a
  tune.

20
(No Transcript)
21
(No Transcript)
22
Thank you
LANSDOWN CENTRE for Electronic Arts
Robin Fencott www.RobinFencott.com Robin_at_RobinFenc
ott

• This appearance was funded by the
• Lansdown Centre for Electronic Arts,
• Middlesex University, London.