Embodied programming with visual and tangible representations

1
Embodied programming with visual and tangible
representations

• Jakob Tholander
• Ylva Fernaeus
• The Kids group Department of Computer and Systems
  Sciences
• Stockholm University/Royal Institute of
  Technology
• www.dsv.su.se/research/kids
• jakobth_at_dsv.su.se
• ylva_at_dsv.su.se

2
Projects

• Splash Games, programming and learning in
  schools and homes
• Weblabs New representational infrastructures
  for learning in science and mathematics
  executable models that children share over the
  web
• ? Yishay Mor on Friday www.weblabs.org.uk

3
Our research interests

• To understand childrens interaction and and
  learning in construction and programming
  activities, in particular the role played by
  representations and external artefacts
• To develop methods for collaborative
  programming and constructions activities
• To develop and refine programming tools based on
  insights from empirical work

4
Our pedagogical approaches
Collaboration
Kids own creations
New programming representations
Webbased sharing and discussion of working models
5
The Paper
6
Representations for learning

• A move towards the concrete
• Graphical user interfaces
• Tangible interaction
• Animations
• Graphs
• ? What is the relation between (alternative)
  representations and theoretical and conceptual
  aspects of learning?

7
Representations of computer programs
AgentSheets (graphical rules)

• Logo (textual)
• to go
• fd 1
• ifelse pc black
• stamp color
• rt 90
• stamp black
• lt 90
• end

ToonTalk (animated)
8
The question of the present study

• What are these programming representations?
• Simplifications of programs
• Easier ways to construct programs
• Easier ways to learn programming concepts
• or
• Alternative representations of programs and
  programming concepts (not better, easier, or
  faster etc)
• ? What possible consequences can we see from such
  a view with respect to interaction and learning?

9
Embodied programming

• Embodied Interaction, Dourish (2001)
• Embodied Reference, Heath Hindmarsh (2000)
• Embodied Social Interaction, Goodwin (2000)
• We introduce the term to
• emphasise that programming with visual and
  tangible forms of representations allows people
  to involve bodily actions such as pointing and
  gesture in a more direct sense in their meaning
  making practices

10
Three micro-analytic investigations of
childrens programming with animated and tangible
representations
Pointing and gesturing outside the computer
Pointing and gesturing inside the computer
Social representation of program elements
11
Sam and Jakob
Play movie!
12
Pointing and gesturing in interactions with and
through visual and animated programming
representations
Deictic references
Discussing dynamics by combining talk and gesture
Referencing program elements by pointing
13
Explains and negotiates with gesture and
static representation
Refining ideas
14
Discussion

• Three issues
• Programming primarily as a mental activity?
• Knowing in tool-intense activites?
• Interpreting learning from embodied forms of
  interaction?

15
Programming primarily as a mental activity?

• Visual and tangible programming representations
  reduce the complexity of programming in terms of
  not having to memorize a complex programming
  syntax, and not having to compose and interpret
  algorithm structures.
• Instead, sense making practice is shifted to
  emphasize embodied and physical thinking and
  action when programming.
• This stands in strong contrast to the traditional
  conception of programming as a cognitive activity
  where the prime of the action goes on within the
  minds of the participating individuals.

16
Knowing in (and out of) tool-intense activites?

• Our analysis shows the problem of separating
  mastery of tools from domain-specific
  (theoretical or conceptual) learning.
• An important assumption we see in much learning
  research is that knowing (still) is viewed as
  closely connected to verbal articulation, while
  non-verbal action is secondary to the process of
  knowing
• We argue that embodied and enacted forms of
  interaction in tool-intense activities need to be
  recognized as just as important parts to learning
  processes as are verbalizations of knowledge.

17
Interpreting learning from embodied forms of
interaction

• As new forms of representing and expressing
  knowledge evolve, the actual content of what is
  expressed and communicated will change. We find
  that this point has largely been overlooked in
  the discussions regarding new representational
  forms for learning.
• But, practical activity in a domain is (still
  often) viewed as something different than
  learning its theoretical content. Hence, gaining
  theoretical knowledge would require other non
  practical kinds of activities, whatever those
  might be.
• Attempting to have students develop knowledge
  that is detached from the representational tools
  they rely upon, is not in line with a perspective
  on learning that emphasizes the fundamental
  relationship between people, activities, and
  artefacts in processes of knowing.

18
More Discussion, Questions!