Livenotes

1
Livenotes

• A System for Cooperative and Augmented
  Note-Taking in Lectures

Matthew Kam, Jingtao Wang, Alastair Iles, Eric
Tse, Jane Chiu, Daniel Glaser, Orna Tarshish and
John Canny University of California, Berkeley,
USA
2
Video
3
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

4
Motivation Problem Statement

• Constructivism
• Learners are not blank slates that teachers write
  on
• Learners need to actively construct their own
  understanding and knowledge
• But large lecture classes are not conducive for
  active learning
• Passive mode of oral dissemination
• Lack of interactivity among students
• Lack of interactivity with instructor

5
Motivation Precedents

• Face-to-Face Tutored Video Instruction (TVI),
  Distributed TVI (Gibbons, Stanford)
• Group review of pre-recorded lectures
• Regular pauses for small-group discussion
• Students using DTVI received grades 0.5 std dev
  higher than non-TVI students (Smith et al. 1999)
• Peer Instruction (Mazur, Harvard)
• Lecture pauses for small-group discussion with
  neighbors
• Improvements in conceptual understanding and
  problem-solving (Crouch and Mazur 2001)

6
Motivation Small-Group, Cooperative Learning

• More than 375 research studies since 1898
  (Johnson and Johnson 1989)
• Cooperative group learning results in greater
• Efforts to achieve
• Higher-level reasoning
• Transfer from original context to new situations
• Generation of new ideas and solutions

7
MotivationBackground Lecture Notes

• (Hartley 1978, Kiewra et al. 1988) Experiments
  on note-taking that compared students annotating
  over
• Complete lecture notes provided by instructor,
  vs.
• Skeletal (i.e. partial) notes, vs.
• No background notes
• Results students were found to achieve maximum
  retention with skeletal notes

8
Livenotes Recap

• Both a technology and educational practice
• Large lecture classes
• Small-group discussions in ongoing lecture
• Cooperative note-taking Combines real-time
  note-taking with discussion
• Augmented note-taking Skeletal slides for
  students to annotate over

9
Related Systems

• No interaction between students
• Classroom Presenter (Washington)
• StuPad, eClass (Georgia Tech)
• No real-time interaction between students, i.e.
  sharing of notes takes place after lecture
• NotePals (Berkeley)
• Limited real-time interaction between students
• OneNote (Microsoft)

10
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

11
Livenotes Evolution

• 2000 Implemented in Java, for WinCE Clios
• Late 2000 to early 2003 5 small-scaledeployments
  using Clios, laptops and Tablet PCs
• Spring 2003 Medium-scale experiment in
  undergraduate class using Tablet PCs
• Since 2003 Ported to Microsoft .NET

12
Livenotes User Interface
Group awareness (e.g. each users page number)
Unique user colors
Import background slides
Pen and keyboard input
13
Client-Server Topology
Group 1

• 802.11b networking
• Large class broken down into many small groups
  (3-7 students)
• One Tablet per group is set to server mode
• Other members Tablets connect wirelessly to
  groups server

Server
Clients

Group n
Server
Clients
14
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

15
Hypotheses

• Cooperative note-takingShared whiteboard
  interface enhances learning through cooperative
  note-taking and discussion
• Augmented note-takingBackground slides enhances
  learning by augmenting student note-taking

16
Experiment

• Spring 2003 undergraduate HCI class
• 21 volunteers, randomly partitioned into
• Cooperative note-takers
• Individual note-takers (control group)

• 4 weeks (7 lectures)
• Preloaded skeletal PowerPoint slides

17
Previous Observation

• From 5 previous deployments, we learned that
• Graduate students engaged spontaneously in group
  discussions
• Undergraduates were not used to discussing
  lecture material with one another

• For this experiment (with undergraduates), we
  held short, live group discussions in the
  classroom

18
Data Collection

• Short quizzes (4 lectures)
• Survey questionnaires
• First week of deployment (38 response rate)
• End of semester (29)
• Qualitative interviews (3 users)
• Transcripts of students notes (1581 pages)

19
Quantitative Analysis

• Unit of analysis mark
• Spatio-temporally contiguous segment of user
  input
• E.g. This lecture is very interesting
• Quantitative hand-coding of 1581 pages

20
Taxonomy of Marks

• Note-taking someone taking notes on lecture
• Commentary someone making a statement
• Question someone soliciting a response
• Answer response to a question, clarification
• Reinforcement contribution to an existing thread

21
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

22
Cooperative Note-TakingRicher Notes

• Cooperative note-taking group engaged in more
  than twice as much activity as individual
  note-taker

23
Cooperative Note-TakingRicher Notes

• Almost one quarter of marks made by cooperative
  note-takers were attributed to group interaction

24
Student Learning

• Survey question How did Livenotes, if at all,
  assist your learning in lecture?
• Early survey after 2 sessions with Livenotes75
  of respondents self-reported affirmatively
• Survey after semester (i.e. 7 lecture sessions)
  83 of respondents self-reported affirmatively

25
Cooperative Note-Taking Taking Turns to Take
Notes

• 66 of survey respondents agreed that cooperative
  note-taking is more useful
• Someone else might note something that I missed
  or hadnt realized.
• I liked how note-taking became a cooperative
  effort someone can take over if another user is
  still inputing some notes, but the prof had
  moved on already.

26
Cooperative Note-Taking Paying Greater Attention

• 36 of students who self-reported learning
  benefit explicitly attributed that to social
  aspect of cooperative note-takingHelped me to
  focus more in lecture. Often I fall asleep/lose
  attention in lecture. Having group members to
  respond to kept me better on track.

27
Cooperative Note-Taking Dual Conversations

• Need to keep up with both lecture and on-tablet
  conversationIt is helpful to be able to
  discuss questions. However, this does take
  attention away from the lecture if you are
  focusing on answering/asking a question.

28
Cooperative Note-Taking Decreasing Distraction

• Is running Livenotes during class distracting?
  (1 extremely distracting, 5 not distracting
  at all)
• Survey after two lectures 2.6 out of 5
• Survey after deployment ended 3.83 out of 5
• From student notes, playful behavior were
  observed to disappear almost completely after 2
  lectures

29
Cooperative Note-TakingUnanswered Group
Questions

• Students did not have time to answer some
  questions because they needed to keep up with
  lecturer
• Some questions were unanswered because no group
  member knew the answer

30
Cooperative Note-TakingInteraction During Pauses

• Group interaction during pauses in lecture
  accounted for over half of group activity

31
Redeeming PowerPoint

• Criticisms leveled at Microsoft PowerPoint
• The Cognitive Style of PowerPoint (Edward Tufte)
• Death by PowerPoint webpage
• I ? Powerpoint (David Byrne, Talking Heads)
• Too boring, passive, does not promote active
  engagement with material
• But students commented that augmented note-taking
  is like having a conversation with the
  professor

32
Augmented Note-Taking Observed Behaviors

• Elaborated on bullets
• Appended bullets to list
• Concurred and disagreed with bullet
• Noted gist of HCI principles
• Noted advantage and disadvantage of HCI technique
• Answered questions in bullets

33
Augmented Note-TakingElaborated on Bullets
34
Augmented Note-TakingAppended Bullets to List
35
Augmented Note-TakingAnswered Questions in
Bullet
36
Augmented Note-Taking Answered Questions in
Bullet

• Students responded to questions in bullets even
  when when they were not cooperative note-takers
• Each group responded to 35 of the questions
• Each question received a response from 36 of the
  groups

37
Augmented Note-Taking Student Learning

• Several high-quality notes in both individual and
  cooperative note-taking groups resulted from
  students working off bullets
• Possibly due to bullets focusing student
  attention to relevant portions of lecture
• A larger proportion (55) of students who
  self-reported learning benefit attributed it to
  augmented note-taking, compared to cooperative
  note-taking.
• Half of this sub-group attributed that to having
  slides at hand to annotate over

38
Quiz Scores

1 Cooperative note-takers. 2 Individual
note-takers. 3 Quiz scores presented in this
table are normalized on a scale of 100.

• No statistical significance
• But sample size was too small due to poor
  attendance at end of semester

39
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

40
Student-Instructor Interaction

• To help instructor assess student learning, we
  deployed feedback feature in last two sessions
• Students provide instructor with real-time,
  anonymous lecture feedback
• Recently allowed students to alert instructor
  that they have questions

41
RecommendationBackground Slides as Scaffold

• Bullets are a lightweight means for lecturer to
  engage actively with students during class
• Posing questions
• Counter-intuitive bullets
• Provocative statements
• Direct student attention to critical parts of
  lecture
• E.g. prompts such as Pros? and Cons? with
  blank spaces for students to fill in

42
Outline

• Motivation
• Solution
• Experiment
• Results
• Implications
• Conclusion

43
Conclusion

• Cooperative note-taking
• Richer variety of notes, higher-order thinking
• More than twice as much notes as individuals
• Members took turns to take notes
• Students kept awake to interact with group
• Augmented note-taking
• Observed dialogue with bullets
• Reflected higher-order thinking
• High-quality notes resulted from working off
  bullets

44
Acknowledgement

• Corporation for Education Network Initiatives in
  California
• Microsoft Research
• National Science Foundation
• Qualcomm
• Volunteers from Computer Science 160, Spring 2003
• Public domain source code by James R. Weeks

45
Questions?
Livenotes can be downloaded from http//www.cs.be
rkeley.edu/mattkam/livenotes
Matthew Kam, Ph.D. student Department of
Electrical Engineering and Computer Sciences,
and Berkeley Institute of Design University of
California at Berkeley, USA mattkam_at_eecs.berkeley.
edu