Evaluation Methodologies for Ubiquitous Computing UBICOMP 01 Workshop

1
Evaluation Methodologies for Ubiquitous
ComputingUBICOMP 01 Workshop

• Chris Quintana
• Center for Highly Interactive Computing in
  Education (hi-ce)
• University of Michigan

2
Context

• Developing learner-centered scaffolded tools
  for science
• Now Wired desktop tools
• New Wireless handheld tools
• Middle/high school project-based science
• Evaluation Approaches
• Effects of scaffolds What did students learn?
• Effects with scaffolds How do students work
  with scaffolds?

3
Effects With Evaluation

• Data collection
• Collect student actions, student work
  patterns,conversations
• Process video
• Some log file analysis
• Evaluate work with each scaffold in the software
  over time and summarize

• Evaluation Criteria
• Accessibility
• Use
• Efficiency
• Accuracy
• Progression
• Reflectiveness

4
New Challenges

• Now Wired desktops wireless handhelds
• Students can work anytime/anywhere
• Studying student work and collaboration/discourse
  patterns
• Challenges
• Observing and tracking student work and discourse
  patterns
• Additions to evaluation criteria