Sensor-Based Interactions

1
Sensor-Based Interactions

• Kami Vaniea

2
Outline

• Coping with Uncertainty
• Making Sense of Sensing Systems Five Questions
  for Designers and Researchers
• Distributed Mediation of Ambiguous Context in
  Aware Environments

3
Uncertainty

• State of knowledge in which one or more
  alternatives result in a set of possible specific
  outcomes, but where the probabilities of the
  outcomes are neither known nor meaningful.

4
Uncertain Communication

• Sensors produce concrete data with uncertain or
  ambiguous interpretation
• Location Data (GPS)
• Temperature
• Computers can make bad decisions based on
  uncertain data
• Interrupting user
• Making poor decisions on users behalf
• Communicating wrong information

5
Outline

• Coping with Uncertainty
• Making Sense of Sensing Systems Five Questions
  for Designers and Researchers
• Distributed Mediation of Ambiguous Context in
  Aware Environments

6
HHI and HCI

• Human to Human Interaction (HHI) has been well
  studied by researchers
• How can we use some of the lessons about Human to
  Human communication to facilitate Human to
  Computer communication?
• Recognize that communication is a shared
  responsibility and requires management and repair

7
Normans Seven Stages of Action

• Forming the goal
• Forming the intention
• Specifying an action
• Executing the action
• Perceiving the state of the world
• Interpreting the state of the world
• Evaluating the outcome

Goal
Execution
Evaluation
8
Five Issues in Communication

• Address Directing communication to a system
• Attention Establishing that the system is
  attending
• Action Defining what is to be done with the
  system
• Alignment Monitoring system response
• Accident Avoiding or recovering from errors or
  misunderstandings

9
Stages of Action vs. Conversation Issues
Address
System
Human
Attention
Action
Alignment
Accident
10
Address How do I address or of many possible
devices?

• GUI Answers
• Keyboard
• Mouse
• Exposed Challenges
• Disambiguate signal-to-noise
• Disambiguate intended target system
• How to not address the system
• Possible Problems
• No response
• Unwanted response

11
Attention How do I know the system is ready and
attending to my actions?

• GUI Answers
• Graphical feedback such as blinking cursors
• Exposed Challenges
• Giving feedback so the user knows it has the
  systems attention
• Periphery feedback
• Possible Problems
• Wasted input if system not responding
• Unintended action
• Privacy and Security problems(?)

12
Action How do I effect a meaningful action,
control its extent and possibly specify a target
or targets of my action?

• GUI Answers
• Use standard GUI widgets, such as clicking and
  selecting text, to interact with the system
• Exposed Challenges
• Identifying and selecting possible actions
• Avoid unwanted selection
• Handling complex operations
• Possible Problems
• Limited operations available
• Failure to execute action
• Unintended action

13
Alignment How do I know the system is doing the
right thing?

• GUI Answers
• Graphical feedback such as text appearing
• Auditory feedback
• Detectible new state
• Exposed Challenges
• Making system state perceivable
• Providing distinctive feedback on results and
  state
• Possible Problems
• Differentiation problems
• Inability to detect mistakes
• Unrecoverable problems

14
Accident How do I avoid mistakes?

• GUI Answers
• Control/guide in direct manipulation
• Stop, cancel, undo or delete
• Exposed Challenges
• Controlling or canceling system actions in
  progress
• Disambiguating what to undo in time
• Intervening when user makes obvious error
• Possible Problems
• Unintended action
• Inability to recover state

15
Discussion

• UbiComp is still a new growing field
• Now is the time to create new methods of
  interaction between Humans and Machines

16
Outline

• Coping with Uncertainty
• Making Sense of Sensing Systems Five Questions
  for Designers and Researchers
• Distributed Mediation of Ambiguous Context in
  Aware Environments

17
Ambiguous Context Data

• Many systems are based on the assumption that
  context data is unambiguous
• However, accurately determining context is a hard
  problem since the system isnt in possession of
  all the facts
• Techniques such as AI algorithms can be applied
  but are not always accurate and require some
  training
• Sometimes it is necessary to ask the human in
  order to get unambiguous context data

18
Architecture of the Extended Context Toolkit
19
Disambiguating Sensor Data

• In an ubiquitous system where all the devices
  behave in a distributed manor disambiguating data
  is a shared problem

20
Issues in Building Realistic Context-Aware
Applications

• Distribution Devices are in a distributed
  environment
• Storage When should ambiguous data be stored?
• Multiple Subscription Types Allow subscribers to
  opt-in/out and choose their type of subscription
• Pre-emption of Mediation Prevent multiple
  simultaneous mediations
• Forced Mediation Request mediation by other
  devices
• Feedback Provide feedback to users about what
  the system thinks is happening

21
Storage

• Should ambiguous data be stored?
• Should mediated data be stored?
• Both?
• How do we store ambiguous data since it is
  actually a graph of options
• Storing only unambiguous data is easier

22
Subscribers

• Opt-in and Opt-out subscriptions allow
  subscribers to get only what they want and no
  more
• Those who get ambiguous data will have to mediate
  it, wait for another to mediate it or force
  mediation

23
Pre-Emption of Mediation

• Mediation may occur simultaneously in several
  components
• If several components are mediating at once the
  first one to succeed pre-empts the others
• Does this mean the users will get multiple
  requests for mediation?

24
Forced Mediation

• If a subscriber is unable or doesnt want to
  mediate it can request another component to
  mediate
• It passes all ambiguous data to the other
  component which mediates if possible

25
Feedback

• Provide feedback about what the system is doing
  and what its current state is
• Feedback can come in many forms from messages to
  speech

26
Examples

• In/Out Board
• Monitored when residents of a single building
  where present or not
• CybreMinder
• A situation aware reminder system which can use
  situational data to trigger reminders
• Word Predictor
• Makes typing words easier since it guesses context

27
Questions?