HWG 1

1
Unobtrusive Augmentation of Everyday
Environments with Computing

• Hans-W. Gellersen
• Lancaster University
• Department of Computing
• Ubiquitous Computing Research

2
Computer-Augmented Environments

• Background
• Ubiquitous Computing more computers embedded
  everywhere
• Augmented Environments primary attention to the
  environment
• Two (or Three) Projects
• PinPlay A wall that is also a network
• Load sensing Table surfaces that are also
  sensor/input device
• (Smart-Its A prototyping platform for augmented
  artefacts)

3
Background

• Ubiquitous Computing
• The Computer of the 21st Century, Mark Weiser,
  SciAm 1991
• Mass deployment of computing in everyday life
• New paradigm for how computers are used
• each person is continually interacting with
  hundreds of nearby interconnected computers
  without explicitly attending to them
• also Invisible Computing, Ambient Intelligence,
  etc
• Augmented Environments
• Back to the Real World, CACM July 1993
• Primacy to interaction with accustomed
  physical/tangible environment
• Leverage the meaning/purpose of our designed
  environments
• Introduce computing in the background

4
Design Perspective

• Toward a hybrid design
• Combine unique capabilities of computer
  technology with properties of physical
  environments
• Focus on foreground activity how people interact
  with their designed environment preserve
  familiarity and accustomed use
• Interaction with Physical Artefacts /
  Environments
• Physical/Tangible Interaction
• Physical affordances suggesting and guiding
  action
• Distributed interaction actions across artefacts
  / space
• Spatial/ambient interaction
• Spatial organisation of action/communication
• Ambient interaction spatial attention model

5
Technical Perspective

• Computers in the Background
• Computers as secondary artefacts
  (embedded/situated)
• Embedded (only a) part of some other artefact
• Situated meaningfully placed, designed for
  specific context(context-made rather than
  context-aware)
• The Environment is the Interface
• Build upon affordances of the primary artefact or
  environments
• Dont break with accustomed uses and familiar
  concepts
• Physical I/O (sensors/actuaors) to tie computers
  to entities in the physical environment
• Networking to enable coherent interaction
• Spatial interaction, proximate networking, etc
• Allow for new interactions/relationships across
  parts of the environment

6
Bishops Marble Answering Machine

• Physical interaction with digital information

7
Wellners Digital Desk

• Why a desktop metaphor when you can have a real
  desktop
• Seamless transitions physical and digital
  interaction

8
Computer-Augmented Environments

• Background
• Ubiquitous Computing more computers embedded
  everywhere
• Augmented Environments primary attention to the
  environment
• Two (or Three) Projects
• PinPlay A wall that is also a network
• Load sensing Table surfaces that are also
  sensor/input device
• (Smart-Its A prototyping platform for augmented
  artefacts)

9
The PinPlay Network

• Concept
• The wall as network bus for the things attached
  to it
• A new type of network to connect everyday objects
  on common surfaces such as boards and walls
• Use of familiar interaction
• pinning objects to the wall
  pinning nodes to the network
• Preserve original functionality
• Embedded computing does not displace original
  uses

10
PinPlay Components

• Network Surface
• Common surface augmented with conductive material
  to create two-dimensional network medium
• Connectors
• Pushpin-like physical connector for socket-less
  attachment of objects
• PinPlay Objects
• any type of device/object with embedded
  computing, mounted onto connector
• Network
• PinPlay behaviour discovery of objects when
  they become attached

11
PinPlay Noticeboard

• First demonstrator build for proof of concept

12
PinPlay Noticeboard

• PinPlay Surface
• Corkboard augmented with two conductive sheets
• Ground layer on top, data/power layer hidden,
  cork as insulator
• Low cost, off-the-shelf,deployable at
  large-scale
• PinPlay Connectors
• Simple connector board with pushpin for two
  separate connection points

13
PinPlay Noticeboard

• Network Technology
• 1-wire bus, Dallas MicroLAN
• 16300 bits/s
• PinPlay Objects
• Smart Notification Pin iButton and switchable
  LED
• Time-in-a-can iButton memory, internal calendar
  and clock

Switch
Switch
Time-in-a-Can
Time-in-a-Can
14
PinPlay Noticeboard

• User interaction
• insert or remove pin
• network detects change
• protocol to determine pin with highest
  priority

• Network control
• External laptop connected as 1-wire network
  node
• runs network controller
• used to pre-set pins with priority and
  deadlines

15
Scaling up

• bigger boards, more nodes

16
Testing versatility

• different objects, different applications

• PinPlay Lightswitch
• place it where you like!

17
PinPlay

• Technology Research
• Network surface development
• Simple and robust protocol design, zero
  maintenance
• Scalability and density (initial target 25
  nodes/sqm)
• Application Research
• Augmented noticeboards and other interactive
  surfaces
• Embedded home control buses
• Networking and free placement of controls (light
  switches, appliance controls etc.)
• Communication bus for wall-attached artefacts
• Clocks, calendars, sensors, digital picture
  frames,

18
Load-Sensing Surfaces

• Concept
• Gravity is ubiquitous no physical thing can
  escape it
• Use this force to build interfaces between the
  physical and the virtual
• Augment common surfaces (floors, tables,
  shelves) this is where gravitation pulls
  objects to
• Technology variety of load sensors and pressure
  gauges with different characteristics

19
Load-Sensing Surfaces

• Principle
• Augment surface at the corners
• Force applied (e.g. by weight of an object, or
  explicit pressure) is detected as load depending
  onposition of the pressure point
• more than just weight
• surface detects weight/pressure
• But also position, traces, etc

20
Load Sensing
Control

• Basic load sensor
• e.g. your kitchen scale
• Load-sensing surface

Force
Load
Scale
Control
F1 F2F3 F4
Load
Surface
Centre of Gravity
Control
DF1 DF2DF3 DF4
Load Change
Surface
Position
21
Load Sensing
Control

• Basic event detection
• Object placement
• Object removal
• Further event processing
• Detect movement
• Detect specific events
• Detect Object ID/Class
• Tracking movement
• Detecting traces on surfaces
• Tracking objects
• Tracking across surfaces
• Correlation of events
• Grouping events associated with the same object

Surface Event Sensor
Load Change
Event
Position
Control
SurfaceMovementTracker
Load Change
Trace onsurface
Position
Control
Observing Event
Trace acrosssurfaces
Event E1
Event En
22
Load-Sensing Surfaces

• Weight Lab
• Various augmented surfaces
• Floor 240 x 180cm, up to 800kg load
• Larger table up to 200kg
• Coffee table up to 8kg, highly sensitive
• Shelves and trays

Floor with embeddedS-load cell
23
Load-Sensing Surfaces
Sensor board with wireless communication
Augmented tables
24
Load-Sensing Surface

• Context Acquisition
• Weight of objects
• Detection depends on sensor range (i.e. small
  weights not detectable on heavy-load surface)
• Application object identification (classes/
  instances)
• Position of objects
• cm-level accuracy (depend. on surface size)
• Table can be pre-loaded
• Multiple objects can be positioned if placed
  non-simultaneously

25
Load-Sensing Surface

• Context Acquistion
• Beyond weight and position basic events derived
  from signal analysis over short time (low-cost
  analysis in time domain)

New object
knocked over
Object put down
removed
26
Load-SensingSurfaces

• Context Acquistion
• Tracking of people/objects
• Prediction ofactivities

27
Load-sensing surfaces

• Surfaces as Context-sensor
• Object position and movement
• Human activity modelling
• Many applications track objects, detect
  breakdown of routines
• Implicit human-computer interaction
• Everyday Surfaces for explicit interaction
• Challenge when do we interact with the table,
  and when with the computer (compare speech input
  problem)

28
Load-Sensing Surface

• Surfaces as Interaction Device

29
Load-Sensing Surface

• Surfaces as Interaction Device

moveleft
moveleft
touch and move right
cup
book
click andrelease
click
click
30
Load-Sensing Surface

• Surfaces as Interaction Device

31
Conclusion

• Unobtrusive infrastructure
• PinPlay and Sensor Surfaces are example for
  infrastructure integrated with everyday use
• Comparatively low-cost, very deployable
• Added functionality
• New forms of interaction
• New applications
• Not compromising existing use and familiarity

32
Future Work

• More interdisciplinary work with designers
• e.g. Equator project with RCA
• series of product design concepts for
  sensor-tables
• A different story how to investigate such
  systems ?
• Inevitably involves hardware prototyping (awkward
  as the aim is not to advance hardware)
• Hardware for demonstrators is hard to reproduce
• Need support for physical prototyping
• Smart-Its
• Small wireless sensor devices designed for rapid
  prototyping