Ubiquitous Computing II: Research Projects

1
Ubiquitous Computing IIResearch Projects

• POSMIS Lab. POSTECH

2
Contents

• EasyLiving by Microsoft
• e-Class by Georgia Tech.
• Paper by DCI (Disappearing Computing
  Initiatives)

3
EasyLiving by Microsoft
4
EasyLiving

• Project for developing a prototype architecture
  and technologies for building intelligent
  environments
• Key features
• Computer vision for person-tracking and visual
  user interaction
• Multiple sensor modalities combined
• Use of a geometric model of the world to provide
  context
• Automatic or semi-automatic sensor calibration
  and model building
• Fine-grained events and adaptation of the user
  interface
• Device-independent communication and data
  protocols
• Ability to extend the system in many ways

5
Elements of EasyLiving
UI and Applications of Computing in the Physical
World
Distributed System Architecture
Sensing and World Modeling
6
Person Tracking System (1/2)

• Stereo processing with commercial software
• Background subtraction and person detection
• Reports sent to central person tracker about 7 Hz

7
Person Tracking System (2/2)

• Multiple sensors of different modalities
• Explicit reasoning about occlusion
• No more expensive multi-camera heads
• One vision computer per room

Voxel representation of 3D space in room
8
Active Badges

• Badges for person tracking
• Larger area, less precision
• Fusion via generic person tracker
• Ultrasound badges (ATT) in lab use
• Diffuse-IR system from Arial Systems
• Signal strength from Wireless LAN
• Bluetooth beacons

9
Keyboard Detection

• Match color and approximate shape

1
3
2
10
EasyLiving Service Scenario
Tom is at home. He enters the living room sits
down at a PC in the corner. He surfs through a
selection of MP3's, and adds them to a playlist.
He gets up and sits down on the couch. His
session follows him to the large wall screen
across from the couch. This screen is selected
because it is available and in Toms field of
view. Tom picks up a remote control sitting on
the coffee table and uses the trackball on it to
request the room controls. They appear in a
window on the wall screen, showing a small map of
the room with the controllable lights. He uses
this interface to dim the lights. Tom opens up
his play list and presses play. The music comes
out of the room's large speaker system. Sally
enters the living room from the sliding doors to
the outside and walks over to the PC. She has to
manually log in, since she hasn't previously
identified herself. She brings up a Word document
that is an invitation to a shindig she and Tom
are hosting. Wanting Toms input, she asks him if
she can use the large room display. He uses the
remote to release control of the wall screen, and
she uses the room's controls on her PC to move
her session to that display.
11
e-Class by Georgia Tech.
12
Objective

• An attempt to study the impact of Ubiquitous
  Computing on education
• Support both teaching and learning in a
  university through the introduction of automated
  support for lecture capture

13
Stage 1 Feasibility Studies (1/4)

• Capture enough of the lecture experience to
  provide students both short long term benefits
• Four phases of Architectural Scheme
• Preproduction
• Live Capture
• Postproduction
• Access

14
Stage 1 Feasibility Studies (2/4)

• LiveBoard
• A 67-inch diagonal upright pen-based computer
• Integrate document and video conferencing and
  multimedia presentation with whiteboard

15
Stage 1 Feasibility Studies (3/4)

• ClassPad
• The pen can be used to write on the slide as the
  lecture progress
• Every time a slide was visited, ClassPad would
  record the time
• After class, the lecturer and student slides were
  converted into a series of HTML

16
Stage 1 Feasibility Studies (4/4)

• Comparison of teacher and student notes

17
Stage 2 A Living Laboratory (1/6)
18
Stage 2 A Living Laboratory (2/6)

• Zen? (zen-star) System
• An Electronic Whiteboard System in Java
• ZenMaster Central Server
• ZenPad

19
Stage 2 A Living Laboratory (3/6)
20
Stage 2 A Living Laboratory (4/6)

• Failed miserably as an access interface
• Return to the HTML only access interface of Stage
  1
• Time-stamp information for every pen stroke
• Draw every pixel
• Begin capturing Web browsing behavior during
  classes
• Stream Weaver make use of client-side image maps

21
Stage 2 A Living Laboratory (5/6)

• HTML only access interface by StreamWeaver

22
Stage 2 A Living Laboratory (6/6)

• Manual effort needed to coordinate activities
• Effort was needed to take a prepared presentation
  and import it into the system
• Need to synchronize
• Log of visited URLs and the digitally encoded
  audio needed to be placed in the central data
  repository
• Stream Weaver had to be invoked manually to
  produce the HTML notes

23
Stage 3 Recent Enhancements (1/2)

• Supporting nontraditional classroom use.
• DUMMBO (Dynamic Ubiquitous Mobile Meeting Board)

24
Stage 3 Recent Enhancements (2/2)

• Allow the students once again to take their own
  electronic notes in class

25
Paper by DCI
26
Disappearing Computing Initiatives

• Mission
• see how information technology can be diffused
  into everyday objects and settings, and to see
  how this can lead to new ways of supporting and
  enhancing peoples lives that go above and beyond
  what is possible with the computer today

http//www.disappearing-computer.net/
27
Paper

• EU project to develop and investigate
  technologies for the integration of paper and
  digital media
• Paper, printing and reader devices
• Server technologies to link information sources
• Technology Overview
• Use inductive ink to print a grid of invisible
  position information on paper
• Inexpensive, pen-like reader device
• Support flexible, dynamic authoring of two-way
  links between digital and paper documents

28
The Myth of the Paperless Office

• For decades, people have predicted the office of
  the future as a paperless office
• Documents generated electronically
• Documents published and distributed
  electronically
• Documents read electronically
• What has happened to this imminent Revolution?
• It is much simpler to scan a book by rapidly
  flicking through pages than to browse through a
  digital document
• Paper also supports forms of collaboration and
  interaction that are difficult to mimic in
  current digital environments
• ? Toward an integration of paper and digital
  technologies

29
Data Integration
30
The Concepts of Shapes, Layers, Users (1/2)

• Shape
• A selectable area of a page
• Layer
• Every shape is positioned on a specific layer.
• Formally,
• there are k layers numbered 1, 2, . . . , k, and
  that each shape Si is associated with exactly one
  layer i.e. layer(Si) j where 1 j k.
• For a given point (x, y), we write (x, y) ? Si to
  denote that the point (x, y) is contained within
  the shape Si. If two shapes Si and Sj are
  overlapping, i.e. there exists some point (x, y)
  such that (x, y) ? Si and (x, y) ? Sj , then the
  shapes must be on different layers i.e. layer(Si)
  ? layer(Sj ).
• At any point in time, the set of active layers is
  a subset of all layers. Let A denote the set of
  all active layers. Then A ? 1, 2, . . . , k.
• For a set of shapes S S1, S2, . . . , Sn, the
  selected shape associated with the given point
  (x, y) is given by the function s
• s(x, y) Sm
• Where
• layer(Sm) maxjSi ? S, (x, y) ? Si, layer(Si)
  j and j ? A

31
The Concepts of Shapes, Layers, Users (2/2)
32
Mapping of page positions to defined shapes
33
Dynamic Links betn Documents
34
Paper Server

• Consists of three main components
• The barcode decoder and visualization component
• XIMA component
• OMS Java data management component
• eXtensible Information Management Architecture
  (XIMA)