On Beyond GUI: Wearable, Heterogeneous, Adaptive

1
On Beyond GUIWearable, Heterogeneous, Adaptive

• Steven Feiner
• Computer Graphics User Interfaces Lab
• Department of Computer Science
• Columbia University
• New York, NY 10027
• Supported in part by AFRL, NSF, ONR, and gifts
  from Alias Systems, IBM, Mitsubishi
  ElectricResearch Labs, and Microsoft
• CHI 2006 Workshop What is the Next Generation
  ofHuman-Computer Interaction? Montreal, Canada
  April 23, 2006

2
Wearable

• Cursorless
• Eyes-free

• Corners of watch bezel serve as tactile
  landmarks to guide users finger

G. Blaskó and S, Feiner. An interaction system
for watch computers using tactile guidance and
bidirectional segmented strokes. Proc. ISWC 2004.
3
Wearable

• Cursorless
• Eyes-free

G. Blaskó and S, Feiner. An interaction system
for watch computers using tactile guidance and
bidirectional segmented strokes. Proc. ISWC 2004.
4
Wearable

• Cursorless
• Eyes-free

G. Blaskó and S, Feiner. An interaction system
for watch computers using tactile guidance and
bidirectional segmented strokes. Proc. ISWC 2004.
5
Wearable

• Cursorless
• Glanceable

G. Blaskó, C. Narayanaswami, and S, Feiner.
Prototyping Retractable String-Based Interaction
Techniques for Dual-Display Mobile Devices. Proc.
CHI 2006.
6
Heterogeneous

• Coordinated use of different displays and
  interaction devices
• Multiple displays
• Projected desktop
• Handheld
• High-res monitor
• See-through head-worn
• Multiple interaction devices
• Multi-touch table
• Tracked hands/heads
• Speech

H. Benko, E. Ishak, and S, Feiner. Collaborative
mixed reality visualization of an archaeological
excavation. Proc. ISMAR 2004.
7
Heterogeneous

• Coordinated use of different displays and
  interaction devices
• Multiple displays
• Projected desktop
• Handheld
• High-res monitor
• See-through head-worn
• Multiple interaction devices
• Multi-touch table
• Tracked hands/heads
• Speech

H. Benko, E. Ishak, and S, Feiner. Collaborative
mixed reality visualization of an archaeological
excavation. Proc. ISMAR 2004.
8
Heterogeneous

• Coordinated use of dimensionalities
• Pull 2D image into 3D model
• Push 3D model into 2D image

H. Benko, E. Ishak, and S, Feiner.
Cross-dimensional gestural interaction techniques
for hybrid immersive environments. Proc. VR 2005.
9
Heterogeneous

• Coordinated use of dimensionalities
• Pull 2D image into 3D model
• Push 3D model into 2D image

H. Benko, E. Ishak, and S, Feiner.
Cross-dimensional gestural interaction techniques
for hybrid immersive environments. Proc. VR 2005.
10
Adaptive

• Change user interface in response to environment
• Modify layout of virtual objects to avoid overlap
  in AR

B. Bell, S. Feiner, T. Höllerer. View
management for virtual and augmented reality.
Proc. UIST 2001.
11
Adaptive

• Change user interface in response to environment
• Modify transparency to reveal important content
  on desktop

E. Ishak, and S, Feiner. Interacting with hidden
content using content-aware free-space
transparency. Proc. UIST 2004.
12
Three Themes for Future UIs

• Wearable
• Cursorless ? Eyes free, glanceable,
• Heterogeneous
• Mix of users, displays, devices,
  dimensionalities,
• Adaptive
• Dynamically modified in response to changes in
  environment, tasks,