Pervasive Pixels

1
Pervasive Pixels

• Henning Schulzrinne (PI)
• Steven K. Feiner
• Gail Kaiser
• John Kender
• Kathleen McKeown
• Department of Computer Science
• Columbia University

2
(No Transcript)
3
(No Transcript)
4
Proposed Research

• Goal seamless mobile multimedia collaboration
  across distance
• Integrate advances across fields
• Collaborative work
• Graphical and visual interfaces
• Spoken language understanding and generation
• Vision sensing and understanding
• Networking and security

5
Contributions

• Contextual information management
• use workflow to determine display content
• multimedia summaries of past and present sessions
• Harmonizing physical and virtual environments
• map changing virtual information onto physical
  displays
• map layout of physical environment onto virtual
  space
• Network services
• clear, flexible interface to common services
• authentication and privacy support
• operating system infrastructure for large displays

6
Features of Research Infrastructure

• Large, instrumented multi-display workspaces
• Networked mobile devices of various capabilities
• Transparent and automatic adaptability to changes
  of place, platform or group
• Support for a wide range of hardware and
  software, from commercial to novel

7
Pervasive Pixels Teams
Contextualized Information Management
Workflow Kaiser
Summarization Kender, McKeown
Database Ross
Harmonizing Physical and Virtual Environments
Project Leader Schulzrinne
Environment Management Feiner
Capture and Display Nayar, Kender
3D Modeling Allen
Network Services
Enabling communications Schulzrinne
Virtual Display Nieh
Access and Privacy Keromytis
8
Pervasive Pixels Teams
Contextualized Information Management
Workflow Kaiser
Summarization Kender, McKeown
Database Ross
Harmonizing Physical and Virtual Environments
Project Leader Schulzrinne
Environment Management Feiner
Capture and Display Nayar, Kender
3D Modeling Allen
Network Services
Enabling communications Schulzrinne
Virtual Display Nieh
Access and Privacy Keromytis
9
Pervasive Pixels Teams
Contextualized Information Management
Workflow Kaiser
Summarization Kender, McKeown
Database Ross
Harmonizing Physical and Virtual Environments
Project Leader Schulzrinne
Environment Management Feiner
Capture and Display Nayar, Kender
3D Modeling Allen
Network Services
Enabling communications Schulzrinne
Virtual Display Nieh
Access and Privacy Keromytis
10
Pervasive Pixels Teams
Contextualized Information Management
Workflow Kaiser
Summarization Kender, McKeown
Database Ross
Harmonizing Physical and Virtual Environments
Project Leader Schulzrinne
Environment Management Feiner
Capture and Display Nayar, Kender
3D Modeling Allen
Network Services
Enabling communications Schulzrinne
Virtual Display Nieh
Access and Privacy Keromytis
11
Pervasive Pixels Teams
Contextualized Information Management
Workflow Kaiser
Summarization Kender, McKeown
Database Ross
Harmonizing Physical and Virtual Environments
Project Leader Schulzrinne
Environment Management Feiner
Capture and Display Nayar, Kender
3D Modeling Allen
Network Services
Enabling communications Schulzrinne
Virtual Display Nieh
Access and Privacy Keromytis
12
The Department of Computer Science

• A growing faculty
• 5 new assistant professors (pictures and areas
  here)
• 1 new full professor Peter Belhumeur
• 1 new full professor 9/02 Julia Hirschberg
• A large student population
• 80 PhDs
• 155 MS
• 380 undergraduate majors
• Internal, fluid collaboration
• Interdisciplinary connections

13
Building Critical Mass

• Networking
• Schulzrinne, Misra, Yemini
• links to EE
• Vision
• Allen, Belhumeur, Kender, Nayar
• links to computer graphics
• Human Computer Interaction
• graphical and visual user interfaces Feiner,
  Kender
• spoken and written language Hirschberg, McKeown
• links to speech and multimedia in EE

14
Areas of Growth

• Security and systems
• Computer engineering
• Computer graphics
• Theory
• Sofware systems (e.g., programming languages)

15
Administrative Support

• Institutional cost share on this proposal 1
  million in total
• 100,000/year for 5 years (Provost)
• 100,000/year for 5 years (Dean)
• Support for growth slots, space
• Financial support
• return in revenue (e.g., Columbia Video Network)
• Technical transfer support
• dedication of Columbia Innovative Enterprise
  services

16
Current Departmental Facilities

• Shared infrastructure of over 20 Sun and Pentium
  servers
• Two teaching laboratories
• 40 seat student lab (CLIC) of Intel Pentium PCs,
  dual projection capability, wireless microphone,
  video mixer and multiple cameras
• 30 seat Microsoft Research Lab
• Nortel Meridian telephone switch
• Research infrastructure of gt 500 nodes plus
  servers
• Staffed by 4 professional system administrators

17
Research Infrastructure

• User-based personalization user location
• magnetic card readers
• active badge system
• triangulation on mobile devices
• visual tracking
• Stationary setups
• Seminar room
• 3 meeting rooms
• 12 faculty offices
• 5 public areas
• portable units

18
Seminar room

• High-resolution DV video camera, 2 pan-tilt
  speaker cameras, Omnicam omnidrectional audience
  camera, video switcher, 16 ceiling mounted
  microphones, electronic whiteboard, 3 XGA
  high-brightness video projectors

19
Meeting Room

• 2 remote-controlled pan-tilt video cameras,
  Omnicam, conference table PZM boundary
  microphones, automatic audio mixer, ceiling
  speakers, 3 high-lumen XGA video projectors

20
Faculty Office

• Mimio electronic whiteboard, XGA video projector,
  Ethernet speaker phone, wal-mounted pan-tilt
  video camera, PocketPCs

21
Public areas

• Walk-by stations and laboratories
• XGA video projector, steerable ceiling-mounted
  dome video camera, embedded ceiling mounted
  netowrked Pentium computer, celing speaker,
  Andrea DA-400 array microphone, Microtouch IBID
  600 whiteboard.
• Picture here

22
Design for walk-by collaboration

• Figure 1 here

23
Functionalities

• Conferencing
• Internet conferencing server to mis IP and PSTN
  audio streams
• Interconnection with analog phone
• Digital hybrid connects digital or analog sound
  to existing telephone system in classroom
• Netowrk voice-over-IP interface attached to
  Nortel Meridian PBX for 20 simultaneous
  conversations
• Multi-processor servers and IA64 compute and
  database server, plus others
• File storage
• Face, speaker and fingerprint recognition
• 2 printers and tape library system in year 4 for
  backup

24
Remaining Agenda

• Research Overview
• Contextualized Information Management (Kaiser)
• Harmonizing Physical and Virtual (Feiner)
• Network Services (Schulzrinne)
• Genomics testbed (Noble)
• Responding to reviews (Kender)
• Telephony demo (Schulzrinne)
• Lunch
• Demos
• Visit with administrators
• Demos
• Small group meetings