PERVASIVE COMPUTING AT NIST

1
PERVASIVE COMPUTINGAT NIST
Martin Herman, Chief INFORMATION ACCESS USER
INTERFACES DIVISION INFORMATION TECHNOLOGY
LABORATORY NATIONAL INSTITUTE OF STANDARDS AND
TECHNOLOGY
2
VISION

• Small inexpensive computers and sensors
• in every device, appliance and piece of
  equipment
• in buildings, offices, classrooms, homes, cars,
  factories
• people carrying portable and wearable computers
• networked to each other and the Internet
• A trillion devices connected to the Net!"
• sensing and reacting intelligently to the
  environment
• Information instantly accessible anywhere and
  anytime
• An emerging new paradigm for IT
• Enormous new global market
• transforming the way U.S. industry does business
• transforming the way people live and learn
• requiring the development of a measurement and
  standards
• infrastructure ensuring that U.S.
  manufacturers are not at a
• disadvantage

3
NISTs Role

• Pervasive computing is in infancy
• Many required technologies are immature have
  high risk
• NISTs unique role
• develop metrics, tests and standards to push
  forward underlying technologies and help
  pervasive computing mature and grow
• develop test data and test scenarios/tasks
• provide impartial, large-scale evaluations across
  many different systems and companies
• develop interoperability specifications and test
  methods
• perform provide tools for conformance testing

4
Current Industry Efforts

• Information appliances
• Web TVs, screen phones, net-enabled smart
  handheld computers, internet gaming consoles,
  internet automotive systems, electronic books
• Integration into Internet
• wireline high-speed access (high-speed data over
  phone lines cable)
• wireless high-speed access (terrestrial/LMDS
  satellite services)
• short-range wireless LANs (HomeRF, Bluetooth)
• networking/communication protocols for
  information appliances (Jini, Universal Plug
  Play, Inferno, Odyssey, T spaces, JetSend)
• Applications
• networked household (TV set-top boxes, handheld
  communicators, PDAs, PCs, laptops,
  printers/scanners, digital cameras)
• networked automobile (GPS, PDAs, cell phones,
  video games, e-mail, voice mail, web surfing,
  navigation assistance)
• BodyLANs (e.g., FedEx integration of handheld
  computers, scanners printers to determine
  status of packages)

5
Proposed NIST Efforts

• Measurements and standards for long-term as well
  as short-term needs.
• Rich, natural human-computer interaction
• Extensive information presentation
  visualization
• Dynamic networking
• Sophisticated multi-media information extraction
• Collaborative working environments
• Security reliability
• System integration and interoperability
• Tight integration of IT with electronic,
  semiconductor, optical, manufacturing, chemical,
  materials, building and fire technologies

6
Worldwide Information Appliance ForecastUnit
Shipments (000)
7
Appliances vs PCsUS Shipments, Consumer
Devices,In Millions
8
Applications Scenarios Personal Assistants

• Extension of todays pocket organizers, smart
  cards, smart mobile phones
• New functions electronic money clip, locator,
  visual communicator
• Provide access to personal data, monitor offices
  homes, transfer funds, make secure transactions
• Increase effectiveness of peoples time extend
  the range of their actions

Personal assistant for youth/elderly Provide ID
information Vital data GPS-based location Remote
transmission of location Help me communication
function Access to medical information Funds
accessible for emergencies Voice I/O Embedded
camera for visual interaction
Personal assistant for adult Secure access to
information Embedded biometric sensors Coordinate
users schedule Retrieve store phone, fax,
e-mail Personal e-business Electronic purchasing
of tickets Electronic toll payments Electronic
banking Integration with office computer,
printer, in-car digital maps, WWW Voice I/O,
voice-to-text, text-to-voice Embedded camera for
visual interaction
9
Applications Scenarios Smart Spaces
Smart Spaces are work spaces embedded with
computers, information appliances, and
multi-modal sensors that allow people to work
efficiently, together or individually, through
unprecedented access to information and help from
computers.

• Identify and perceive users and their actions
  and goals
• Speech, natural language, computer vision input
• Interaction with information-rich sources
• Provide for the use of mobile devices and
  receivers
• Provide extensive information-presentation
  capabilities
• Understand and anticipate user needs during task
  performance
• Distributed and local collaboration, including
  collaboration with field personnel mobile
  workers
• Provide improved memory and summaries of
  activities and deliberations for later use

Smart Space Examples Business meeting
room Medical consultation room Training/education
facility Military command center Crisis
management center
10
Technologies for ITL Involvement

• Rich, natural HCI
• Speech natural language understanding
• Interactive dialogue
• Text-to-speech generation
• Detecting, localizing, tracking people
• Recognizing people
• Activity gesture recognition
• Situation awareness
• Summarizing activities events recorded by
  sensors
• Computer vision processing
• Distributed multi-sensor fusion integration
• Integrated information presentation
  visualization

• Dynamic networking
• Mobile/wireless communications
• Real-time system communications protocol
• Data compression
• Distributed databases
• Multi-media information extraction
• Document retrieval
• Question answering
• Collaborative working environments
• Distributed local collaboration
• Security
• Computer/Network Security
• Biometric authentication
• System integration technologies
  interoperability

11
(No Transcript)
12
Perceptive Interface Overview
Display Services
World Model
Pattern Classifiers
Acoustic DSP
Video DSP
13
Smart Space Test Bed
Large Screen Display

• 64 ADCs/Microphones
• TI 320C32 60MHz with 512k of RAM
• 3 cameras/video capture
• Gigabit ethernet
• Servers for data
• Pervasive devices
• Palm tops
• Tablets
• Wireless LAN

Camera Elements
Equipment Room
Microphone Phased Array
Array Beams
14
One-dimensional Microphone Array Geometry
IAUI Prototype Analog Arrays
59 Microphones at 15 mm Inter-sensor distances
45 Microphones at 20 mm Inter-sensor distances
15
Frontal Face Detection
Template
Template matching
Skin pixel extraction
Face localization
Largest cluster of skin pixel coordinate
16
Face feature extraction
Edge detector
Cropping
Hough Transform

• Crop largest skin pixel cluster from frame
• Apply
• Edge detector to emphasize circular shapes
• Hough Transform for eye-extraction
• Face Normalization to gallery conditions

17
(No Transcript)