Intelligent HumanComputer Interfaces

1
Intelligent Human-Computer Interfaces

• Reinhold Behringer
• Leeds Metropolitan University

2
About Myself

• Running Stream Professor of Creative Technology
  since 2005.
• Before
• Research scientist at Rockwell Scientific (RSC),
  Thousand Oaks, CA
• Dr.-Ing. from UniBw Munich, Germany
• Interest
• Intelligent systems

3
About my Employer

• Leeds Metropolitan University
• Innovation North (Faculty for Information and
  Technology)
• Leading the
• Centre for Creative Technology
• Work areas
• RD of Human-Computer Interaction Technology
  Systems
• Context computer music, computer games, computer
  graphics

4
BarCamp

• Session or presentation?
• Interaction
• Some food for thought
• Ideas
• Concepts

5
Intelligent HCI

• Human-Computer Interface (HCI)
• Increased intelligence of computing system
• More intuitive use by humans.
• Non-expert users can use computers.
• Current problems
• Still interface hurdles keyboard/mouse is ok for
  office applications, but not suitable for
  computer as team partner
• Especially for mobile applications novel way of
  interacting is necessary.

6
Creative Technology Oxymoron?

• What is Creative Technology?
• Technology used in the creative domain.
• Creative domain
• Graphics, multimedia, music, games, video/film.
• What is an Oxymoron?
• Definition by Wikipedia
• Figure of speech that combines two normally
  contradictory terms.
• Greek Oxy sharp, moros dull.

7
Creative Technology Oxymoron?

• Premise in answering this question
• Technology itself is not creative.
• But it can enable human creativity
• Providing new methods for creating art, games,
  music.
• Can be supporting either conventional methods, or
  enabling completely new art forms.
• Initial answer yes, the term Creative
  Technology is an Oxymoron.
• But Artificial Creativity is a current
  research topic.

8
Intelligence at Interface

• Computer made more intelligent
• Automatic recognition
• Automatic actions which make sense.
• Goal
• Computer similar to human, in its interaction and
  responses.
• Will allow a more intuitive interaction.

9
My Projects in Computer Intelligence

• Computer Vision
• Automatic Cars
• Augmented Reality
• 3D Photo Browser in Google Earth
• The Computer as Musician
• Planned projects
• Video/graphics into music, music into graphics
• Wearable computing surveillance by the individual

10
Computer Vision

• Automatic recognition of
• Scene
• Context
• People
• Objects
• Applications
• Robotics
• Security / surveillance

11
PROMETHEUS Project (1996)

• Autonomous road vehicle drives in public road
  traffic among other among other traffic
  participants.
• Computer Vision is employed as a tool for
  detecting road markings.

12
Principle of Action

• Rene Descartes (1677) Tractatus de Homine

• Visual feed-back of recognition of lane markings
  keeps vehicle centred within the lane.

13
Results

• Driving from Munich to Denmark (and ret.) 1600
  km, 95 automatically
• Usual top speed 130 km/h (80mph)
• Top speed 180 km/h (110 mph)
• Collaboration with Mercedes
• technology used in truck, lade-departure warning
  system.

14
DARPA Grand Challenge

• Competition of driverless vehicles
• 2004 and 2005 in California desert
• 2007 in urban area.
• Prize awarded
• 2005 2M
• 2006 total of 3.5M

15
(No Transcript)
16
The SciAutonics Vehicle

• Robust Autonomous Sensor-Controlled All-Terrain
  Land-Vehicle RASCAL.

17
System Architecture
Obstacle detection
Sensors
GPS, compass
Road / path tracking
Behavior control central brain
Path planning map
Sensor fusion
RF E-stop, remote control
Vehicle control system
High level control
steering
brake
throttle
D/N/R
Odometer, INS
Low level control servos
Vehicle
18
Results Computer Vision
19
Video of RASCAL in May 2005
20
2007 Winner Boss from CMU
21
6 Cars No Major Crash
22
Robots and Human-Driven Cars
23
DARPA Grand Challenge

• Pictures from Motor Trend
• http//www.motortrend.com/features/auto_news/2007/
  112_0711_darpa_urban_challenge_reflections
• Winner CMU, got 2 M
• 2nd Stanford, got 1M
• 3rd VT, got 500k

24
Augmented Reality

• Output paradigm of placing information
  (perception artefacts) into the users perception
  of the real world as if they are part of the real
  world.
• Intuitive, seamless interaction with
  information in spatial real-world context.

25
Augmented Reality Applications

• Industrial maintenance,
• Training,
• Navigation,
• Hazard warning,
• Wherever spatial context information is provided.

26
Head-Worn Displays
SONY Glasstron (1998)
Ivan Sutherland (1960s)
Microvision HWD (2006)
Etched glasses (MicroOptical) (1999)
Lumus (2006)
27
Tracking

• Obtain users position and orientation relative
  to object of interest.
• Methods
• Computer Vision
• Specific markers can be recognized by system.
• Magnetic sensing.

28
Recognition of Markers
29
AR for System Diagnostics

• Industrial Pump Diagnostics
• Live sensor data were visualised on head-worn
  display.
• Tracking was achieved with computer vision
  methods for recognising and tracking of markers.

30
Head-Worn Display
31
Video
32
(No Transcript)
33
Future Possibilities

• Augmented Reality in truly mobile devices
• Mobile phones with cameras.
• Games with Augmented Reality.
• Computer Vision as tool for automatic capture of
  context.

34
Geographic Visualisation of Photos

• Fusion of individual photographs with location
  Geo-tagging
• Services Flickr, Panoramio
• Current state overlay on map.
• Future integrate photo with 3D model
• Either by texture mapping onto 3D model
• Or by overlay into the users view.

35
(No Transcript)
36
Location of Picture in Map
With Google Maps display
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
The Computer as Musician

• Can computer replace a musician in a band /
  orchestra?
• Requires
• Following the music score
• Being able to synchronize to human
  instrumentalists.

42
Beat Detection

• Capture music with microphone.
• Process signal to detect beat.
• Complex algorithms required
• Problem is solved only for very simple cases.
• Leeds Met Centenary PhD student Michael Ward is
  working on this project.

43
Conducting Music

• Tempo flow is felt by a musician
• If playing, tempo is done intuitively.
• For orchestra, conductor provides tempo reference
  to players.

44
Proposed Solution

Visual tracking of baton / hand
t
Create tempo map
Bridge the latencies by prediction and
extrapolation, based on music score.
Digital score in sequencer.
45
System Concept
2D point sequence 50 or 60 fps
Visual Tracking
Interpolation of motion timeline
Spline curve of motion
Acoustic Tracking
Analysis of motion timeline
Tempo and beat position Possible expression
Notes from MIDI file
Extrapolation to current time
Play synthesizer
46
Possible Applications

• Automated events for performance, e.g. lights,
  stage events, sounds, synchronous to music and
  determined by expression of conductor.
• Additional accompanying electronic instrument,
  playing synchronously with orchestra.
• Shaping time-flow of digital synthesizer
  rendition of music.

47
Future Projects

• Correlation of graphics/video and music/sound
• How do they relate to each other?
• How can music be represented graphically?
• How can visuals be translated into sound/music?
• System for automatic logging and recording.
• Automatic diary of activities, based on motion
  tracking, audio (microphone) and video (camera)
  input.

48
Conclusion

• Interesting projects with increased machine
  intelligence
• Automatic cars
• Computer as musician
• Novel interfaces
• Computer Vision as input
• Augmented Reality as output

49
Issues Raised

• Ethics
• Technology impact
• Funding
• Would a Challenge-based funding work in the UK?
• Intellectual
• Can machines be intelligent or creative?

50
Ethics

• Privacy?
• Ubiquitous recording.
• Fighting back ubiquitous surveillance by
  individual recording empowering the individual.
• Steve Mann (2007) "Smart Clothing Wearable
  Multimedia Computing and 'personal imaging' to
  Restore the Technological Balance Between People
  and Their Environments" ", Proceedings of the
  Fourth ACM International Conference on
  Multimedia, February 1997
• Develop new technology, which has negative
  consequences for humans?
• Hiroshima of Information Technology
• Much of interesting technology is funded by
  military.

51
Contact

• Reinhold Behringer
• Leeds Metropolitan University
• Headingley
• Leeds, LS6 3QS
• 0113-812-3716
• r.behringer_at_leedsmet.ac.uk

52
Topics

• Automatic Vehicles
• Artificial Creativity
• Novel Interfaces
• Computer Vision
• Augmented / Mixed / Mediated Reality
• Privacy concerns
• Challenge-based funding model