vOICe 1.5

1
vOICe 1.5

• Enabling Technology - Final Project
• Gabe Su

2
Motivation

• Flaws of current system
• Hard to learn
• Not how humans view images
• Slow update time (1 sec)
• Current vOICe system is actually pretty good if
  you can learn it
• Has been compared to learning a foreign language
• My project should be considered a supplement to
  the vOICe system rather than a replacement

3
My Idea

• Do some sort of image processing to pick out
  important features
• Faster updates (100 ms or less)
• Constant sound coming from objects
• Let users own sense of motion be used to extract
  depth information

4
What I Did

• Convert vertical lines in the image into sound
• For helping blind users avoid half-open doors,
  walls, etc. Also may help with general spatial
  awareness.
• I didnt have enough time to do actual real-time
  system
• For this project, took a video and processed it
  with Matlab
• However, algorithm should be fast enough to work
  in real time

5
What I Did

• For each frame
• Step 1 Image processing
• Find all vertical lines
• Find most important vertical lines
• Analyze/label objects their position, size,
  etc.
• Step 2 Convert to sound
• Different sound for each object

6
Step 1 Image Processing

• Original Image

7
Step 1 Image Processing

• Blur/Vertical Line Filter
• Blur to get rid of noise

8
Step 1 Image Processing

• Vertical Median Filter
• Get rid of non-vertical lines and short vertical
  lines

9
Step 1 Image Processing

• Find the x longest lines
• Connected Components

10
Step 1 Image Processing

• Convert to 1D intensity map
• Hard to distinguish between high and low sounds
  anyways

11
Issues When Converting to Sound

• Cant use the same sound for every line
• Need to use different sound per line
• How many sounds/positions can one hear at a time?
• If you want one sound to follow a particular
  object, how to track that object?

12
Step 2 Converting to Sound

• Still working on this part
• Chose to track top 5 lines per image
• Assign a different sound to each line
• Why 5?
• I only had 5 different beep-type sounds
• Track same object in next frame based on minimum
  distance moved
• Wont always work, but hopefully good enough for
  most cases

13
Future Improvements

• Actually get it real-time
• Try it out with a webcam/headphones/laptop
• Better image processing
• Experiment with different filters, etc.
• More intuitive accurate conversion to sound
• In addition to sound position, assign image
  information to pitch, intensity, etc.
• Other types of image processing
• Face/Shape/Letter/Color recognition
• Will get better as computers get better

14
Demos (no sound yet)

• http//www.cs.unc.edu/okkay/projectvids.htm