Extraction of Overlayed Text from TV Video Sequences

1
Extraction of Overlayed Text from TV Video
Sequences

• Justin Weeks and Keshav Attrey
• Computer Vision
• CS 4495 Fall 1999
• for Prof. Thad Starner

2
Project Goals

• Extract non-trivial text from video
• Using contrast
• Using motion
• Do not use OCR
• Language independent
• Potentially faster
• High potential for false positives

3
Description

• Detection over a text-filled video involves
• Determining where the text is.
• Determining when the text changes.

4
Safe Assumptions

• Text is horizontally aligned in some way.
• Text objects are close to other text objects.

5
Potentially Unsafe Assumptions

• Text is horizontally aligned on bottom.
• Text is all one color.
• Text is not too large.

6
Wheres the Text in this Video?

• Contrast method
• Take 16 color ranges from black to white.
• In each color range look for objects that are
• Horizontally aligned
• Not too big
• Close to one another
• Output for this frame is the composite of
  filtering for all slices.
• Compare each frames output to determine text
  changes.

7
Wheres the Text? Continued

• Difference method
• Determine intensity differences between current
  frame and past frames.
• When difference is small, increase probability
  matrix entry for this pixel.
• After several frames, we have a good idea of what
  portions of image are text.
• Mask current frame and previous frame with
  current text-pixel estimation.
• If the correlation between these maskings is low,
  then this frame contains new text.
• Output text-pixel estimation and reset
  probability matrix.

8
Welcome to the Results Show!

• Single frame results
• Color thresholding with known text intensity
  range is trivial.
• Color thresholding with unknown intensity range.
• Multi-frame results (temporal)
• Determining what is text.
• Determining text changes.
• Quest for the ideal probability threshold.
• Programmatically vs. Manually

9
Conclusion

• Identifying text without knowing what the
  characters look like is hard.
• Im not impressed with the results
• Good considering the data
• Need higher resolution video to minimize effect
  of noise
• Higher frame rate would yield higher per-pixel
  text-certainty.
• Doing it real-time seems to be out of reach.
• There must be a better way...