Using Computer Vision to Test Web Display

1
Using Computer Vision to Test Web Display

• Xu Liu
• liuxu_at_cs.umd.edu

2
Why test web display?

• Display Bugs

Bug Opera 7.54
Normal IE6
3
More bugs
BUG - Opera
Normal - Firefox
Normal - Mozilla
Normal - IE
4
Where comes these bugs?

• Different parsers on DHTML, CSS
• Web designer doesnt follow W3C standard
• IE tolerant buggy HTML
• Java Script, ActiveX, Flash

5
Which kinds of bugs do we have?

• Text/Image Overlap
• Incorrect blank area
• Missing Text/Image

Generally they are all layout problems
6
How do we detect these bugs

• Is HTML source helpful?
• Yes, but we need a correct parser which is being
  tested
• HTML does not have straight forward relationship
  with display
• Eyes always tell the truth directly use the
  image of snapshot

7
Find the outlier

• Let them vote
• IE, Firefox, Mozilla, Netscapte, Opera, MyIE
• Anyone seems distinct from others is probably an
  outlier
• Assumption Major are correct, Minor are
  incorrect

8
Basic Question

• How do we compare 2 images? Are they same?

Missing Front and end
These 2 look different but they should be
considered the same
9
Simpler Question

• How do we compare 2 sequences?

S1 1 2 3 4 5 6 8 7 S2 1 2 4 5 5 5 6 7
Direct Compare S1-S2001101205
Dynamic Time Warping !!
In fact S1-S20, they should be considered
the same
10
Dynamic Time Warping (DTW)

• A widely used technique in signal processing
• Speech recognition, Image matching

Compare S1,S2
11
Can we directly apply DTW to compare web pages?

• No!

If we directly compare 2 pages Most of the error
will be omitted Local VS Global
Segmentation First!!
12
A Segmented Page
13
How do we derive segmentations

• Edge Detection First

Over Segment Merge
14
Next

• For every page
• We have all its segments
• For every segment
• If it CANNOT be found in other pages, possibly it
  is an error

15
Result
http//www.microsoft.com/smallbusiness/default.msp
x by Opera 7.54
16
Result
http//www.microsoft.com/learning/default.asp by
Opera 7.54
17
Shortage and Future Work

• Make segmentation more accurate
• Make the system completely automatic