Automatic Image Rescaling Preserving Design Intention

1
Automatic Image Rescaling Preserving Design
Intention

• Research Update
• Prasad Gabbur

2
Goal
Original
3
Approach

• Split input image into background (bg) and
  foreground (fg) layers
• Scale the bg and fg layers separately

Background
Foreground
4
Background layer scaling

• Scale and shift elements to fit new page
• Ignore aspect ratio
• Classification of background elements

Area
Horizontal (Top, Bottom)
Vertical (Left, right)
5
Background layer scaling

• Scale and shift of background elements

Area
To fit new page
Horizontal
To fit new page width
Vertical
To fit new page height
Sx gt Sy
Sy gt Sx
6
Foreground layer scaling

• Scale and shift elements to fit new page
• Preserve aspect ratio
• Classification of foreground elements

Corner (TL, TR, BL, BR)
Horizontal (Top, Bottom)
Vertical (Left, Right)
7
Foreground layer scaling
Corner
Horizontal
Vertical
Sy gt Sx
Sx gt Sy
8
Foreground layer scaling

• Scaling preserves aspect ratio (Scale factor
  min (Sx, Sy))
• Shifting preserves distance ratio (dL/dR const,
  dT/dB const)

Shift of horizontal elements
Shift of vertical elements
dT_old
dL_new
dR_new
dT_new
dL_old
dR_old
Horizontal
Vertical
dB_old
dB_new
Sy gt Sx (dT_new / dB_new) (dT_old / dB_old)
Original
Sx gt Sy (dL_new / dR_new) (dL_old / dR_old)
9
Element extraction

• Elements are connected regions in the foreground
  or background layer

Background layer(s)
Foreground layer(s)
Multilayer image
Extract alpha channel
Label connected components
10
Connected component labeling

• Group together spatially connected pixels as a
  single component
• Each component is assigned a unique integer label

4-connectivity
8-connectivity
11
Connected component labeling

• Region coloring algorithm (4-connected) Ballard
  Brown, 1982
• Each pixel (Xc) in the image is scanned with the
  following mask

new_label 1 If (Xu ? background Xl ?
foreground), then label (Xc) label (Xl) Else
if (Xu ? foreground Xl ? background),
then label (Xc) label (Xu) Else if (Xu ?
foreground Xl ? foreground), then label (Xc)
min ( label (Xl), label (Xu) ) Else label
(Xc) new_label new_label new_label 1
12
Connected component labeling

• Basic region coloring algorithm is slow
• Requires multiple passes through the image
• A faster version is realized with the help of a
  custom data structure
• An array of the above data type can store
  information about all connected components
• Only one pass through the image is necessary

13
Connected component labeling

• A single raster scan of the image gives rise to
  following structure

Image with two connected components
Data structure at the end of a single image scan
14
Connected component labeling

• Links in the array can be visualized as a tree
  structure
• Nodes in the tree are equivalent labels of a
  connected component

A fictitious connected component
15
Connected component labeling

• Trees with different configurations are possible
  depending on region complexity
• All branches merge at the bottom

One branch
Two branches
Four branches
16
Connected component labeling

• Resolving label equivalences
• All the equivalent labels are assigned the least
  value among them by stepping through the tree

17
Connected component labeling

• Resolving label equivalences

Resolve
18
Elements

• Each connected component in the background or
  foreground layer is an element
• Geometric properties (bounding box center and
  limits) are computed as part of the labeling
  process
• Elements are classified based on the geometric
  properties

Background layer
Foreground layer
19
Scaling issues

• Scale up
• Sparse distribution of pixels in the output image
• Bilinear interpolation to fill in pixel values
• Scale down
• Aliasing due to sub-sampling
• Low pass filtering before sub-sampling

Thanks to Jian Fan, HP Labs.
20
Results

• Background layer

Scaled (Sx gtS y)
Original
Labeled
Scaled (Sy gtS x)
21
Results

• Foreground layer

Scaled (Sx gtS y)
Original
Labeled
Scaled (Sy gtS x)
22
Next

• Stitch together the foreground and background
  layers
• Work on an XML design for input

23
Thank you!