Aucun titre de diapositive

1
Image processing Image and image sequence
compression bases
Session 2005-2006
Ecole Centrale Marseille
2
Goal

• Give in 2 hours some technics of compression
• Introduction of mathematical toolsBased on
  information theory

3
Plan
I) Introduction II) Tools III) Lossless
compression (redundancy) IV) Lossy
compression scalar quatization vector
quatization V) Norms
4
Digital images and sequences
L
L
C
C
Image
Sequence
Different formats or standards
L x C pixels by image
RGB,YUV(NTSC,PAL,SECAM)
8 bits / pixel
5
Formats
RGB red, green blue YUV Y black and white
component U, V color information YCbCr
scaled and offset versions of YUV
6
Compression why ?
Problem
Amount of data
Transmission
Decrease de bitrate with compression technics
Elimination of redundancy Quantization ...
Storage
7
Amount of datas
Exemple of format PAL itu-r 601
Resolution 720 columns and 576 rows YCbCr
422 Interlaced (frame ou trame ? image)
Y 720 x 576 414720 CbCr 2 x 360 x 576
414720 Total 829440 bytes / image
Si 25 images / sec (50 trames / sec)
Total 20,7 Mb / sec
8
Evolution
Increase of quality, increase the size of
images, a lot of images ...
VISION
More and more data
9
Compression
Data compression addresses the problem of
reducing the amount of data required to represent
given quantity of information
NxM pixels
Examp l e
x0
y0
M
Coordinates, radius, gray levels (5 values)
N
10
Plan
I) Introduction II) Tools
11
Transmission of information
Mathématical theory of communication W. Weaver et
C.E. Shannon
source of information
destination
message
message
signal
source of noise
transmitter
receiver
12
compression system model

Source encoder
Channel encoder
Channel decoder
Source decoder
Channel
f
f
noise
encoder
decoder
exemple
Symbol encoder
mapper
quantizer
Inverse mapper
Symbol decoder
. . .
Transform DCT,FFT,WT
SQ,VQ
13
f(x,y) is fed into the encoder which creates a
set of symbols from the input data

f
f
May or may not be an exact replica of

f
f

Error free (lossless compression) Lossy
compression

?
f
f
14
Measures
Quality error between original signal and
received signal
Bit rate quantity of bits
Possibility complexity of the system (software
or hardware)
15
Fidélity criteria
16
Fidélity criteria
where 255 is the maximum value of the signal
17
Fidélity criteria
Subjective
Value Rating 1 excellent extremely high
quality 2 fine high quality 3 passable 4
marginal 5 inferior very poor
image 6 unusable cant watch it
18
Measuring information
A random even E that occurs with probability P(E)
is said to contains
Units of info
Self information
P(E)1 event always occurs P(E)0 event never
occurs P(E)0.999 gt small information
Log2 ? bits
19
Measuring information
The average information per source output is
ENTROPY
H ? ? uncertainty ? ? information ?
Shannon
20
Definitions
N1 bits before compression N2 bits after
compression
Compression ratio
Relative data redundancy
21
Technics
Image compression
Sequence compression
Lossless compression quality is preserved
low compression
Lossy compression quality decreases high
compression
22
cameraman
Cam10.jpeg
623
200
23
Cam99.jpg
Cam50.jpg
625
625
24
Plan
I) Introduction II) Tools III) Lossless
compression (redundancy)
25
Error free compression
Redundancy reduction
- coding reduction - interpixel redundancy
dpcm,... - psychovisual redundancy
26
Code length (Longueur de code)
Code 1 ? L3
Code 2 ?
27
Huffman coding
Assignes fewer bits to the more probable gray
levels than to the less probable gt data
compression
Most popular
Variable-length code
Code optimal, code unique
28
Huffman coding
Two steps 1/ create a series of source
reductions by ordering the probabilities of the
symbols under consideration and combining the
lowest proba symbols into a single symbol that
replace them in the next source reduction 2/
code each reduced source, starting with the
smallest source and working back to the original
source. (start with 0 or 1 arbitrarily)
29
(No Transcript)
30
Example
Lavr 0.4 x 1 0.3 x 2 0.1 x 3 0.1 x 3
0.1 x 4 0.06 x 5 0.04 x 5 2.2
bits/symbol
Ex 010100111100 means a3 a1 a2 a2 a6
31
Arithmetic coding
Un mot de code (nombre) pour une suite de symboles
Precision pb
32
Run length coding
Represent each row of an image or bit plane by
sequences of lengths that describe successive
runs of black and white pixels. fax, standard
compression,...
Basic concept code each group of O s (1 s)
encountered in a left to right scan of a row by
its lenght.
(pattern recognition)
33
Rlc example
Original data
5 5 10 10 10 6 6 6 6 6 6 18
9
13 bytes
10 bytes
2 5 3 10 6 6 1 18 1 9
Compressed data
(10/130.77)
34
Bit plane coding
Decompose a multilevel image into a serie of
binary images and compress each image (binary)
via one of several well known binary compression
method
Ex code each group of O s (1 s) encountered
in a left to right scan of a row by its length.
8 binary images
8 bits image
35
Lemple-Ziv and Welch coding
Pattern recognition
If a sequence of non identical data values is
repeated more than once in the data stream, then
this repetition can be exploited to compress the
data.
5 10 7 10 6 5 10 7 10 6 10
Original data
Compressed data
Ca Cb Ca Cb 10
36
Predictive coding
Simple predictive method
21
DCT
i
x
x
x
x
j
0 no prediction 1 A 2 B 3 C 4 AB-C 5 A(B-C)/2 6
B(A-C)/2 7 (AB)/2
37
Plan
I) Introduction II) Tools III) Lossless
compression (redundancy) IV) Lossy
compression transform quantisation scalar
quantisation vector quantisation
38
Compression system
compression
coding
transform
code binary flow
coefficients
index
image
! Attention au mot codage en anglais et français
39
Statistics
40
Transform coding
Source image
reorder data same information
Entropy encoder
T
quantizer
DCT,FFT,WT
SQ,VQ
8x8 blocs
Compressed image
Table specifications
Table specifications
T-1
Entropy decoder
dequantizer
Table specifications
Table specifications
Reconstructed image
41
Transform coding
A transform is used to map the image into a set
of transform coefficientswhich are then quantized
and coded
For natural images, a significant number of
coefficients have small magnitudes and can be
coarsely quantized (set to zero) without
significant distortion. If information of image
is well organised ! Goal decorrelate pixels.
Pack as much info as possible into the smallest
number of pixels Compression is not achieved
during the transform !
42
JPEG
43
Discrete Fourier Transform
FDCT
8x8 bloc
IDCT
Source image
44
Quantizers
Scalar quantizer one pixel at a time Vector
quantizer group of pixels
45
Scalar quantization
Consider the input x . As a matter of convention
x is considered to be mapped to yi?R if x lies
in sj, si1. yi belongs to a finite set of L
values
X 12.46
Y12
Degradation of signal
46
Definition
Uniform scalar quantizer
x1
x2
x3
xN-2
xN-1
Y1
Y2
Y3
YN
YN-1
Quantizer design problem select the best Xi and
Yj for a particular optimization criterion and
pdf of input
Decision levels Reconstruction levels
47
Uniform optimal quantizer (1D)
Lloyd-Max quantizer
Optimum mean square
Criterion minimise rms for a fixed number of
quantization levels
Reconstruction levels are the centroids of areas
under p(x) over specified decision
interval Decision levels are half way between
reconstruction levels
48
Caracteristic
Y
Y8
x8
x1
x
Quantizer error difference between input and
the reproduction level
Y1
e
x8
x1
x
(Truncation error if input is out of range)
49
Quantization noise
Assume e step e quantization error (xi-yi)
Hyp e small and xi equally distributed in
-e/2,e/2
p(e)
p(e) is uniform in -e/2,e/2
1/e
e/2
-e/2
e
Noise power
50
Signal to noise ratio
Example sine wave with amplitude Vm
Power of signal
Quantization noise
51
Compander
Pb low values are badly quantized
Pdf should be cte between ? Vmax
Companding compressing expanding
(Loi u)
52
SQ --gt Vector quantization
Shannon ? of samples --gt ? efficiency of
coding Zador ? of reproduction vectors
--gt ? mse
53
Definition
?
?
Y
,
Y
?
Y
,...,
Y
codebook
L
1
2
n
C
C
C
?
R
?
?
?
i
?
j
?
i
,
j
?
1
,
.
.
.
,
L
et
?
si
i
i
j
X(x
Ci voronoi Yi codebook vectors
54
Shéma de principe
55
(No Transcript)
56
Remarks
Beaucoup de blocs dans l image sont
similaires Beaucoup de blocs possibles ont peu de
chance d apparaître Réduire le nombre de
vecteurs de reproduction possibles ex 1024
vecteurs (10 bits) (taille dictionnaire) débit
moyen 10 / (4x4) 0.625 bpp Minimiser la
distorsion Type de source (image transformée en
général)
57
Vector quantization

• Codebook design
• minimum distortion
• fixed codebook size / storage
• Codebook search
• fast - efficient
• Decoding
• reconstruction

Codebook
1
value
i
index
i
L
58
Codebook design
Classification LBG algorithm
Linde - Buzo - Gray 1980
Codebook design
Minimisation of distortion criteriatraining
vectors
adapted
complex
Iterative algorithme
Quantization
Full search, ...
complex
Lattices
Conway Sloane 1980
fast
- codebook vectors regularly spaced in the space
- fast algorith
59
Lattices
n
exemple
3
Z
60
Vector quantization

• Hardware and software point of viewon lineoff
  linecomplexityCD, satellite transmission, data
  basesstandards -gt hardware implementation

61
VQ multirésolution
Allocation de bitscroissante
Dictionnaire multirésolution
Allocation des débits
Décomposition multirésolution
62
C50
C120
originale
63
Plan
I) Introduction II) Généralités III)
Compression sans perte réduction de
redondance IV) Compression avec
pertes quantification scalaire quantification
vectorielle Compression de séquences
64
Méthodes

• 1ère méthode
• Décomposition spatio-temporelle
• Quantification vectorielle algébrique
• 2ème méthode
• Estimation - Compensation de mouvement
• Quantification vectorielle algébrique

65
Décomposition spatio-temporelle
Q V A
R
Q V A
LR
Q V A
L
mC
LL
Décomposition temporelle
Décomposition spatiale
66
Principle
j'
j"
j
Compensation
?
i"
?
?
?
i'
?
?
?
i
j'
j
?
?
t1
?
?
i'
Estimation
?
?
i
?
?
j
?
?
t
-gt
d
-gt
v
?
i
-gt
u
t-1
67
Motion estimation


?

?
?
It
It
Désindexage
Indexage
Quantification


-

Désindexage


It

Retard


It
Retard
It
Estimation Compensation
Estimation Compensation
Retard
Retard


It1

It1

It-1
It-1
Décodeur
Codeur
68
Compression de Miss America
original
C100
C220
69
Bibliography

• Digital Image Processing, R.C. Gonzalez R.E.
  Woods, Addison Wesley
• Vector quantization, R.M. Gray, IEEE ASSP
  Magazine, april 1984
• Image coding using vector quantization a review
  , N.M. Nasrabadi R.A. King, IEEE Trans. On
  Com., vol36 n8, august 1988.
• Image data compression, A.K. Jain, Proc. Of IEEE,
  vol69 n3, march 1981
• Travaux, thèse et notes personnelles

70
End
Thank you for your attention
71
Extra slides
72
Video Compression Standards

• Video Compression Standards extend the transform
  based, still image compression techniques of the
  previous sections to include methods for reducing
  temporal or frame to frame redundancy
• Depending on intended application, the standards
  can be grouped into two broad categories
• Video teleconferencing standards
• Multimedia standards

73
Videoconferencing standards

• A number of teleconferencing standards have been
  defined by International Telecommunication Union
  (ITU)
• H.261 intended for operation at affordable
  telecom bit rates with delay of less than 150ms
• H.262 based on MPEG2 standard for multimedia
  compression
• H.263 designed for very low bit rate video, in
  the range of 10 to 30 kBits/s
• H.320 desgined for ISDN lines
• All are based on MPEG type compression scheme

74
Multimedia Video Compression

• Developed for Video on Demand, HDTV, CD image
  video database
• MPEG-1 for high quality video as on CDROM
  supporting bit rate os 1.5MBits/s
• MPEG2 addresses applications involving video
  quality between NTSC/PAL and CCIR a range that is
  suitable for cable TV broadcast, narrow channel
  satellite broadcasting, 2 to 10Mbits/s
• MPEG1 and MPEG2 are meant for storage and
  transmission efficiency

75
MPEG standards

• MPEG4 provides
• Improved video compression efficiency
• Content based interactivity
• AV object access and efficient integration of
  natural and synthetic data
• Universal access
• All these make segmentation necessary though no
  approach is proposed for segmentation.
• It requires change on the creation side similar
  to multitrack audio capture
• Targets bit rates of 5 to 64 Kbits/s for mobile
  devices and 4Mbits/s for TV and film

76
MPEG 7

• A sophisticated system meant for content
  management
• Encodes high semantic information
• Shows the dedication of MPEG community to go from
  a mere compression scheme to information
  management system

77
MPEG 21

• Builds in the concept of digital rights
  management
• Watermarking algorithms for videos and images

78
Lossless predictive coding
See tr
Eliminates interpixel redundancy of closely
spaced pixels ? extract and code only new
information difference between actual and
predicted value of that pixel
fn
en

Binary flow
input
Symbol encoder
-
encoder
predictor
Nearest integer

fn
en
fn
Symbol encoder
Binary flow


decoder
predictor

fn
79
Codebook

• Codebook size C vectors k pixels /
  vector I size of index R bit rate

I 6 bpp - 12bpp C 32 - 4096 k 4 - 32 R
0.5 - 2 bpp
Ordres de grandeur
80
Q. V. Algébrique
Quantification Vectorielle AlgébriqueQuantifi
cation VectorielleDictionnaire réseau
régulier de points
??Définition du réseau ??Définition de la
relation de quantification ??Indexage
Quantifier
81
Alogorithme LBG
(0) INITIALISATION  Séquence d'apprentissage
formée de vecteurs X (training vectors) (les N
premiers vecteurs, sélection aléatoire,
splitting)  Dictionnaire initial formé de
vecteurs Y (1) CLASSIFICATION des vecteurs
X Obtention d'une PARTITION de la
séquence Critère de distorsion à minimiser 
d(X,Y)  X-Y2 (2) OPTIMISATION du
représentant de chaque classe de la
partition Calcul du centre gravité cen(C) de
chaque classe C Nouveau dictionnaire  Y  cen(C)
82
Codebook

• Méthodes classiques par apprentissage
• Méthode géométrique

- Vecteurs de reproduction calculés par
classification - Quantification par Full
Search - Codebook adapté à la statistique
/apprentissage - Points dun réseau régulier (pas
de calcul) - Algorithme rapide de
quantification. - Troncature du réseau et scaling
adaptés à la statistique.
83
Codebook design

• Algorithm of CLASSIFICATION LBG Linde - Buzo -
  Gray 1980
• Minimisation distortion criteria
• Iteratif
•  KOHONEN neural network
• TREILLIS Conway Sloane 1980
• Codebook organised in the space

??Définition du réseau ??Définition de la
relation de quantification ??Indexage
84
Coding redundancy
Variable length coding
L gray levels --gt histogram
nk of times that the kth gray level appears
n total number
L(rk) of bits used to represent each value of
rk
Average of bits to represent each pixel
Average length of codewords
TOTAL of bits of pixels x Lavr
85
Algorithme de Huffman

• 1. Soit L une liste des probabilités des symboles
  de la source qui sont associées aux feuilles
  d un arbre binaire.
• 2. Choisir les deux plus faibles probabilités
  dans L et construire un nœud parent. Etiquetter
  une branche par 1 et l autre par 0
• 3. Remplacer les deux probabilités par la somme
  et l associer au nœud formé. Terminer si la
  liste contient un seul élément Sinon aller à
  l étape 2

86
Arithmetic coding
- It generates nonblock coding - One to one
correspondance between source symbols and
codewords doesn t exist - An entire sequence of
source symbol is assigned a single arithmetic
code word which defines an interval of real
numbers between 0 and 1 - As of symbols
increases --gt the interval to represent it
becomes smaller - Each symbol of the message
reduces the size of interval in accordance with
its probability of occurrence
87
Arithmetic coding
Un mot de code (nombre) pour une suite de symboles
Precision pb
88
Lossless predictive coding
See tr
Eliminates interpixel redundancy of closely
spaced pixels ? extract and code only new
information difference between actual and
predicted value of that pixel
fn
en

Binary flow
input
Symbol encoder
-
encoder
predictor
Nearest integer

fn
en
fn
Symbol encoder
Binary flow


decoder
predictor

fn
89
Predictive coding
Simple predictive method
21
DCT
i
x
x
x
x
j
0 no prediction 1 A 2 B 3 C 4 AB-C 5 A(B-C)/2 6
B(A-C)/2 7 (AB)/2
90
QS --gt Quantification Vectorielle
Shannon à montré que les performances du codage
d une source sans mémoire sont améliorées (on
approche les courbes débit-distortion)si l on
augmente la taille des blocs d échantillons. Za
dor a montré aussi que la QV peut donner une
meilleure erreur quadratique moyenne par
dimension dans le cas d un nombre de vecteurs de
reproduction asymptotiquement grands
91
Definition
?
?
Y
,
Y
?
Y
,...,
Y
codebook
L
1
2
n
C
C
C
?
R
?
?
?
i
?
j
?
i
,
j
?
1
,
.
.
.
,
L
et
?
si
i
i
j

• define codebook
• quantize
• code (index)

92
Etapes de quantification vectorielle
0) Quantifier 1) Trouver lindex I dun vecteur
de reproduction Yi
2) Trouver le vecteur de reproduction Yi
connaissant lindex I
93
Codebook

• Codebook size C vectors k pixels /
  vector I size of index R bit rate

I 6 bpp - 12bpp C 32 - 4096 k 4 - 32 R
0.5 - 2 bpp
Ordres de grandeur
94
Q. V. Algébrique
Quantification Vectorielle AlgébriqueQuantifi
cation VectorielleDictionnaire réseau
régulier de points
??Définition du réseau ??Définition de la
relation de quantification ??Indexage
Quantifier
95
Alogorithme LBG
(0) INITIALISATION  Séquence d'apprentissage
formée de vecteurs X (training vectors) (les N
premiers vecteurs, sélection aléatoire,
splitting)  Dictionnaire initial formé de
vecteurs Y (1) CLASSIFICATION des vecteurs
X Obtention d'une PARTITION de la
séquence Critère de distorsion à minimiser 
d(X,Y)  X-Y2 (2) OPTIMISATION du
représentant de chaque classe de la
partition Calcul du centre gravité cen(C) de
chaque classe C Nouveau dictionnaire  Y  cen(C)
96
Codebook

• Méthodes classiques par apprentissage
• Méthode géométrique

- Vecteurs de reproduction calculés par
classification - Quantification par Full
Search - Codebook adapté à la statistique
/apprentissage - Points dun réseau régulier (pas
de calcul) - Algorithme rapide de
quantification. - Troncature du réseau et scaling
adaptés à la statistique.
97
Codebook design

• Algorithm of CLASSIFICATION LBG Linde - Buzo -
  Gray 1980
• Minimisation distortion criteria
• Iteratif
•  KOHONEN neural network
• TREILLIS Conway Sloane 1980
• Codebook organised in the space

??Définition du réseau ??Définition de la
relation de quantification ??Indexage