Image Compression

1
DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF
JOENSUU JOENSUU, FINLAND

• Image Compression
• Lecture 4
• Arithmetic Code
• Alexander Kolesnikov

2
Arithmetic code

• Alphabet extension (blocking symbols) can lead to
  coding efficiency
• How about treating entire sequence as one symbol!
• Not practical with Huffman coding
• Arithmetic coding allows you to do precisely this
• Basic idea - map data sequences to sub-intervals
  in 0,1) with lengths equal to probability of
  corresponding sequence.
• 1) Huffman coder H ? R ? H 1
  bit/pel
• 2) Block coder Hn ? Rn ? Hn 1/n
  bit/pel
• 3) Arithmetic code H ? R? ? H 1
  bit/message (!)

3
Arithmetic code History

• Shannon 1948 a code string can be a binary
  fraction
• of pointing to the
  subinterval
• but Huffman code appeared ...
• Ellias 1963 successive subdivision of the
  interval
• -gt development of Golomb-Rice codes
• Chalkwijk 1972, Cover 1973
• Rissanen 1976 arithmetic code
• Pasco 1976 arithmetic code

4
Arithmetic code Algorithm (1)
0) Start by defining the current interval as
0,1). 1) REPEAT for each symbol s in the input
stream a) Divide the current interval L, H)
into subintervals whose sizes are
proportional to the symbols's
probabilities. b) Select the subinterval L,
H) for the symbol s and define it as the
new current interval 2) When the entire input
stream has been processed, the output should
be any number V that uniquely identify the
current interval L, H).
5
Arithmetic code Algorithm (2)

0.70
6
Arithmetic codeAlgorithm (3)
Probabilities p1, p2, , pN. Cumulants C10
C2C1p1p1 C3C2p2 p1p2 etc.
CNp1p2pN-1 CN11 0) Current interval
L, H) 0.0, 1.0) 1) REPEAT for each symbol
si in the input stream H ? L (H ?
L)C(si1), L ? L (H ? L)C(si) 2)
UNTIL the entire input stream has been
processed. The output code V is any number
that uniquely identify the current interval
L, H).
7
Example 1 Statistics
Message 'SWISS_MISS' Char Freq Prob
C(si), C(si1)) S 5 5/100.5 0.5,
1.0) W 1 1/100.1 0.4, 0.5) I 2 2/100.2 0.2
, 0.4) M 1 1/100.1 0.1, 0.2) _ 1 1/100.1 0
.0, 0.1)
8
Example 1 Encoding
S 0.5 0.5, 1.0) W 0.1 0.4, 0.5) I
0.2 0.2, 0.4) M 0.1 0.1, 0.2) __ 0.1
0.0, 0.1)
9
Example 1 Decoding
V ? 0.71753375, 0.71753500)
S 0.5 0.5, 1.0) W 0.1 0.4, 0.5) I
0.2 0.2, 0.4) M 0.1 0.1, 0.2) __ 0.1
0.0, 0.1)
10
Example 1 Compression?

• V ? 0.71753375, 0.71753500)
• How many bits do we need to encode a number V
• in the final interval L, H)?

0 1
00 01
10 11
000 001
010 011
101 101
110 111
0000 0001
1110 1111
1
0

• m4 bits 1624 intervals of size ?1/16.
• The number of bits m to represent a value in the
  interval
• of size ? m ?-Log2(?)? bits.

11
Example 1 Compression (1)

• V ? L, H) 0.71753375, 0.71753500)
• Interval size (range) r
• r0.50.10.20.50.50.10.10.20.50.50.0000
  0125
• The number of bits to represent a value in the
• interval L, H)L, Lr) of size r
• m?-log2r? ?-log2(0.0000125)? ?19.6? 20
  bits

12
Example 1 Compression (2)

• Entropy 1.96 bits/char
• Arithmetic coder
• a) Codeword V L ? V ? H
• V (0.71753407)10 (0.1011011110110000010
  1)2
• ... 20 bits ...
• 0.71753375 ? 0.71753407 ? 0.71753500
• b) Codelength m
• m?-log2(r)? ?-log2(0.0000125)? ?19.6?
  20 bits
• c) Bitrate R20 bits/10 chars 2.0 bits/char
• Huffman coder (1321144211)/102.2
  bits/char

13
Example 2 EOF (1)
Binary alphabet pa0.999, pb0.001. Message
Lower end L 0 (!) Range r(H-L)
(pa)10003.710-4 ?11.41?12 bits
0 ? V ? H Codeword V (0)10(0)2
(!)
14
Example 2 EOF (2)
Alphabet pa0.998, pb0.001, pEOF0.001. Message
Lower end L pEOF 0.001 HL1.3510-4
0.00135 Range r(H-L) (pa)1000pEOF
1.3510-4 0.001 ? V ?
0.001135 m?-log2((0.998)1000(0.001))??12.85?13
bits Code length 13 bits Code
V(0.0000000001001)2 2-10 2-12 0.0011...
Bitrate R13 bits /1000 pels 0.013
bits/pel Huffman code 1000 bits/data or 1.0
bit/pixel
15
Problem of practical implemention

• Input filesize 1 Mbyte
• Compressed filesize 100 Kbytes
• Size of codeword is 100 Kbytes?
• How to perform calculation for arithmetic coder?

16
AC Implementation with shifting
1) Scaled LH L?L10000 H?H10000-1 2)
New LH Push the digit and shift left
(?10)
In practice the binary arithmetic is in use!
17
Binary arithmetic code Q-Coder

LPS (less probable symbol) pLPSQ MPS (more
probable symbol) pMPS1-Q
si MPS LPS r ? r(1-Q)
?R-Q rQ ?Q L ? L
Lr(1-Q) ?Lr-Q
HLr
Qr
Lr(1-Q)
r(1-Q)
Keep 0.75 ? r ? 1.5 !
a) r ? r-Q or Q Encoding b) r ? 2r
Renormalization
L
18
Binary arithmetic code Renormalization
Out 1 Out 0
Out 01 or 10
Two renormalizations
19
Properties of arithmetic code

• In practice, for images, arithmetic coding gives
  15-30 improvement in compression ratios over a
  simple Huffman coder. The complexity of
  arithmetic coding is however 50-300 higher.

20
Contents

• Arithmetic code
• Implementation details