Information Theory

1
Information Theory

• Mathematical prereqs
• probability
• logarithms
• Codes
• Communication Channels

2
Logarithms
3
Logarithm Identities
4
Binary representations
1 bit 0, 1
2 bits 00, 01, 10, 11
3 bits 000, 001, 010, 011, 100, 101, 110, 111
4 bits 16 symbols
5
Quantifying Information
information gained in number of bits
symbol set
expected information gain
Examples
6
(No Transcript)