Tutorial on Bayesian Decision

1
Tutorial on Bayesian Decision

• ZHENG Haomian
• Department of Computing
• Hong Kong Polytechnic University

2
Outline

• Review in Probability Theory
• Bayesian Theory
• Applications for Recognition and Classification
  Problem

3
Gaussian Distribution (1)

• Coin Example
• P(head)P(tail)0.5
• N times
• Gaussian when N is very large

4
Gaussian Distribution (2)

• Score of Quiz Example
• Mean
• Variance
• Most student near the Mean

5
Related Parameters for Bayesian

• Prior Probability
• Posterior Probability
• Conditional Probability

6
Bayesian Rule

• P(A) is the prior probability or marginal
  probability of A. It is "prior" in the sense that
  it does not take into account any information
  about B.
• P(AB) is the conditional probability of A, given
  B. It is also called the posterior probability
  because it is derived from or depends upon the
  specified value of B.
• P(BA) is the conditional probability of B given
  A.
• P(B) is the prior or marginal probability of B,
  and acts as a normalizing constant.
• Intuitively, Bayes' theorem in this form
  describes the way in which one's beliefs about
  observing 'A' are updated by having observed 'B'.

7
Example Problem for Bayesian
8
Solution for Example (1)

• Intuitively we know
• X should be in Class A if it satisfied
• Otherwise it should be in Class B
• How to Calculate?

9
Solution for Example (2)

• Then we can generate the Decision Rule
• Think About
• Hint Exponential Probability

10
Solution for Example (3)

• Solution

11
Decision Problem (1)

• Communication Problem
• In communication, we transmit only 0 and 1
• Usually they are converted into -1 and 1
• Channel with 0-mean Gaussian Noise
• How to decide the signal at the receiver side?

12
Decision Problem (2)
13
Decision Problem (3)

• Face Recognition Problem

14
Thank You