Bayesianness, cont

1
Bayesianness, contdPart 2 of... 4?
2
Administrivia

• CSUSC (CS UNM Student Conference)
• March 1, 2007 (all day)
• Thats a Thursday...
• Thoughts?

3
Bayesian class general idea

• Find probability distribution that describes
  classes of data
• Find decision surface in terms of those
  probability distributions
• Bayesian decision rule Bayes optimality
• Want to pick the class that minimizes expected
  cost
• Simplest case costmisclassification
• Expected cost expected misclassification rate

4
5 minutes of math

• For 0/1 cost, reduces to
• To minimize, pick the that minimizes

5
Bayes optimal decisions

• Final rule for 0/1 loss (accuracy) optimal
  decision rule is
• Equivalently, its sometimes useful to use log
  odds ratio test

6
Bayesian learning process

• So where do the probability distributions come
  from?
• The art of Bayesian data modeling is
• Deciding what probability models to use
• Figuring out how to find the parameters
• In Bayesian learning, the learning is (almost)
  all in finding the parameters

7
Back to the H/W data
8
Prior knowledge

• Gaussian (a.k.a. normal or bell curve) is a
  reasonable assumption for this data
• Other distributions better for other data
• Can make reasonable guesses about means
• Probably not -3 kg or 2 million lightyears
• Assumptions like these are called
• Model assumptions (Gaussian)
• Parameter priors (means)
• How do we incorporate these into learning?

9
5 minutes of math...

• Our friend the Gaussian distribution
• 1n 1-dimension
• Mean
• Std deviation
• Both parameters scalar
• Usually, we talk about variance rather than std
  dev

10
Gaussian the pretty picture
11
Gaussian the pretty picture
Location parameter µ
12
Gaussian the pretty picture
Scale parameter s
13
5 minutes of math...

• In d dimensions
• Where
• Mean vector
• Covariance matrix
• Determinant of covariance

14
Exercise

• For the 1-d Gaussian
• Given two classes, with means µ1 and µ2 and std
  devs s1 and s2
• Find a description of the decision point if the
  std devs are the same, but diff means
• And if means are the same, but std devs are diff
• For the d-dim Gaussian,
• What shapes are the isopotentials? Why?
• Repeat above exercise for d-dim Gaussian