Probability for NLP

1
Probability for NLP

• Andrew Roberts
• http//www.comp.leeds.ac.uk/andyr
• 15th February 2005

2
Probability Basics

• Experiment a process that yields outcome.
• Sample space the set of all possible outcomes.
• Event a subset of the sample space.
• eg getting heads when tossing a coin.
• Mutually exclusive if events cannot occur
  simultaneously.

3
Probability notation

• P(A) the prob. of event A occurring
• P(A?B) prob. of A or B
• P(AnB) prob. of A and B
• P(A) prob. of A not occurring (the complement
  of A)

4
Finite probability

• Experiment that has a finite number of outcomes.
• Therefore, each event has a finite probability.
• Experiment
• Throw a die. A number gt 4. B even number.
• Find P(A), P(B), P(A?B) and P(AnB)

5
Uniform distributions

• Every outcome in the sample space is equally
  probable.
• eg tossing coins
• rolling dice
• drawing a card from a deck

6
Joint probability

• P(A,B) is the prob. of having both A and B.
• Joint probability distributions describe the
  probs. of all possible permutations of events
  using random variables.

7
Joint probability (II)

• Find
• P(y ? z)
• P(x z)

8
Conditional probability

• P(A B) the prob. of A happening given that B
  has occurred.

9
Independence

• If events A and B do not influence each other
  then
• P(A B) P(A)
• and
• P(AnB) P(A)P(B)

10
Chain rule
11
Chain rule (II)

• Allows us to compute sequences of events.
• eg. prob. of sequence of letters, t, h and
  e
• P(t,h,e) P(t)P(h t)P(e th)

12
Bayes theorem
13
Bayes theorem (II)

• Meningitis causes stiff neck 50 cases
• Prob. of meningitis is 1/50000
• Prob. of stiff neck is 1/20
• What is the prob. of having meningitis if a
  patient has a stiff neck?

14
Additional resources

• MA12 examples
• http//www.comp.leeds.ac.uk/andyr/teaching
• Google directory
• http//directory.google.com/Top/Science/Math/Prob
  ability
• Wikipedia
• http//en.wikipedia.org/wiki/Probability