Probabilistic Stochastic CFGs

1
Probabilistic (Stochastic) CFGs

• PCFGs do worse than n-gram models, which take
  local lexical context into account need to use
  heads with nonterminals
• Bias of small trees over large trees
• Recursions
• Inside-outside probabilities
• Textbook describes Chomsky, bottom-up algorithm
• and left-corner top-down algorithm

2
Problems

• O(n3m3) where n nonterminals
• m words
• Local maxima very sensitive to initialization
  parameters use simulated annealing
• Learning requires 3 times more nonterminals than
  original grammar
• Learned nonterminals may not have meaning.