Ideas for project

1
Ideas for project
2
Church et al. article

• Article discussing appropriate statistical
  measures for lexicography
• Explains use of MI (to measure similarity) and
  t-score (to measure difference)
• Available at
• http//wortschatz.uni-leipzig.de/sbordag/aalw05/R
  eferate/ 02_Assoziationen_Church/Church91.pdf

3
Searching for synonyms

• Try using WordNet online
• http//wordnet.princeton.edu/

4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Doing a t-test

• To compare two words w1, w2 by seeing if their
  cooccurrance with a third word wc is
  significantly more likely than chance
• Use the formula for the t-test as given in Church
  et al.
• Result is in standard deviations
• t gt 1.65 means difference is significant

9
Getting the data

• Probability is estimated using corpus data
• All you need is the frequency of the two word
  sequences
• You can get these from the usual BNC search
• You dont need a value for N as the equation can
  be simplified

10
(No Transcript)
11
Examples from Church et al.
w1 w2 wc f(w1 wc) f (w2 wc) t
strong powerful showing 33 0 5.74
strong powerful support 152 11 11.04
strong powerful \,_PUN 1118 414 17.99
strong powerful defence 24 4 3.78
strong powerful economy 12 2 2.67

strong powerful than 8 158 -11.64
strong powerful figure 4 23 -3.66
strong powerful minority 0 7 -2.65
strong powerful of 4 52 -6.41
strong powerful post 0 2 -1.41