Partsofspeech

1
Parts-of-speech
2
Word prediction again

• ... can ??? ...
• Restriction of alternatives
• bigram frequency
• Any other information available?
• The string can may represent three different
  words
• can -- verb can -- aux
• can -- noun

3
POS can help!

• ... can/AUX ???
• ... can/NOUN ???
• ... can/VERB ???

4
POS type of word

• Knowing the type of a word will help when
• - predicting words
• - assigning structure to a sentence to
• - determine its meaning
• - pick out parts (phrases in it)
• - determine the content of a text
• - ...

5
Lexical words

• Content-bearing words
• Nouns
• Verbs
• Adjectives
• Adverbs

6
Lexical parts-of-speech

• Usually defined by
• Meaning
• Structure
• Distribution
• Function

7
Open word classes

• Easily acquire new members
• ? high number of members
• Differs between speakers, domains, ...

8
Closed classes

• Stable
• Low number of members
• Stable between speakers, domains, ...
• Grammatical meaning function words
• (short words)

9
Grammatical word classes

• Prepositions
• Auxillary verbs
• Pronouns
• Negation
• Conjunctions
• Coordinating words, phrases, clauses
• Subordinating clauses
• Articles
• Infinitive marker
• ...
• Lexical ? grammatical

10
Problems with assigning a POS to a word

• Ambiguity Many wordforms are ambiguous between
  different parts-of-speech
• Unknown words You cant know every word!

11
Part-of-speech Tagging
12
An example simple stochastic POS-tagging

• Predict the next tag
• Will you fax these ...

13
Will you fax these ...

• Bigram frequencies
• PRON?VERB
• PRON?NOUN
• Lexical frequencies
• fax NOUN
• fax VERB
• Contextual probability
• Lexical probability

14
Longer sequencies of (ambiguous) words to tag

• Viterbi algorithm
• Dynamic programming

15
Sparse data

• Smoothing methods

16
Different approaches

• Stochastic tagging
• Rule-based tagging
• Transformation-based tagging

17
Similarities

• Same kind of data needed
• (Usually) uses some of window with finite length

18
What do you need?

• A set of labels (of word types) to attach to the
  words The tagset
• Knowledge base (Lexicon), already tagged texts,
  ...
• Some method for choosing among alternative tags
  for a word
• Some method for handling unknown words

19
ENGTWOL tagger

• Broad-covering lexicon
• Manually constructed disambiguation rules
• pick tag X if ... discard tags X, Y, Z if
  ...

20
Transformation-based tagging

• Retag the training data with some default
  mechanism the most frequent tag, ...
• Find rules that can change the default tagged
  into the (correctly tagged) data.
• Iterate! (use the changed data as input in the
  retagging procedure)

21
Transformation-based tagging

• Rule templates
• Rules may be implemented as finite state
  tranducers

22
Unknown words

• Assign
• All tags in the tag set
• All lexical tags
• The most frequent tag
• The most frequent lexical tag
• ...

23
Tagset

• Depends on the language and on the application.
• POS
• How many different classes?
• POS morphological features

24
Performance

• Training data
• 95-97
• Comparison between stochastic and
  machine-learning techniques (Megyesi, 2002)
• Stochastic tagger best
• Different weaknesses ambiguities, unknown words
• Different sensitivity to size of training corpus

25
Example

• Hungarian
• Transformation-based tagging
• 600 different tags
• Training data 5K word
• Performance 60-70