Introduction to Machine Learning

1
Introduction to Machine Learning

• Dmitriy Dligach

2
Representations

• Objects
• Real-life phenomena viewed as objects and their
  properties (features)
• Feature Vectors
• Examples
• Text classification
• Face recognition
• WSD

f0 f1 fn
3
Supervised Learning

• Vector-value pair
• (x0, y0), (x1, y1), , (xn, yn)
• Task learn function y f(x)
• Algorithms
• KNN
• Decision Trees
• Neural Networks
• SVM

4
Issues in Supervised Learning

• Training data
• Why are we learning?
• Test data
• Unseen data
• Overfitting
• Fitting noise reduces performance

5
Unsupervised Learning

• Only feature vectors are given
• x0, x1, , xn
• Task group feature vectors into clusters
• Algorithms
• Clustering
• k-means
• mixture of gaussians
• Principal Component Analysis
• Sequence labeling
• HMMs

6
Supervised Example Decision Trees
7
A Tree
8
Word Sense Disambiguation (WSD)

• bat (noun)
• http//wordnet.princeton.edu/perl/webwn
• http//verbs.colorado.edu/html_groupings/

9
Another DT Example

• Word Sense Disambiguation
• Given an occurrence of a word, decide which
  sense, or meaning, was intended.
• Example, run
• run1 move swiftly ( I ran to the store.)
• run2 operate (I run a store.)
• run3 flow (Water runs from the spring.)
• run4 length of torn stitches (Her stockings had
  a run.)

10
WSD

• Word Sense Disambiguation
• Categories
• Use word sense labels (run1, run2, etc.)
• Features describe context of word
• near(w) is the given word near word w?
• pos words part of speech
• left(w) is word immediately preceded by w?
• etc.

11
Using a decision Tree

• Given an event (list of feature values)
• Start at the root.
• At each interior node, follow the outgoing arc
  for the feature value that matches our event
• When we reach a leaf node, return its category.

pos
noun
verb
I saw John run a race by a river.
near(race)
near(stocking)
yes
no
yes
no
near(river)
run1
yes

• run4

run3
4pm
12
WSD Sample Training Data
Features Word
POS near(race) near(river) near(stockings) Sense
Noun No No No run4
Verb No No No run1
Verb No Yes No run3
Noun Yes Yes Yes run4
Verb No No Yes run1
Verb Yes Yes No run2
Verb No Yes Yes run3
13
Unsupervised Example K-Means

• Distance between two objects
• Cosine distance
• Euclidean distance
• Algorithm
• Pick cluster centers at random
• Assign the data points to the nearest clusters
• Re-compute the cluster centers
• Re-assign the data points
• Continue until the clusters settle
• Hard clustering vs. soft clustering

14
Interactive Demos

• K-Means
• http//home.dei.polimi.it/matteucc/Clustering/tuto
  rial_html/AppletKM.html
• SVMs
• http//www.csie.ntu.edu.tw/cjlin/libsvm/GUI

15
ML Reference

• Tom Mitchell Machine Learning
• http//home.dei.polimi.it/matteucc/Clustering/tuto
  rial_html/AppletKM.html
• http//www.aaai.org/AITopics/html/machine.html