Introduction to Machine Learning

1

• Chapter 10
• Introduction to Machine Learning

2
Lets play a fun kids game

• Guess Who?
• You can ask me any yes/no questions you like
  about the appearance of my mystery person.
• By asking a series of questions (presumably the
  smallest number of questions needed) you want to
  deduce who is the mystery person.
• Lets try it.

3
What kind of intelligence do you need to play
this game?

• Could you use search techniques?
• Could you use logic techniques?
• Do you learn things during the game?
• Do you learn during an individual game?
• Did you learn anything that you would use in
  future games?

4
Which of these is learning

• From P and P- Q
• learn Q.
• deduction
• From Q and P - Q
• learn P.
• abduction
• From P(x1) and P(x2) and P(x3),
• learn that for all x, P(x).
• induction

5
Learning Adaptation

• Modification of a behavioral tendency by
  expertise.
• (Webster 1984)
• A learning machine, broadly defined is any
  device whose
• actions are influenced by past experiences.
  (Nilsson 1965)
• Any change in a system that allows it to
  perform better
• the second time on repetition of the same
  task or on another
• task drawn from the same population. (Simon
  1983)
• An improvement in information processing
  ability that results
• from information processing activity.
  (Tanimoto 1990)

6
Why bother with Learning

• Learning is essential for unknown environments
• When designers lack omnscience
• Learning is ueseful as a system construction
  method
• Just expose the agent to reality rather than try
  to write it down
• Learning modifies the agents decision mechanisms
  to improve performance

7
How do you learn?

• We are going to study several techniques
• Learning by example / Training
• Decision Trees
• Artificial Neural Networks

8
Learning by Example / Training

• Learning problems usually involve classifying
  inputs into a set of of classifications.
• Learning is only possible if there is a
  relationship between the data and the
  classifications.
• Training involves providing the system with data
  which has been manually classified.
• Learning systems use the training data to learn
  to classify unseen data.

9
Rote Learning

• A very simple learning method.
• Simply involves memorizing the classifications of
  the training data.
• Can only classify previously seen data unseen
  data cannot be classified by a rote learner.

10
Classification example
11
Concept Learning

• Concept learning involves determining a mapping
  from a set of input variables to a Boolean value.
• Such methods are known as inductive learning
  methods.
• If a function can be found which maps training
  data to correct classifications, then it will
  also work well for unseen data hopefully!
• This process is known as generalization.

12
Classification example
New data
Test set
Train set
Loan Yes/No
Model
Learning system
13
Classification example
Features height, weight
x
x
o
x
x
o
x
o
x
o
o
Height
x
o
x
o
o
x
x
x
x
o
o
x
o
o
x - weight-lifters o - ballet dancers
x
Weight
14
Classification example - Simple Model
Features height, weight
Decision boundary
x
x
o
x
x
o
x
o
x
o
o
Height
x
o
x
o
o
x
x
x
x
o
o
x
o
o
x - weight-lifters o - ballet dancers
x
Weight
15
Classification example - Complex model
Features height, weight
Complex Decision boundary
x
x
o
x
x
o
x
o
x
o
o
Height
x
o
x
o
o
x
x
x
x
o
o
x
o
o
x - weight-lifters o - ballet dancers
x
Weight
Note A simple decision boundary is better than a
complex one - It GENERALIZES better.
16
Concept Learning

• Concept learning involves determining a mapping
  from a set of input variables to a Boolean value.
• Such methods are known as inductive learning
  methods.
• If a function can be found which maps training
  data to correct classifications, then it will
  also work well for unseen data hopefully!
• This process is known as generalization.

17
Hypotheses

• A hypothesis is a vector of variables
• In concept learning, a training hypothesis is
  either a positive or negative (true or false).
• A ? is used to indicate that any value will be
  suitable.
• A Ø is used to indicate that no value will be
  suitable.

18
Hypotheses - Example

• Each hypothesis represents a set of driving
  conditions.
• If a hypothesis is positive, then it represents a
  safe scenario.
• For example

• This represents the hypothesis that it is safe to
  drive fast in rain 10ft behind the next car
  having drunk 2 units of alcohol.
• This would be a negative training example, as
  clearly it is not safe!

19
General to Specific Ordering

• This hypothesis is the most general hypothesis.
  It represents the idea that it is safe to drive
  in any conditions
• hg
• The following hypothesis is the most specific
  hypothesis it says it is not safe to drive in
  any conditions
• hs
• We can define a partial order over the set of
  hypotheses
• h1 g h2
• This states that h1 is more general than h2
• One learning method is to determine the most
  specific hypothesis that matches all the training
  data.

20
Version Spaces

• A version space is the set of hypotheses that
  correctly map all the training data to their
  categories.
• A simplistic learning method would be to start
  from a version space of all hypotheses and to
  systematically remove all the ones that do not
  match the training data.