Learning

1
Learning Reward

• Chapter 6

2
Learning Objectives

• Describe the principles of classical conditioning
  
• Define acquisition, extinction, spontaneous
  recovery, stimulus generalization and
  discrimination.
• Give an example of a second order conditioning.
• Explain development of phobias based on the rules
  of classical conditioning.
• Describe a behavioural technique for treatment of
  phobias
• Define the cognitive perspective on learning
• Describe the principles of operant conditioning
• How is operant conditioning different from
  classical conditioning?
• Describe the process of shaping.
• Describe the Premack principle.
• Describe different schedules of reinforcement and
  how they affect learning.
• How does observation of others produce learning?
• What is the biological basis of reward?
• How does learning occur at the neuronal level?

3
Behaviorism

• Pioneer John Watson
• Inspired by Pavlov
• Emphasizes
• Overt behaviour
• Environment
• Inspired by John Lockes idea of human infants
  as Tabula Rasa

4

• Classical Conditioning

5
Classical Conditioning

• Watson Inspired by Pavlovs experiments
• Salivary reflex (dogs fig 6.3)
• Classical conditioning
• A neutral object that elicits a reflexive
  response when it is associated with a stimulus
  that already produces that response

6
Classical Conditioning
7
Principles of Classical Conditioning

• Fig 6.4
• US The stimulus that has the natural property to
  elicit a response in the organism
• UR The automatic and unlearned response to a US
• CS A neutral stimulus
• CR An acquired response to a CS
• It is learned

8
Classical Conditioning
9
Classical Conditioning Acquisition

• The initial learning of a behavior
• The gradual formation of an association between
  the conditioned and unconditioned stimuli
• Pavlov contiguity the critical element in the
  acquisition of a learned association
• Subsequent research

10
Classical ConditioningExtinction

• Unlearning the association made between CS US
• When CU no longer predicts the unconditioned
  stimulus ?
• CR Extinguishes (CR disappears)
• Figure 6.5

11
Classical ConditioningSpontaneous Recovery

• After a while, the extinguished CS again produces
  a CR
• Even a single pairing of the CS with the US will
  reestablish the CR
• Is temporary and will quickly fade
• New Acquisition The organism learns that
  original association no longer holds true

12
Acquisition, Extinction, Spontaneous Recovery
13
Classical ConditioningStimulus Generalization

• After the CR is established, not only the US but
  also the stimuli that are similar to CS will be
  able to produce CR.
• But the farther the CS is from this similar
  stimulant, the less it produces CR
• Occurs when stimuli that are similar but not
  identical to the CS produce the CR
• E.g.
• The adaptive role of generalization

14
Classical Conditioning Stimulus Discrimination

• Learning to differentiate between two similar
  stimuli if one is consistently associated with
  the unconditioned stimulus and the other is not
• E.g.
• The adaptive role

15
Generalization Discrimination
Varying Pitch Frequencies
16
Classical ConditioningSecond-Order Conditioning

• A conditioned stimulus becoming associated not
  with US, but with other stimuli that themselves
  are associated with the US,
• E.g.,
• Occurs implicitly, without our awareness or
  intention

17
Classical Conditioning Phobias

• Phobias
• Fear conditioning
• Animals can be classically conditioned to fear
  neutral objects
• Watson Rayner (1920) 11-month-old Albert
• White rat alone- not afraid
• White rat paired with loud noise
• White rat alone- afraid
• Rabbit alone- afraid
• Fur coat alone- afraid
• The most important brain structure for fear
  conditioning

18
Classical Conditioning Phobias

• Treatment of phobias
• Counter conditioning
• Exposure to small doses of the feared stimulus
  while engaging in a pleasurable task
• Systematic desensitization (Wolpe, 1997)
• CS/CR1 (fear) ? CS/CR2 (relaxation)

19
Classical Conditioning

• Pavlov
• Contiguity as the critical condition for
  acquisition
• Equipotentiality
• Subsequent research
• Evolutionary significance of conditioned food
  aversions
• The role of biological preparedness in developing
  fears

20
Classical Conditioning

• The cognitive perspective on learning
• Prediction expectancy
• The Rescorla-Wagner model
• Unexpectedness of the US
• The blocking effect

21

• Operant Conditioning

22
Operant Conditioning

• Thorndikes Puzzle box
• Studying how cats learn
• The Law of effect
• Satisfying consequences underlie learning
• Behaviours followed by
• Positive outcomes are strengthened
• Negative outcomes are weakened

23
Thorndikes Puzzle box
24
Thorndikes Puzzle Box Results
25
Operant Conditioning

• Skinner Operant Behavior
• Operates on the environment
• Have effects which are instrumental in
  producing desired results
• Operant conditioning Instrumental learning

26
Operant Conditioning

• Shaping
• Process of rewarding approximations of desired
  behaviour to train animals and humans
• Rats in Skinner box
• Dogs to shake hands
• Humans toilet training

27
Operant ConditioningShaping (cont.)

• Reinforcer
• Primary Reinforcer Reinforcers that are innately
  satisfying (e.g., food, water, sexual pleasure)
• Secondary Reinforcer Reinforcement acquires its
  value through experience (e.g., praise, money)

28
Operant ConditioningShaping (cont.)

• Potency of reinforcers
• Some reinforcers are more powerful than others
• The value of a specific reinforcer could be
  determined by the amount of time an organism
  engages in that behavior when free to choose
  anything
• The Premack principle
• A more valued activity can be used to reinforce
  the performance of a less valued activity

29
Operant ConditioningReinforcement Punishment

• Reinforcement Increases the probability of
  behavior
• Positive R.administration of a pleasant stimulus
  (i.e., reward)
• Negative R.removing an aversive stimulus
• Punishment Reduces the probability of behavior
• Positive P. administration of an aversive
  stimulus
• Negative P.removal of a pleasurable stimulus

30
Fig. 6.12
Reinforcement Punishment
31
Schedule of Reinforcement

• How often should a reinforcer be given?
• The (time, frequency, consistency, and
  persistency of) behaviour is affected by the
  schedule of reinforcement
• Continuous reinforcement
• Partial reinforcement

32
Schedule of Reinforcement

• Continuous reinforcement
• Reinforcing every response
• Is rare in the real world
• Partial reinforcement
• Reinforcing only some responses according to the
• Number of behavioral responses or
• Passage of time
• Is more common than continuous reinforcement

33
Schedule of Reinforcement Partial Reinforcement

• Ratio schedules
• Reinforcing only some responses, based on
  occurrence of correct responses (units of
  behavior)
• E.g. Reinforcing every third or tenth occurrence
  of the behavior
• Interval schedules
• Reinforcing only some responses based on a time
  interval (units of time)
• E.g. Reinforcing once an hour

34
Schedule of Reinforcement Partial Reinforcement

• Fixed schedules
• Reinforcing consistently given following a
  specific number of occurrences or after a
  specific amount of time
• The rate of reinforcement is predictable
• Variable schedules
• Reinforcing at different rates or at different
  times
• i.e., after an average number of responses or
  time lapse on an unpredictable basis

35
ComparingContinuous Partial Reinforcements
36
Schedule of Reinforcement Partial Reinforcement

• Combination of
• Fixed Variable schedules
• Ratio Interval schedules
• ? ? ?
• Fixed ratio
• Fixed interval
• Variable ratio
• Variable interval

37
Partial Reinforcement Learning Curve
38
Behaviour Modification

• The application of operant conditioning
  principles to change human behaviour
• Five steps to better self-control
• 1. Define the problem
• 2. Commit to change
• 3. Collect data about self
• 4. Design a self-control program
• 5. Make the program last

39
Limitations ofOperant Conditioning

• Biological constraints of operant conditioning
• E.g.1. Raccoons
• E.g.2. Pigeons
• Reinforcement does not always have to be present
  for learning to take place
• Tolmans idea of latent learning
• Cognitive map
• Reinforcement affects performance more than
  learning
• Insight learning

40
Latent Learning
41
Similarity Between Classical Operant
Conditioning

• Both The processes of learning associations
• i.e., Learning that two events are connected

42
Differences BetweenClassical Operant
Conditioning

• Learning an association between
• Two stimuli
• A behaviour a consequence
• The organism is
• Passive
• Active

43
Observational Learning

• Learning fear aggression through observation
• Imitation Modeling
• Banduras Bobo doll observational studies
• Learning aggression through observation

44
Observational Learning

• Vicarious reinforcement
• Learning to do or not to do an action by
  observing others being rewarded or punished for
  performing the action.
• Observation of another person engaging in an
  action ?
• Activation of Mirror Neurons
• Adaptiveness of observational learning

45
The Biological Basis of Reward

• Intracranial Self-Stimulation (ICSS)
• Acts on same brain regions as natural reinforcers
• Activates dopamine receptors (in the nucleus
  accumbens, a part of limbic system)
• Secondary reinforcers dopamine

46
Learning at the Neuronal Level

• Learning is a result of alterations in synaptic
  connections (Donald Hebb)
• Cells that fire together wire together
• One neuron exciting another ?
• Strengthening the synapse between the two?
• The firing of one neuron becomes increasingly
  likely to cause the firing of the other

47
Learning at the Neuronal Level

• Habituation, a decrease in
• Behavioral response following repeated exposure
  to non-threatening stimuli
• Neurotransmitter release in the synapse
• Sensitization, an increase in
• Behavioral response following exposure to a
  threatening stimulus
• Neurotransmitter release in the synapse

48
Learning at the Neuronal Level

• Long-Term Potentiation (LTP)
• Occurs when the likelihood of an action potential
  in a post-synaptic neuron is increased by
  electrical stimulation of a pre-synaptic neuron