Extinction of Conditioned Behavior

1
Extinction of Conditioned Behavior
Effects of Extinction

• the rate of responding decreases

• response variability increases

• experiment by Neuringer, Kornell, Olufs (2001)

2
Extinction does not eliminate the original
learning

• Spontaneous Recovery

• Renewal

• Reinstatement

• US-devaluation

3
What is learned during extinction?
The non-reinforcement of a response in the
presence of a specific stimulus produces an
inhibitory S-R association that suppresses that
response whenever S is present
4
Partial Reinforcement Extinction Effect
Extinction is slower if a partial reinforcement
schedule, rather than CRF, was in effect before
extinction
Possible explanations

• Discrimination hypothesis
• Frustration hypothesis
• Sequential memory hypothesis

5
Stimulus Control of Behavior

• Stimulus control

• Stimulus discrimination

• Generalization Gradient

6
Stimulus and Response Factors in Stimulus Control

• sensory capacity

• relative ease of conditioning various stimuli

• overshadowing

• type of reinforcement

• certain types of stimuli are more likely to gain
  
• control over the instrumental response with
  positive
• reinforcement than with negative reinforcement

• experiment by Foree LoLordo (1973)

• type of instrumental response

• experiment by Dobrzecka, Szwejkowska,
• Konorski (1966)

7
Foree LoLordo (1973)
2 groups of pigeons trained to press a foot
treadle in the presence of a LT compound stimulus
For 1 group of pigeons, reinforcement was food
For the other group of pigeons, reinforcement was
the avoidance of shock
8
Foree LoLordo (1973)
Pigeons trained with food reinforcement responded
much more when tested with the light alone than
when tested with the tone alone
Pigeons trained with shock-avoidance
reinforcement responded much more when tested
with the tone alone than when tested with the
light alone
Selective association?
9
Dobrzecka, Szwejkowska, Konorski (1966)
10
Dobrzecka, Szwejkowska, Konorski (1966)
Two components to the auditory cues - quality
- location
Dogs trained on the left/right discrimination
responded mainly on the basis of the location of
the auditory cues
Dogs trained on the go/no go discrimination
responded mainly on the basis of the quality of
the auditory cues
Selective association?
11
Stimulus elements versus configural cues
So far, weve assumed that subjects treat
stimulus elements as distinct and separate
features of the environment (i.e., quality and
location of sounds)
However, subjects could treat a complex stimulus
as an integral whole that is not divided into
parts or elements
Configural-cue approach
Stimulus-element approach
12
Learning Factors in Stimulus Control
Effects of Training on Generalization and
Discrimination

• Nondifferential Training
• - S always present.

• Differential (or Discrimination) Training
• - Presence/Absence Training
• reinforced in the presence of S, not
  in its absence.

- Intradimensional Training reinforced in
the presence of S and not
reinforced in the presence of S-
13
Flat gradient
Non-Differential
More peaked gradient
Less generalization more discrimination
Presence/Absence
Most peaked gradient
Least generalization most discrimination
Intradimensional
14
What is learned in discrimination training?

• respond during the S

• do not respond during the S-

• or both (i.e., Spence)

15
Contextual Cues and Conditional Relations
Control of behavior by discrete stimuli (i.e., L
or T) - discrete cues are those which are
present for a brief time and have a clear
beginning and end
Contextual cues are the visual, auditory, and
olfactory cues of the room or place where the
discrete stimuli are presented
Contextual cues can provide an additional source
of control of learned behavior
16
Control by Contextual Cues
Conditioned Place Preference
barrier
Rats given drug and put in Black side
Rats given saline and put in White side
17
On test
Barrier removed
Rats placed in the center
18
On test
Rats spend more time in drug paired context
19
Contextual cues can control behavior if they
serve as a signal for a US or reinforcer -
experiment by Akins (1998) described on p. 243
Do contextual cues control behavior when they
serve as background rather than explicit signal
for reinforcer?
20
Experiment by Thomas, McKelvie, Mah (1985)
Trained pigeons on a line-orientation
discrimination in context 1
Context 1 S vertical line (90º) S-
horizontal line (0º)
After the discrimination was learned, the context
was changed and the discrimination training
contingencies were reversed
Context 2 S horizontal line (0º) S-
vertical line (90º)
After learning the second problem, pigeons given
generalization tests in which lines of
orientation between 0º and 90º were presented in
the two contexts
21
Results
22
Thomas et al (1985) showed that

• contextual cues can come to control instrumental
• behavior

• contextual control can occur without one context
  
• being more strongly associated with reinforcement
• than the other

• direct associations between context and
  reinforcer
• cannot explain the pattern of results

• each context associated with a different S/S-
• contingency

• Conditional Relations

23
Control by Conditional Relations
Binary relations

• S-R

• S-O

• R-O

In some cases the nature of a binary relation
is determined by a third event modulator

• S-(R-O)

This is termed a conditional relation
S/S- discriminations are conditional relations
24
Conditional Control in Pavlovian Conditioning
Rescorla, Durlach, Grau (1985)
Reinforced trials Noise Key light - food
Nonreinforced trials No Noise Key light no food
25
Conditional Control in Pavlovian Conditioning
In instrumental conditioning, modulators (i.e.,
S and S-) are called discriminative stimuli
In Pavlovian conditioning, modulators are called
facilitators or occasion setters

• A positive occasion setter signals that a CS-US
  contingency is in effect
• A negative occasion setter signals that a CS-US
  contingency is not in effect

26
Positive Occasion Setting (Facilitation)
Training If Occasion Setter is present then
CS 1 US If Occasion Setter is absent then CS
1 no US Result Thus CS 1 CR only when Occasion
Setter is present
27
Distinction between excitation and modulation
1. CS signals US BUT occasion setter signals the
relationship (if - then) between CS and US. 2. CS
elicits CR BUT occasion setter does not elicit a
CR 3. CS does not facilitate responding to a new
CS BUT an occasion setter can. 4. CS can
support second order conditioning, BUT occasion
setter does not. 5. CS presented alone results
in extinction of CR BUT occasion setter
presented alone, no extinction of CS " CR
28
Negative Occasion Setting
Training If Occasion Setter is present then
CS 1 no US If Occasion Setter is absent then
CS 1 US Result Thus CS 1 CR only when Occasion
Setter is absent is this familiar?
Conditioned Inhibition may be opposite of
facilitation, not excitation