Operant conditioning

1
Operant conditioning
2
Operant conditioning
In classical conditioning, the presence of one
stimulus (e.g. meat powder) is conditional on the
presence of another stimulus (e.g., a
bell) What else can an animal learn, besides the
relationshipof two stimuli?
3
Operant conditioning
It is also possible for the animal to generate a
responseand for that response to have
consequences
Act cute, you get pet
Poop on the rug, you get scolded
4

• Note that the thing to be learned is not a UR.
  Animal emits a response (pooping, acting cute),
  and it is rewarded or punished.

5
Edward Thorndike
6
Thorndikes Law of Effect

• If a response in the presence of a stimulus is
  followed by a satisfying event, the association
  between the stimulus and the response is
  strengthened. If the response is followed by an
  annoying event, the association is weakened.

7
Today well cover

• Basics of operant conditioning
• What makes operant conditioningeffective.
• The problem of definition in o.c. (not just that
  animals seekrewards).

8
Thorndikes method was limited because each trial
took so long.
9
A stripped-down environment
10
Free operant curve, from a cumulative recorder
Steep slopemany responsesShallow slopefew
responses
11
What would the curve look like if 20 bar presses?
food?
12

• To really teach the animal you would shape its
  behavior. . .

13
Fixed ratio
Consistent ratio of number of responses number
of reinforcers
Steady response Easy to extinguish
Example factory Piece work
14
Variable ratio
Set ratio of number of responses number of
reinforcers, but can vary locally
Example slot machine
Rapid responseHard to extinguish
15
Fixed interval
First response after a specific amount oftime
since the last reinforcement
Example studying for exams
Little response untiljust before
reinforcementthen rapid responseFairly easy to
extinguish
16
Variable interval
First response after a some amount of time since
the last reinforcement amount of time can vary,
locally
Example checking email
Steady responseHard to extinguish
17
(No Transcript)
18
Contingencies dont just add good stuff. . .
Result
e.g., spanking
Increase probability of behavior
Decrease probability of behavior
e.g., food
Add to environment
Positive Reinforcement
Punishment
Action
Negative Punishment(Extinction)
Take away from environment
Negative Reinforcement
e.g., escape
e.g., being grounded
19
Complex contingencies Would this work?
Bar press reinforced, but ONLY when red light is
on.
YES! This is called differential reinforcement
20
How does differential reinforcement apply here?
21
Reinforcer food. Response hovering Differentia
l signal looking up.
22
Whats happening, and what should the birds do?
23
Whats happening differential sign has
changed What should the birds do stop responding
24
Moments later, birds are leaving
25
Operant conditioning--what makes it effective?

• Schedule of reinforcement
• Temporal contingency
• Belongingness
• Quality, quantity of reinforcer
• What else the animal might do

26
T-maze temporal contingency
27
Condition 1 immediate reward (.5 sec)
!
28
Condition 2 delayed reward (5 sec)
!
29
Effectiveness--temporal contingency

• The delay between the animals act that you are
  reinforcing, and the reinforcer.

30
WHY does learning drop off with delay??
31
Condition 2 delayed reward (5 sec)
!
32
Operant conditioning--what makes it effective?

• Schedule of reinforcement
• Temporal contingency
• Belongingness
• Quality, quantity of reinforcer
• What else the animal might do

33
Belongingness

• Thorndike tried to condition his cat to yawn or
  scratch to escape box--he proposed belongingness

34
Instinctive drift

• A concept related to belongingness instinctive
  drift (Breland Breland.)

35

• Motivational state can also influence a hungry
  animal does more food-seeking behaviors. . .

Digging, scratching, rearing
Digging
36
Quality/quantity of reinforcer

• Works as you would expect.

37
What else might the animal do?
Its not as simple as the animal Maximizes good
things, minimizesbad things.
Even humans dont do this, if the situation
getsmoderately complex.
38
Example
Variable interval
Variable ratio
Whats the optimal strategy?
39
Variable interval
Variable ratio
Optimal is to hit VR almost exclusivelyand
occasionally hit the VI. Instead, they respond
to equalize ratios of work/reward
40
The problem of definition
What is a reinforcer?
41
The problem of definition
Thorndike called a reinforcer something
thatbrings about a satisfying state of
affairs. How do we know when animal is
satisfied? Presumably, when the animal will work
toachieve this state of satisfaction.
42
But thats circular
Whats a reinforcer?
Something that increases behavior, that animal
will work to get
Whats pleasurable?
What will the animal work for (e.g., peck)?
Something pleasurable.
43
Another definition physiological homeostasis
Animal seeks to lessen thirst, hunger,
etc.Definition of reinforcement is based on
biological drives.
Learning a stamping in of the work that needs
to be done to reduce hunger.E.g, I must not
only consume and chew to get nourishment. I also
must press the bar, then consume, then chew.
44
Problems
Too many drives were proposed.
Animals (and people) do things that seem more
likely to raise drives, not lower them
45
Reinforcement as behavioral regulation

• Premack principle Given two responses arranged
  in an operant conditioning procedure, the more
  probable response will reinforce the less likely
  behavior.

46
Which do you want to do play pinball or eat
candy?
Must eat candy to play pinball
Must eat candy to play pinball
These kids treat candy eating as work do it to
get to play pinball.
These kids eat candy but dont care that they
have earned pinball time.
47
Behavioral homeostasis bliss pointa clever,
not-quite-right idea
Normally, animal likes to be at gray spot (15
minutes ofeach--now it cant be at gray spot.
What will it do?
48
IN THEORY you should be able to predict what
animal willdo--it will select spot on blue line
that is as close as possible to its bliss
point. IN REALITY this predictionsometimes
works, sometimes doesnt.
49
Reinforcement--final word

• In the end, we still dont have a good definition
  of the concept. Premack Principle is as close as
  we get. Nevertheless, the concept of
  reinforcement seems useful.

50
Applications

• Animal training
• Biofeedback
• Education
• Token Economies

51
Biofeedback
Operant conditioning of the autonomicnervous
system.
For years, not explored because no one thought
it could possibly work.
52
Apply operant conditioning principles to
education
1. Make sure student doesnt make mistakes
guidebehavior. 2. Review frequently.
53
(No Transcript)
54
Little enthusiasm. Teachers didnt like itfor
their own reasons. Students were bored.
55
Token economies
Used in some mental health institutions, and
some classrooms.
56
Mrs. Ahlersmeyers 3rd grade class, Lafayette
Elementary, Lafayette, IN

• Students earn a salary (marbles).
• Outstanding work or behavior earns bonuses.
• Students allowed 5 sick days per quarter, after
  that, they are docked pay.
• Students charged rent for their use of desk, and
  for any school property lost or damaged.
• Students docked pay for inappropriate behavior.

57
Use is controversial because it seems
dehumanizing (mental patients) or because it
seems that youre paying students for behavior
that they should want to do.
58
Applications

• Animal training
• Biofeedback
• Education
• Token Economies