Federico Sanabria

1
Beyond Efficiency A Model of DRL with Attention,
Activity, Timing, and Impulsivity Modules.

• Federico Sanabria
• Department of Psychology
• Arizona State University
• February 5th, 2007
• Winter Conference on Animal Learning and Behavior
• Winter Park, CO

2
Differential Reinforcement of Low Rates

• DRL schedule (Ferster Skinner, 1957)
• Target Interval
• Outputs
• Activity Response rate
• Efficiency Rate of reinforcement, responses per
  reinforcer
• (But see Richards, Sabol Seiden, 1993)
• Measure of impulsivity (Evenden, 1999) Response
  inhibition, behavioral regulation.

3
Recent Psychopharmacological studies using DRL

• Antidepressants vs. Anxiolitics (ODonnell et
  al., 2005)
• Amphetamines (Bardo et al., 2006)
• Cocaine (Cheng et al., 2006)
• Polychlorinated biphenyl (PCB Sable et al.,
  2006 Stewart et al., 2006)
• Methylmercury (MeHg Paletz et al. 2005 Stewart
  et al., 2006)

4
Differential Reinforcement of Low Rates (DRL)
DRL 5 s
5
Differential Reinforcement of Low Rates (DRL)
DRL 5 s
6
Differential Reinforcement of Low Rates (DRL)
DRL 5 s
7
Differential Reinforcement of Low Rates (DRL)
Model
Timing
p Attention ? Impulsivity ?/? Timing ?
(Non-timing) Activity rate
8
Fits to Binned Frequencies

• Cannabinoid Receptor Agonist (AM411)

Veh AM411 p .78 .93 ? 12.3 s 8.7 s ?/?
0.36 0.46 ? 2.43 s 0.82 s
9
Fits to Binned Frequencies

• Reward size (water)

30?l 300?l p .79 .67 ? 56.7 s 42.3 s ?/?
0.37 0.40 ? 2.49 s 3.41 s
10
Some Problems

• Individual IRT distributions are rarely reported.
  
• Response rates and efficiency ratios (which may
  be derived from model parameters) are the usual
  DV.
• Even when reported, IRTs are usually binned.
• Model should be fitted to cumulative plots of
  individual subjects.

11
Fits to

• Rodent model of ADHD

12
In Synthesis

• Current measures of DRL performance confound
  informative behavioral processes.
• The proposed model provides meaningful measures
  (attention, impulsivity, timing, activity).
• It is based on a simple hypothetical mechanism.
• It is modular.
• It provides good descriptions of data.
• Future directions
• Analysis of performance by ADHD patients and SHR
  rats
• Real-time analysis of mechanism operation
• Orthogonal control of modules

13
Acknowledgments
Peter Killeen
Kweku Osafo-Acquaah
Hyewon Rhieu
Cindy Bazua
Espen Johansen
Eric Thrailkill
Weihua Chen
Lewis Bizo
Kayla Cranston
Ricardo Pellón
NIMH 1R01MH066860 NSF IBN 023682.
Paul Jellison
14
Thank You
15
DRL Timing Components
M
T
R
16
DRL Timing Components
M
T
R
17
DRL Attention Component
M
T
R
18
DRL Motor Component
M
T
R
19
DRL 60 s
20
DRL Performance - Timing
21
DRL Performance - Timing
1-p
p
Timing
Motor
Response
?,?
?
22
DRL Performance - Attention
1-p
p
Timing
Motor
Response
?,?
?
23
DRL Performance - Activity
1-p
p
Timing
Motor
Response
?,?
?