A Web-Based Teaching Tool for Psychotherapy Research and Training

1
A Web-Based Teaching Tool for Psychotherapy
Research and Training Developed by Jakob Valen
and Leigh McCullough
Making Psychotherapy Better Step By Step
2
There have been daunting challenges in
Psychotherapy Process Research. These include
the following.

• Archives of thousands of VIDEOTAPES from
  Randomized Controlled Clinical Trials of
  Psychotherapy.
• Development of many PROCESS MEASURES to evaluate
  therapy
• Training of hundreds of RESEARCH RATERS

3
CHALLENGES IN TRAINING PROCESS RESEARCHERS

• 2,000 VIDEOTAPED SESSIONS PER TRIAL
• 2 HOURS PER SESSION TO EVALUATE
• 4,000 HOURS PER TRIAL or
• 10 YEARS PER PERSON
• To rate one clinical trial
• With just one instrument

4
But we need to rate each session with a number
of well-established PROCESS MEASURES from a wide
variety of domains of psychotherapy

• For example
• Patient-Therapist Interaction
• (Alliance, Attunement, Ruptures)
• Therapist Interventions (Clarification,
  Support, Cognitive Strategies)
• Patient Responses
• (Maladaptive cognitions, Affect, Defense)

5
In short, it would take a small army of
research raters a few decades to complete all
the ratings that are needed
6
Meanwhile. The Traditional Method of Training is
too labor Intensive and time consuming. Ongoing
small group teaching by the originator of the
instrument can take months and yield only a few
reliable raters.
See Note Below
7
So what are we to do?
8
Leigh McCullough, a Visiting Professor in
Trondheim, Norway, was explaining this problem
when a doctoral student named Jakob Valen had an
idea..
9
There is an archive being built in Norway The
Trondheim Psychotherapy Research Program
Videotape Archives Library (Tapes from at least 3
Clinical Trials will be studied)
STDP Psychodynamic
CBT Cognitive
DBT Behavioral
10
There is also a large number of graduate
students in clinical psychology at the University
in Trondheim, Norway (NTNU)
11
Why not Build a Web-Based System for Process
Research
Slide 38
so that many raters could be trained to evaluate
the thousands of videotapes that have been
collected from the many clinical trials of
psychotherapy?
12
So Jakob Valen began to build the
program Jakobsladder.com And Leigh McCullough
helped him develop it
13
Jakobsladder Internet Training

• Students could go to their computer,
• log on to jakobsladder.com
• and select an instrument they would be
  interested in learning.

Now, so can you.
14
The Introductory Page of Jakobsladder
15
FIRST, REVIEW THE LIST OF INSTRUMENTS THAT ARE
AVAILABLE Each one assesses an important aspect
of psychotherapy
See Note Below
16
THEN SELECT AN INSTRUMENT TO EVALUATE THERAPY
PROCESS
For example, select the DMRS
AAS - THE AFFECT ATTUNEMENT SCALE ATOS
ACHIEVEMENT OF THERAPEUTIC
OBJECTIVES SCALE (COMMON FACTORS) DMRS DEFENSE
MECHANISM RATING SCALE
17
When you select the DMRS ( The Defense
Mechanisms Rating Scale), you go to this page.
18
The menu to the left allows you to select the
DMRS Manual or to select segments to be rated
See Note Below
19
When you click on one of the Ratings
buttons you will receive transcripts of sessions
- as you see above.
See Note Below
20
EXAMPLE FROM THE TEXT See if you can find a
defense mechanism in this selection.
21
As you read the transcript and note a
defense mechansim, click on the menu to the
right, where you see Defence Mechanism 1.
22
The Defense Drop-Down Menu guides the process

• When you select the Defense Mechanism button
• You will be asked to do three things
• Select a defense from the drop down list
• Briefly state your reason for the selection
• Cut and paste the example from the text that
• illustrates the defensive behavior

23
See Note Below
24
The Defense drop- down menu will give you a list
of Defenses to choose from
Use the cursor to select a defense, in this case
Reaction Formation
25
The selected defense will appear in the defense
box, and the rater will be able to type in the
reason for the selection
26
The rater proceeds through the transcripts
until all defenses have been identified. This
process provides important information on the
psychological functioning of the patient for both
research and clinical training.
27
Lets take another exampleMaybe you want to
learn about common factors in psychotherapy such
as insight, motivation, or experiencing of
feeling

• Go to the
• Instrument List
• and select
• the ATOS Scale

28
This is the Home Page for the ATOS Scale
See Note Below
29
Use the Menu to select the ATOS Manual or
Practice Ratings
See Note Below
30
With the ATOS Scale. You can choose to rate a
Transcript Or you can rate from
Videotapes. First well show an example of a
Transcript.
31
This is what the transcript page looks like.
32
Close up of the ATOS Transcript Rating Page
See Note Below
33
Or you can PRACTICE RATING A VIDEOTAPED SEGMENT
34
It is much more enjoyable and educational to
watch and discuss a video with another rater
See Note Below
35
This is the page that is shown when videotapes
are rated.
See Note Below
36
See note below
37
Some examples of data the ATOS rater enters
38
The rater then can compare his or her rating to
the Gold Standard ratingsas well as see an
explanation of why the group of experts gave the
rating they did.

60
45
15
See note below
39
60
40
Another example of feedback provided by the
ATOS program.
41
When the rater has completed rating approximately
20 segments on a specific instrument, the rater
may receive a grade or a proficiency rating in
the form of a RELIABILITY STATISTIC
RESULTS OF RATINGS
.43
See Note Below
42
JakobsLadder ICC Rating
GRADING SYSTEM Above .7 GOOD .4 -.7
FAIR Below .4 POOR
43
Sometimes the rater will receive this message
Youre not quite reliable, but keep practicing
and youll improve!
44
So , JUST KEEP PRACTICING
45
AND PRACTICING
46
And before too long, youll be able to say,
IM RELIABLE!
.75
47
When the rater becomes reliable on the rating
instrument, a Reliability Certificate can be
printed out
BENTE JOHANSEN has met criteria for Reliability
on the DMRS Scale At .75
The ICC reliability rating can then be given to
the Principal Investigator of a study to verify
that the rater is able to begin research ratings.
Also, a reliability score also informs
clinicians-in-training that they satisfactorily
understand the clinical concepts that are being
evaluated such as the types of defense
mechanisms a patient uses or the degree of affect
experienced in the session.
48
Once research ratings have been entered, they are
stored on an Excel file to be easily accessed and
analyzed by the Investigators in the study.
EXCEL FILE
49
Also available is a program for a Videotape Grid
for each study. Like a library card catalog,
each session is identified by a box and the signs
in the box indicate whether it is available,
checked out, rated by one or more process
instruments and by how many raters. See note
below for more info.
See Note Below
50

• Thus, JakobsLadder
• is not only useful for
• TRAINING PROCESS RESEARCHERS
• But is also an excellent tool for
• TEACHING PSYCHOTHERAPY

51
AIRLINE PILOT STANDARDS Just as
pilots have to log hundreds of hours in the test
cockpit before being given the responsibility of
flying a plane and transporting live passengers,
we plan to develop simulated therapy sessions
that will adequately prepare therapist trainees
before they are given the responsibility of
working with suffering patients. Someday we
hope there will be archives of videotapes
available for therapist trainees to review and
evaluate hundreds of hours of pre-selected
segments of therapy demonstrated by senior
therapists, so that therapists-in-training will
have ample exposure to how therapy might
optimally be conducted.
52
How does Freud fit into all this?
53
Freud envisioned a Scientific Psychology
54
?
What might Freud think of JakobsLadder?
55
A web based tool for teaching psychotherapy

• JAKOBSLADDER
• Developed by
• Jakob Valen and Leigh McCullough

Making Psychotherapy Better Step By Step
56
Hmm...
The End
57
OTHER INFORMATION ABOUT HOW TO LEARN MORE
ABOUT THE ATOS RATING SYSTEM
58
www.affectphobia.com To obtain ATOS
manual for coding psychodynamic processes
59
A treatment manual that describes the
interventions on which the ATOS Scale is based
60
Great Resources for Psychotherapy Process
Research! But how do we train all these
graduate studentsto rate thousands of tapes?