Towards measuring learning experiences as outcomes of psychosocial Intervention

1
Towards measuring learning experiences as
outcomes of psychosocial Intervention

• Willem Melief

2
Why the development of the learning concept?

• A The intention to improve the scientific base of
  the outcome measurement instrument Sater
• B The consistent finding of learning experiences
  as outcomes in the application of Sater

3
What is Sater ?

• A standardized system to measure Benefits of
  Social Work Intervention Experienced by Clients
• Developed by Verwey-Jonker Institute
• Willem Melief
• Meta Flikweert
• Norbert Broenink

4
Developed on Requests of stakeholders

• Regular requests for research about
• outcomes of social work
• client experiences
• client satisfaction
• By stakeholders
• Agencies for Social Work
• Representatives of clients
• Financiers and government

5
Characteristics of SATER

• The use of client problem definitions
• Client experiences linked to problem categories
  and intervention aspects
• Clients enter answers in computer questionnaire
  for optimal privacy
• Standard table and graph output produced by a few
  standard SPSS command files
• Standard report format

6
Measurement of Client Experiences

• Two elements
• 1 Experienced benefits in relation to the
  experienced problems
• Benefits ? results, outcomes
• 2 Satisfaction about personal treatment and
  treatment process
• .

7
Sater Data

• Clients of 6 agencies
• 5 in a pilot 1 in a further application
• Database of appr. 1275 client interviews
• 6 agency reports
• 
  

8
Benefit categories

• Problems partly or totally solved
• Prob. Sol.
• Learned to accept and live with unchangeable
  conditions
• Acceptance
• Learned to handle problems
• Handling
• Experienced support and understanding
• Sup/underst

9
Experienced Benefits in N 250 (Gron) 1025
(Pilot)
10
Change oriented benefit categories

• Problem solving
• Learning
• Learning to handle problems including solving
  them
• Learning to accept and live with Unchangeable
  reality

• 33
• 50
• 20

11
Arguments in favour of the further development of
learning

• The frequent occurrence in benefits reported
  by clients
• Connection to social work methodology
• Perspective of lasting, repeatable effects of
  intervention
• Connections to neighbouring fields
• psychotherapy
• psycho-education

12
Questions to be answered about learning effects

• 1 What are the learning effects that correspond
  to the sort of learning reported by the clients
  in our research?
• 2 How can we measure these learning effects? .

13
Kinds of learning

• Remembering historical dates
• Remembering the periodic table of chemical
  elements
• Acquiring skill in mental arithmetic
• Acquiring skill in writing correct language
• Mastering the art piano playing

• Becoming a accomplished football player
• Acquiring skills in controlling ones violent
  mental impulses
• Finding out how to relate to other people
• Acquiring skills in solving (specific) problems
• Acquiring skills in staying out of (specific
  kinds of) trouble

14
Psychosocial learning
Knowledge elements
Behavioral skills
Repertoire of Learned Elements At disposition of
person
Attitudinal elements
Thought procedures
Selection from repertoire of elements matching
the Psychosocial situation
Adequate handling of the Psychosocial situation
15
Causes for inadequate handling of the situation

• Missing elements in the repertoire
• Shortcomings of elements in the repertoire
• Lack of information
• Insufficient skills
• Insufficient training in practice
• Shortcomings in the ability to assess the
  situation,
• Shortcomings in the control and co-ordination of
  the use of elements from the repertoire.
• Factors blocking adequate use of elements
• Overwhelming emotions (fears, anger, depression,
  etc)
• Serious psychopathology
• Serious shortcomings in elementary conditions
• Serious uncontrollable behaviour of other people

16
Main helping strategies

• Removing the blockades first
• Shortcomings in elementary conditions
• Overwhelming emotions
• etc
• to free the road for
• Teaching clients to accomplish lasting results
• necessary information
• skills
• ability to assess the situation
• ability to apply the right element from his
  repertoire

17
Literature Search

• Social Work Literature
• Very little direct or systematic treatment of
  learning effects
• Description of intervention outcomes that can be
  interpreted as learning
• for example behaviour modification directed
  treatment
• A Dutch approach of learning as model for
  emanicipatory social work and empowerment
• Social Science Literature
• The work of the Dutch psychologist and scholar
  A.D. De Groot,
• who intoduced the concept of fundamental
  learning experiences.

18
Traditional measurement of learning effects

• In traditional educational research
• Learning effects behavioural effects
• Acquired knowledge
• Skills
• Attitudes
• This approach is very valuable !!, but

19
Coverage problem

• but is not enough
• The limitation to only measuring behavioural
  effects causes a
• Coverage problem
• subjective fundamental learning experiences are
  not measured (covered)

20
Fundamental Learning Experiences

• Conscious experience of person that
• He has learned what behaviours (skills,
  attitudes) he has at his disposition ready to
  be used
• He has learned when certain behaviour is
  adequate and when not
• He has learned to flexibly use available
  behaviours adapted to new situations

21
Fundamental Learning Experiences

• Have as much to do with things (knowledge
  behaviour) that are always as they are and
  reactions that always have to be the same
• as with
• Exceptions, things that are (sometimes) different
  and reactions that sometimes have to be
  different.

22
(No Transcript)
23
Relevance for social work

• Terminology and content are very similar to our
  own rudimentary way of thinking.
• Problems of clients relate to shortages in
  conscious application of behavioural elements due
  to
• lack of insight in what one has at ones
  disposition
• lack of skill to assess the situation and select
  the right reaction to it.

24
Preliminary conclusions

• Measuring learning experiences by clients are
  important
• Provides core insights that are missing in
  current outcome research
• This information cannot be measured in another
  way
• There is a promising connection to theoretical
  thinking
• Antagonism in the research and professional world
  is a serious obstacle

25
Opposing positions

• Pro objective measurement
• -What is not objectively measurable does not
  count (is irrelevant)
• -Subjective experiences of clients are not to be
  trusted (have no validity)
• versus
• Against objective measurement
• -reduces the uniqueness of the client
• -subjective data are too important to be
  discarded
• -mostly strong preference for qualitative
  research

26
Supplementary Model

• Quantitative outcome measurement using objective
  measurement OK!!
• Base of evidence base methods
• But should be supplemented by
• Quantitative measurement of subjective
  experiences, which also OK
• Learning experiences
• Satisfaction
• Qualitative research certainly also OK
• , very usefull
• Social Work Literature
• Very little direct or systematic treatment of
  learning effects
• Description of intervention outcomes that can be
  interpreted as learning
• for example behaviour modification directed
  treatment
• A Dutch approach of learning as model for
  emanicipatory social work and empowerment
• Social Science Literature

27
The future

• Expand the theoretical base for measuring
  learning experiences
• On preference including contacts with scholars I
  missed in other countries.
• Convert the very crude part about learning
  experiences in the Sater system to a more
  sophisticated measuring tool
• To correlate experience outcomes with outcomes
  of objective measurement
• , very usefull
• Social Work Literature
• Very little direct or systematic treatment of
  learning effects
• Description of intervention outcomes that can be
  interpreted as learning
• for example behaviour modification directed
  treatment
• A Dutch approach of learning as model for
  emanicipatory social work and empowerment
• Social Science Literature
• The work of the Dutch psychologist and scholar
  A.D. De Groot,
• who intoduced the concept of fundamental
  learning experiences.