Hows Things

1
Hows Things?

• Quantifying Outcomes in
• Behavioural Intervention
• Brian Mc Donald, R.N.I.D. P.Dip.B.C.Psych.
• Behaviour Support Specialist
• 23/5/07

2
Rationale

• National Council for the Professional Development
  of Nursing Midwifery Measurement of Nursing
  Midwifery Interventions Guidance Pack No
  Behavioural Measures Included!
• Interventions should focus on real, tangible,
  quantifiable, qualitative, applied results for
  the individual in the 1st instance and the
  service provider in the 2nd instance
• Focus should be on what has improved for the
  person
• This should be considered in terms of quality of
  life improvements that are significant to that
  individual

3
Outcome Measures

• OUTCOMES implies an end to intervention
• FUTURE DIRECTIONS (or similar) might be a
  better term
• Outcome Measure is a common management term that
  is a measure
• of the result of a system relative to the aim
• An outcome measure is used to measure the success
  of a system
• (e.g. an outcome measure could be percentage
  reduction in the
• frequency of aggressive incidents in residential
  service following a
• period of intervention)

4
Related Terms

• Output Measures
• A measure of activity (e.g. the number of hours
  individual case-work completed by
• the behavioural person! A focus on output
  measures does not inform success. Care
• should be taken to define good outcome measures
  to use in determining the success
• of interventions
• Process Measure
• Measures of the performance of a process. Provide
  real time feedback that can be
• acted on quickly, when a process first provides
  data that something is amiss
• Activity Measure
• A measure of the level of resources committed to
  a process
• Operational Definition
• Example
• Resources allocated to an individual to address
  severe behavioural difficulties which
• are operationally defined in a residential unit
  would represent an activity measure.
• An in-process measure could be adherence to the
  behavioural plan. An outcome

5
Operational Definitions

• Description of term (e.g. aggression) as applied
  to a specific situation
• When collecting data it is vital to define terms
  very clearly in order to
• assure all those collecting and analysing the
  data have the same
• understanding
• Therefore, operational definitions should be very
  precise and be written to
• avoid possible variation in interpretations
• More specifically an operational definition often
  specifically states how to
• measure the item being defined
• Example
• How would you count the number of aggressive
  incidents in a day?
• The number of physical contacts made, the number
  of threats, the number
• of missed attempts at hitting

6
Hows Things???

• Generic global statements made by staff teams
• Anecdotal subjective statements that often have
  no backing or evidence
• Should not be discounted as they can be
  indicative of the mood/morale of the team
• May indicate an environment conducive or
  detrimental to behaviour change and therefore a
  starting point for the work to be done

7
Statistics Reports

• Statistics, Means, Standard Deviations,
  Coefficients, etc. have their rightful place in
  clinical journals, research, etc. but not at an
  IPP, PCP meeting, Personal Outcome Reviews, Staff
  Meetings, etc.
• Clinical reports must be translated into
  information that is usable by service-users and
  their families and carers
• Scientific journal quality commentary on
  behavioural measurement only serves to distance
  those that are end-users of the service
• For example
• The FI 300-s schedule was then implemented, and
  hand mouthing increased again to near baseline
  levels (M 26). The reintroduction of time-out
  (FR-1) again produced a reduction in hand
  mouthing (M 4.4), and responding remained low
  as the time-out schedule was gradually thinned to
  the FI 300-s schedule. Across the 57 sessions of
  the thinning procedure, hand mouthing occurred in
  an average of 3.4 of the intervals
• (Lerman, Iwata, Shore DeLeon, 1997)

8
What do we need to know?

• Look at this from a number of possible
  perspectives
• Service-user
• Ask the individual or, alternatively, consider
  from their perspective HOW IS MY LIFE BETTER?
• Staff team
• Is the work environment safer?
• Service (Day/Res./Other)
• Is the service of a higher quality?
• Service provider
• Are we getting value for money 11 staffing,
  extra night staff, assistive equipment, etc.
• Clinician
• Were the interventions employed effective?

9
Formulation

• Important to constantly evaluate and re-evaluate
• Assessment
• Best approaches to employ
• Targets for intervention
• (staff team, environments, behaviours,
  individual, etc.)
• It is ethically questionable to address an
  individuals behaviour without 1st looking at the
  environment, programmatic and service provision
  elements

10
Evidence Based Practice

• Evidence based practice
• Most, hopefully all, settings engage in practice
  that has a sound, contemporary, valid
  scientific basis
• Practice based evidence
• The sharing of ideas, innovations and practice
  initiatives through presentations at seminars,
  journal submissions, poster presentations, etc.
  with a view to raising the quality of work of all

11
Why Quantify Outcomes?

• Quantification of effect
• Qualification of effort
• (service-users supports)

12
Who Should Quantify Outcomes?

• Simply put
• those that are implementing, or directing the
  implementation of, interventions
• Measurement needs to be integrated into working
  practices
• Be conscious of pressures already on frontline
  staff

13
Practical Considerations

• The goal of records is to obtain accurate
  useful information that is gathered in as
  efficient a manner as possible
• Design of records should be respectful of the
  target user
• Facilitate the user with tick-lists,
  individualised records, headings, accurate
  direction, training and supervision/support

14
Interventions

• Examples of commonly employed models
• Applied Behaviour Analysis
• Positive Behaviour Support
• Multi-element Behaviour Support
• Cognitive Behaviour Therapy

15
Applied Behaviour Analysis

• A discipline devoted to the understanding and
  improvement of human behaviour
• (Cooper, Heron Heward, 1987)
• Focuses on objectively defined, observable
  behaviours of social significance
• Seeks to improve the behaviour under study whilst
  demonstrating a reliable relationship between the
  procedures employed and the behavioural
  improvement
• Uses the methods of science
• Description
• Quantification
• Analysis

16
Positive Behaviour Support

• A bringing together ideas inherent in Applied
  Behaviour Analysis (a toolkit) and Social Role
  Valorisation (a values base)
• P.B.S. has a lifespan perspective that involves
  long-term planning for each person rather than
  short-term crisis management
• Behavioural challenges should be responded to in
  the least restrictive environment
• Individuals should be maintained in an
  environment that allows for maximum contact with
  the broader community - inclusion participation

17
Positive Behaviour Support

• A holistic approach to helping people to engage
  in adaptive socially desirable skills whilst
  overcoming patterns of destructive and
  stigmatising behaviours
• Interventions must be based on a full behaviour
  assessment of the individual and the
  physical/social environment in which they live
  and work
• Interventions must be multi-element open to
  review/evaluation on a regular ongoing basis
• Staff working with individuals with challenging
  behaviour must receive the necessary training
  support
• A range of service options should be available
  based on an assessment of the individuals
  support needs
• Intervention should be of a non-aversive/non-intru
  sive nature the least restrictive alternative
  in the least restrictive environment should be
  adhered to as a guiding principle
• Some challenges http//www.aase.edu.au/2004_Conf_
  papers/S.Vanderaa.pdf

18
Multi-element Behaviour Support

• Ecological/Environmental restructuring
• Direct treatments
• Reactive/management strategies
• Positive Programming
• Mediator interventions
• Both proactive and reactive elements to the
• Behaviour Support Plan
• www.callaninstitute.org

19
Multi-element Behaviour Support

• Multi-element Behaviour Support plans are
  non-aversive
• They involve changes to the environment to
  achieve a better fit with the
• needs characteristics of the person
• They teach the social communication skills
  necessary to overcome
• challenging behaviours
• Behaviour Support Plans also include strategies
  for generating rapid short-
• term change for behaviours that are difficult or
  potentially dangerous
• There are effective reactive strategies, which
  are used to manage crises
• without compromising the safety or the dignity of
  all concerned
• Finally, Behaviour Support Plans address the need
  to establish supervisory
• systems to sustain the achievements gained

20
Cognitive Behaviour Therapy??

• Willner, Jones, Tamsy Green (2002) conducted a
  study in which 14 individuals with intellectual
  disabilities were referred for anger management
  and randomly assigned to a treatment group
  (group-based anger treatment) or a waiting-list
  control group (no treatment)
• They found the approach was effective in
  decreasing anger
• Their discussion directs future researchers to
  investigate whether cognitive behavioural
  treatment is more effective than purely
  behavioural interventions

21
Cognitive Behaviour Therapy??

• Hagiliassis, Gulbenkoglu, DiMarco, Young Hudson
  (2005) describe the evaluation of a group program
  designed specifically to meet the anger
  management needs of a group of individuals with
  various levels of intellectual disability and/or
  complex communication needs
• They reported that clients from the intervention
  group had made significant improvements in
  self-reported anger levels compared with clients
  from a comparison wait-list group
• They also note an absence of measured
  improvements in quality of life
• Very few intervention plans actually teach
  people with learning difficulties socially
  acceptable ways of expressing anger or
  frustration and challenging behaviour may be the
  one way in which people in such circumstances can
  exert any control over the way in which they
  live (Blunden Allen, 1987).

22
Measuring Outcomes

• Baseline - start at the beginning
• Within treatment measures
• (process measures)
• Post-treatment is there such a thing in
  intellectual disabilities?
• (positive behaviour support ethos is that
  intervention is ongoing)

23
Basic Behavioural Measures

• Frequency how often it occurs
• Duration how long it occurs for
• Rate how many times it occurs over a period of
  time
• We may be looking to increase (behavioural
  deficits) or decrease (behavioural excesses)
  results based on these measures

24
Episodic Severity

• Defined
• A measure of the intensity or gravity of a
  behavioural incident
• Sample Objective Measures
• Rate
• Duration
• Scaled severity ratings of stress/harm/injury
• Costs of property repair or replacement/staff
• replacement, etc.

25
Episodic Severity

• Quantifiable reductions in behavioural measures
  do
• not always imply overall improvement in the
  situation
• 1 tantrum a week lasting 30 mins
  (injuries/property damage)
• (low rate/high E.S.)
• vs.
• 6 tantrums a week each lasting 5 minutes
  (screaming)
• (high rate/low E.S.)
• Episodic severity gives a global yet case
  specific picture
• This allows for increased criteria for success of
  approaches

26
Case Study
27
Instruments Tools

• Some examples
• Periodic Service Review
• HoNOS LD
• Outcome Rating Scale
• Self-reporting
• Basic behavioural measures
• Episodic severity
• Graphs

28
Periodic Service Review

• Managers negatives that contribute to poor
  outcomes
• Not spending enough time in the community
• Not providing enough systematic instruction to
  the individuals receiving services
• Not achieving satisfactory outcomes for
  individuals being served
• Not collecting data
• Not being consistent/accurate in data collection
• Not following agreed-upon procedures
• Not following the individualised
  service/education plan
• Not providing sufficient functional
  age-appropriate activities
• Not respecting the individuals being served
• Not following schedules
• Not interacting positively with individuals being
  served
• Not doing any thing at all

29
Periodic Service Review

• Reasons offered
• High staff turnover
• Poor staff motivation
• Poor wages
• Lack of staff training
• Lack of staff skills
• Lack of agreement in philosophy or effectiveness
  of strategies employed
• Too little time in the day
• Dislike of individuals being served
• Lazy staff
• Lack of intelligent staff
• Not enough staff or other resources

30
Periodic Service Review

• What is it?
• A management system that monitors directs
  improvement across a broad range of staff
  agency indicators
• Used to assess staff quality consistency
• Used to evaluate the implementation of
  behavioural recommendations
• Can be customised to be employed in residential,
  educational, day and other services across a
  number of populations
• http//www.epsr.com/(bb30li45utlzewzxr2gslt2q)/vis
  itor/home.aspx

31
Periodic Service Review

• Individual Performance Standards of Behaviour
  Specialist
• Develop behaviour assessment intervention plans
• Submit plans on a timely basis
• Write quarterly progress reports
• Write termination reports
• Submit termination reports on a timely basis
• Provide service as authorised
• Submit attendance sheets
• Attend weekly unit meetings
• Attend 11 meetings with supervisor
• Train staff and key social agents
• Maintain case record
• Etc.

32
HoNOS LD

• Health of the Nation Outcome Scales
• HoNOS are a well researched and tested selection
  of clinical outcome measures developed by the UK
  Royal College of Psychiatrists Research Unit
  (CRU)
• Measures health and social functioning across 12
  items that are individually graded for severity
  on a 5 point scale related to behaviour,
  impairment, symptoms and social functioning
• Primary aim is to measure change in an individual
  over 2 or more points in time as a measurement of
  outcome for therapeutic interventions
• Measures change in the levels of problems that a
  person has had and is not meant to be a
  comprehensive description of the individual
• Also measures no change or deterioration

33
HoNOS LD (Sample Qs)

• Please indicate if the following behaviour
  problems have been present over the past 4 weeks
  based on the following rating
• 0 No problem during the period rated
• 1 Mild Problem
• 2 Moderate Problem
• 3 Severe Problem
• 4 Very Severe Problem
• 9 Unknown
• Behavioural Problems (External Aggression Towards
  Others)
• No behavioural problems directed towards others
• Irritable, quarrelsome, occasional verbal abuse
• Frequent verbal abuse, verbal threats, occasional
  aggressive gestures, pushing or pestering
  (harassment)
• Risk, or occurrence of, physical aggression
  resulting in injury to others requiring simple
  first aid or close monitoring for
• prevention
• Risk, or occurrence of, physical aggression
  producing injury to others serious enough to need
  casualty treatment and requiring
• constant supervision or physical intervention for
  prevention (e.g. restraint, medication or
  removal)
• Behavioural Problems (Aggression Directed
  Inwards)

34
HoNOS LD

• Benefits
• Clinician friendly instrument that can be used in
  routine practice irrespective of professional
  background
• Global nature of the instrument lands itself to a
  holistic approach
• Limitations
• Nature of intellectual disabilities is that there
  is a greater reliance on informant-based data
  leading to greater opportunities for error
• Broad nature may be less well suited to detecting
  the subtle changes in those with chronic
  conditions

35
HoNOS LD

• Supply of the pack is available from-
• BILD Publications Book Source32 Sinlas
  StreetGlasgowG22 5DU
• Tel No 0141 5581366Email Derek_at_booksource.net

36
Outcome Rating Scale

• Looking back over the last month, mark the line
  to indicate how you think the person may be
  feeling about how
• well he or she has been doing in the following
  areas of his or her life, where marks to the left
  represent low
• levels and marks to the right indicate high
  levels
• Individually
• (Personal well-being)
• I-------------------------------------------------
  ----------------------------I
• Interpersonally
• (Family, close relationships)
• I-------------------------------------------------
  ----------------------------I
• Socially
• (Work, School, Friendships)
• I-------------------------------------------------
  ----------------------------I
• Overall
• (General sense of well-being)
• I-------------------------------------------------
  ----------------------------I

37
Self-reporting

• Self-report can be a reliable valid means by
  which to collect information from people with
  intellectual disabilities
• People with intellectual disabilities can deal
  with abstract concepts providing their knowledge
  understanding are assessed the therapist is
  prepared to take on a didactic role
• Self-regulation (therefore generalisation
  maintenance of therapeutic gain) can be achieved
  but only if the environment supports respects
  individual rights opportunities to practice
  self-determination are present
• (Kroese, Dagnan Loumidis, 1997)

38
Modified Anger Log

• Johns Anger Diary
• Date Time? Where?
• What Happened?
• Fighting Shouted At Arguing Made Fun Of
• Who?
• Friend Relative
  Stranger Staff
• 
  

39
Stress/Anger Thermometer
40
Note re Graphs

• When measures of behaviour are plotted on graphs
  immediately after an observational period ongoing
  access to a complete record of the behaviour is
  provided
• This allows for constant evaluation and
  re-formulation
• Ensures approaches and clinical decisions are
  responsive to performance
• Graphs are also a highly effective feedback
  mechanism to service-users, staff teams, families
  and management

41
Time-out Room Use
Exclusionary Timeout Reactive Strategies Only
Revised Excl. Timeout Comprehensive Beh.
Management Guidelines
D.R.O. Introduced
42
Further Research

• Involves small focus groups of parents and
  professionals with first-hand knowledge
• about how autistic behaviours impact on
  day-to-day functioning for the individual
• and caregivers
• The aim is to learn which behaviours pose
  difficulties or are a concern so that these
• can be appropriately captured in the measure
• Also focuses on gathering the list of behaviours
  generated and narrowing it to the
• most relevant
• The measure will then be tested to make sure that
  it is user friendly, captures
• behaviours accurately is statistically sound
• Future research will be done to assess how well
  it measures change in behaviour
• http//www.togetherforautism.ca/client/aso/TFA.nsf
  /object/Kagan-KushnirDevelopingNewBehaviouralM
  easure/file/Kagan-KushnirDevelopingNewBehaviou
  ralMeasure.pdf

43
References

• Blunden, R. Allen, D. (eds) (1987) Facing the
  Challenge An Ordinary Life for People with
  Learning Difficulties and Challenging Behaviour.
  London Kings Fund Project Paper, Number 74
• Cooper, J.O., Heron, T.E. Heward, W.L. (1987).
  Applied Behaviour Analysis. New Jersey Prentice
  Hall
• Hagiliassis, N., Gulbenkoglu, H., DiMarco, M.,
  Young, S. Hudson, A. (2005). The Anger
  Management Project A group intervention for
  anger in people with physical and multiple
  disabilities. Journal of Intellectual
  Developmental Disability, 30, 8696
• Kroese, B.S., Dagnan, D. Loumidis, K. (1997).
  Cognitive Behaviour Therapy for People with
  Learning Disabilities. London Routledge
• LaVigna, G.W., Willis, T.J., Shaull, J.F., Abedi,
  M. Sweitzer, M. (1994). The Periodic Service
  Review A Total Quality Assurance System for
  Human Services Education. Maryland Brookes
• Lerman, D.C., Iwata, B.A., Shore, B.A. DeLeon,
  I.G. (1997). Effects of Intermittent Punishment
  on Self-injurious Behaviour an Evaluation of
  Schedule Thinning. Journal of Applied Behaviour
  Analysis, 30, 187201
• Roy, A., Matthews, H., Clifford, P., Fowler, V.
  Martin, D.M. (2002). Health of the Nation Outcome
  Scales for People with Learning Disabilities
  (HoNOS-LD). British Journal of Psychiatry, 180,
  61-66
• Taylor, J.L. Novaco, R.W. (2005). Anger
  Treatment for People with Developmental
  Disabilities. New Jersey Wiley Sons Ltd.
• Willner, P., Jones, J., Tamsy, R. Green, G.
  (2002). A Randomised Controlled Trial of the
  Efficacy of a Cognitive-Behavioural Anger
  Management Group for Clients with Learning
  Disabilities. Journal of Applied Research in
  Intellectual Disabilities, 15, 224235

44
Contact

• brian.mcdonald_at_galwayca.ie
• behavioursupport_at_gmail.com