Problem Intervention Investigation, Formulation, Design, Implementation

1
Problem InterventionInvestigation, Formulation,
Design, Implementation Evaluation
2
Overview

• Approaches to organisational problems scientist
  vs (?) practitioner
• Deciding appropriate interventions - collecting
  data about the problem
• Intervention designs
• Evaluation

3
Dealing with Organisational Problems The
Role(s) of the Research Practitioner
4
Scientist vs Practitioner Approaches to dealing
with organisational problems
Case study example an organisation wants to
design a new structured interview for its
graduate training scheme, which will include some
measure of their leadership potential. Difference
s between the scientist and the practitioner
in how they tackle this ? Occupational
psychologists as Scientist-Practitioners
5
Defining the problem - methods of investigation

• Prerequisites
• Appropriate to research objective
• Elicit an appropriate form of data
• Feasible (time, resource, organisational
  constraints)
• Piloted
• Ethical
• Agreed and accepted by the organisation
• Used appropriately
• Are experienced and confident with

6
Methods of Investigation (see also M46 M21
notes for Data Collection Methods employed in
organisations)

• Interviews
• amount of structure
• specific types
• Q-Sort (Stephenson, 1936)
• Rep Grid (Kelly, 1955)
• Critical Incident Technique (Flanagan, 1954)
• Focus groups
• Tips for successful interviewing
• Advantages ?
• Disadvantages ?

7

• Psychometrics
• Measures of a psychological construct which have
  been designed to have psychometric rigor
  (validity and reliability) for that context, and
  responses usually expressed as standardised
  scores.
• Examples
• Team Working Team Climate Inventory
• Employee Stress Occupational Stress Indicator
  (OSI), Spielberger State-Trait Anxiety Inventory
  (STAI),
• Workplace Reaction Job Involvement Scale,
  Organisational Commitment Questionnaire, Job
  Satisfaction Index
• Cognitive Ability Graduate Managerial
  Assessments (GMA) Watson-Glaser Critical
  Thinking, Ravens progressive matrices
• Personality 16PFi, NEO-PIR, OPQ
• Interest Vocational Preference Inventory (VPI),
  Strong Vocational Interest Blank.
• .. Other common techniques observation,
  questionnaires, diaries.

8
Interventions

• an ongoing system of relationships for the
  purpose of helping (Argyris, 1970)
• a planned programmatic activities pursued by
  both clients and consultants in the pursuit of
  organisational development (French, Bell
  Zawacki, 1994)
• Blake Mouton (1994) 9 categories of OD
  interventions (e.g. theory intervention, cultural
  intervention, procedural intervention,
  experimental intervention)
• French et al (1995) differentiate according to
  focus of interventions (individual, dyads, teams,
  intergroup, organisations)

9
Models of Intervention

• 1. The Intervention Model (Argyris, 1971)
• Assumptions organisations cannot diagnose /or
  solve own problems
• Activities of consultant data collection gtgtgt
  develop options
• 2. The Planned Change Model (Lippitt, Watson
  Westley, 1958 Schein, 1972).
• Assumption organisation will detect problem
• Activities of consultant diagnostic activity
  (as contracted), specific interventions and
  evaluation.

10

• 3. The Action Research Model
• Assumption the diagnostic process is integral
  to the intervention.
• Activities of consultant iterative process of
  intervention-evaluation
• 4. Purchase of Expertise Model
• Buyer (organisation) defines need, decides
  organisation does not have time/resources
• 5. Doctor-Patient Model
• Consultant gives organisation health check
  treats any symptoms of ill health
• 6. Process Consultation Model (Schein, 1969
  1988)
• Consultant tries to give client org insight into
  process events collaboratively decide
  intervention

11
Interventions

• Organisation Employee
• Focused Focused

12
Intervention Evaluation

• systematic collection of descriptive and
  judgmental information necessary to make
  effective training decisions related to the
  selection, adoption, value and modification of
  various instructional objectives (Goldstein,
  1991)
• 1. Why is evaluation necessary?
• 2. Who should do the evaluation
• 3. What aspects should be evaluated, and when
  should this be done?
• 4. What kinds of measurements (approaches and
  methods) need to be used?

13
1. Why is evaluation necessary?

• Patrick (1992)
• Training programme improvement
• Feedback to- and decisions about trainees
• Administrative and organisational decisions
• Validation of the training intervention
• N.B. Summative vs Formative evaluation (Scriven,
  1967).
• 2. Who should carry out the evaluation?
• Person running the training course?
• Trainees?
• Line manager? N.B. Issues of bias, sensitivity
• HR/Personnel? of feedback, accuracy,
• Others? sufficient knowledge, etc...

14
3. What aspects should be evaluated, and when?

• Whitelaw (1972) Hamblin (1974) Kirkpatrick
  (1967)
• Level 1 reactions of trainees to content and
  methods
• Level 2 learning outcomes attained
• Level 3 job behaviour in the workplace
• Level 4 Effect on trainees department /
  local org effects
• Level 5 Ultimate effect on organisation
• N.B. Criterion development
• criterion referenced vs norm-referenced.
• Criteria relevance (Goldstein, 1991)
• criteria deficiency
• criteria contamination
• Phillips (1990). Evaluation usually limited to
  trainee reactions

15
4. Evaluation Methods (Approaches Techniques)

• A. Trainee Reactions
• Questionnaire/Post-course survey of(what
  dimensions)?
• B. Learning Outcomes Attained
• Post-training test
• Measure of retention of training
• C. Job Behaviour
• Techniques to measure on-site performance ?
  Objective subjective performance criteria

16
4. Evaluation Methods (Approaches Techniques) II

• D. Department / Local Organisational Effects
• Quality/Quantity of production
• Safety N.B. Issues?
• Turnover (in team)
• Team climate/attitudes towards work
• E. Ultimate Organisational Effects
• Usually financial effect (utility analysis), but
  is maximising profits always an organisations
  main objective?

17
N.B. Formative Evaluation

• Why focus on the processes of the intervention?
• Explains why certain results were obtained
• Helps develop an explanatory basis for the
  development of an intervention
• Determine the dynamics/subtleties of the
  intervention

18
Evaluation Designs

• One group, post-measure only
• X T2
• One group, pre- and post measures
• T1 X T2
• Two groups, pre- and post measures, experimental
  control
• Re T1 X T2
• Rc T1 T2
• Two groups, pre- and post measures,
  non-equivalent control
• Re T1 X T2
• Ro T1 T2

19
Threats to accurate evaluation

• Internal validity
• the extent to which training has brought about
  new learning
• vs
• External validity
• the extent to which training will generalise to
  subsequent groups of trainees in same/different
  organisations

20
Conclusions

• Organisations are in a constant state of flux
• Employees have needs that are dictated by
  personal growth, career development, work-life
  interactions, etc.
• Adequate analysis of employee needs provides the
  basis for interventions
• Interventions need to be evaluated