Assessment Centres and Simulations Reference Checking

1
Assessment Centres and SimulationsReference
Checking

• Selection Tools and Techniques
• Part 2

2
Outline

• Assessment Centres and Simulations
• Other Tools and Techniques
• Psychometric Testing
• Realistic Job Preview
• Reference Checking

3
Assessment Centres
4
The Recruitment Process
5
Accuracy of Selection Techniques
(Robertson Smith, 2001)
6
Frequency of the Use of Selection Methods in
Australia
(Dessler et al., 1999)
7
Work Samples

• Ask applicant to complete a job activity,
  behavioral or verbal, under structured testing
  conditions.
• Direct evidence of ability or skill.
• If well constructed are representative of job
  tasks, equipment, etc
• Not using verbal information to make inferences
  (so you minimize impact of faking good and poor
  verbal fluency).
• So why doesnt everyone use them if they are so
  good?
• Are limited in the validity if they are not based
  on good job analysis and if they are not
  representative of job tasks (same for ACs)
• Some do assume there is some knowledge, skills
  and abilities to perform the job

(Gatewood and Feild, 1998)
8
Simulations (AC)

• Exercises are designed so that participants have
  several opportunities to demonstrate each of the
  patterns of behaviour being evaluated
• Assessors are trained and share their evaluations
• A group of simulations, work samples, interviews
  and psych tests Assessment Centre

(Gatewood and Feild, 1998)
9
(Gatewood and Feild, 1998 p. 606)
10
(No Transcript)
11
Developing Performance Tests

• Perform a job analysis
• Identify the tasks that are to be tested
• Develop the testing procedures
• consider time required, difficulty, resources
  available, how you will score it, is the task
  representative of the job
• Score the test results
• set standards and rules
• Train judges

(Gatewood and Feild, 1998)
12
Types of abilities to be developed in Training
Assessors

• Understanding the behavioural dimensions
• Observing the behaviour of participants
• Categorising participant behaviour as to
  appropriate dimensions
• Judging the quality of participant behaviour
• Determining the rating of participants on each
  behavioural dimension across the exercises
• Determining the overall evaluation of
  participants across all behavioural dimensions
• selection decisions are most accurate when
  behavioural consistency is the major
  characteristic of the selection program
  (Wernimont and Campbell, 1968 cited in Gatewood
  Field, 1998 p. 589)

(Gatewood and Feild, 1998)
13

• Strengths
• demonstrated validity of these measures
• often are good at having low adverse impact
  (especially non-verbal performance tests like
  motor tests and ACs)
• face validity for applicants
• can serve as realistic job previews
• AC results (when kept secret) have shown
  predictive validity for career and managerial
  success (promotions job perf)
• managers used as assessors develop their
  competence aswell

• Weaknesses
• cost to develop and run ACs can be prohibitive
• some studies have found verbal and paper pencil
  intelligence tests have been just as predictive
  of managerial performance as ACs (Tziner Dolan,
  1982)
• construct validity has not been there for ACs
• Convergent correlations between e.g. interview,
  case study, intray are low .05-.15 are they not
  being assessed reliable or are each behaviors
  manifested differently in different exercises?
• Lack of Discriminant Validity
• High correlations between behaviors on the same
  exercise
• They appear valid at predicting job performance
  and career movement but we dont know why
• self-fulfilling prophesy? probably not

(Gatewood and Feild, 1998)
14
Video

• Two Whole Days

15
Questions?
16
Other Selection Tools and Techniques
17
Options are

• Application
• CV
• Application Form
• Biodata
• Psychological Testing
• Work samples
• Simulations (a group Assessment Centre)
• Interview
• Realistic Job Preview
• Reference Checking

Still to Cover
18
Psychological Testing

• Ability
• mental cognitive abilities of applicant
• physical muscular strength, cardiovascular
  endurance, movement quality
• mechanical ability
• clerical ability
• Personality
• refers to the unique organisation of
  characteristics that define the individual
  (includes thoughts, feelings and behaviours)
• identify job tasks and then identify traits that
  are linked to the tasks
• Measurement Methods
• Self report Questionnaires Neo PI-R, California
  Personality Inventory, MBTI
• Projective
• Interest Inventories test interest, still have
  to assess capability to actually do it

(Gatewood and Feild, 1998)
19
Examples of Cognitive Ability Tests

• Example of verbal test questions
• The relationship between WORD and SENTENCE is
  best expressed similarly by which of the
  following pairs of words
• engine and car
• bible and book
• cell and organism
• sonnet and text
• stem and flower

• Examples of numerical test questions
• What is the following approximately equal to
  5/93/45/7?

Examples of items in abstract and spatial
format which of the following completes the
series above? a b c
20
Examples of Mechanical Ability
(Dessler et al., 1999 p. 273)
21
Examples of Personality Test Items

• I tend to be reserved and keep my problems to
  myself.
• a. True
• b. ?
• c. False

• I believe I am self-disciplined
• Strongly agree
• Agree
• Neutral
• Disagree
• Strongly disagree

22

• Strengths
• interesting
• user-friendly
• relate back to in-depth theories of personality
  (content validity)
• predictive validity has improved since 1991 and
  the proliferation of Five Factor Models

• Weaknesses
• self-report - response distortion
• Ones et al., 1995 showed faking can increase
  scores by nearly 1/2 SD
• underestimate situational factors that can impact
  on behaviour
• criterion related validity can be less than other
  tools
• Barrick and Mount (1991) r0.24

(Gatewood and Feild, 1998)
23
Examples of Interest Inventories

• For each group of 3 activities indicate your
  choice for the most preferred activity and the
  least preferred
• Visit a museum of science
• Visit an advertising agency
• Visit a factory in which typewriters are made

• For the list of occupations,
• Mark (Y) if it interests or appeals
• Mark (N) if you dislike or find disinteresting
• Leave blank if you are undecided
• 1. Criminologist
• 2. Private Investigation
• 3. Restaurant Worker
• 4. Detective
• 5. Photoengraver
• 6. Truck Gardener

24

• Strengths
• interesting
• user-friendly
• can relate back to career
• Are different ones for different age groups (e.g.
  school leavers, employees)
• Useful for predicting membership of an occupation
  (so good for career guidance) not for predicting
  success within it

• Weaknesses
• Often used inappropriately especially in
  selection where there is little evidence
• Often not used in concert with ability/aptitude
  information so the career advice is limited

25
Criteria for a good test

• Clear instructions for administration, scoring
  and interpretation
• Economy time, cost, resources
• Technical criteria
• Reliability
• Validity (including generalisability)
• Norms and standardisation

26
Practical Value of Psych Tests

• Testee
• dishonest or uninformed answers to PQ
• anxiety
• practice effects for cognitive ability tests
• Informed consent
• Right to feedback
• Right to privacy and confidentiality

• Tester
• cost
• lack of norms to compare to
• culture bias - EEO
• lack of professionals to administer tests
• lack of professionals to interpret
• predictive validity

Issues to consider in practical setting
27
Sources of Information about Tests

• Test manuals
• Test catalogues
• Reference volumes
• Mental Measurements Yearbook
• Tests in Print
• http//www.unl.edu/buros/
• APA Guide
• Finding Information about Psychological Tests
  (1995)
• Journal Articles
• About Tests
• Trends in testing and assessment

• Online Databases
• PsychINFO
• Educational Testing Service
• http//www.ets.org
• Suppliers
• ACER (Australian Council for Educational
  Research)
• The Psychological Corporation
• SHL (Saville and Holdsworth Limited)
• Psychological Assessments Australia

28
Best-Practice guidelines to follow when you
choose and use psychometric tests

• Applications
• What is known of the role?
• Match test to job description
• Have tests been used successfully before?
• Norms - appropriate?
• Resources required
• Accessibility of results

• What other procedures will be used?
• Data protection
• When/How will the integration of results take
  place? By who?
• Who is responsible for feedback? What will be
  provided?

29
Realistic Job Preview

• Provision of positive and negative aspects of the
  job aim is to portray the job realistically
• RJP provides another method for self-selection
• job applicants remaining should have realistic
  expectations of job demands and characteristics
• Dugoni and Ilgen (1981)
• They lower expectations, increasing job
  satisfaction
• Enhance ability to cope with unpleasant job
  circumstances
• Perceive greater openness and honesty with a
  company that candidly reveals to them what their
  job is like
• Booklet, video, on-site visit

(Muchinsky, 1997)
30
Reference Checking

• How
• In person, Mail / Email, telephone, letters of
  reference
• Why use them
• Verify information provided
• To predict job success
• Uncover background information that may not have
  been provided earlier or identified by the
  selection techniques used
• Often serves more as a basis for negative
  selection
• Issues
• Duty of care to prepare accurate references
• Defamation and discrimination

(Gatewood and Feild, 1998)
31
Reference Checking - Checklist

• Obtain permission (preferably in writing) you can
  contact them, at least at end of interview
• Require applicants to give complete references
  (previous employers and personal if relevant)
• When contacting referees, ask questions dealing
  with job-related issues
• Do not ask for information from referees that
  cannot legally be asked on an application
• Train reference takers on how to solicit and
  record reference information
• Record and file responses (including refusals)
• Conduct criminal checks where relevant (child
  care worker, armaguard, etc)
• Conduct other checks are required (I.e. divers
  license)

(Gatewood and Feild, 1998)
32
Questions?
33
Exercise