Traditional Versus Technology Enhanced Assessment Centers: Pros and Cons

1
TraditionalVersus Technology EnhancedAssessment
CentersPros and Cons
Just Because We Can, Should We?

• Presented at the 20th Annual Conference of the
  Society for Industrial and Organizational
  Psychology
• Los Angeles, CA
• April 15, 2005

May not be quoted without permission of at least
one presenter.
2
Panel
Service Provider
ClientEnd-User
Program Designer
Assessor
Chair
Scott Eggebeen, Ph.D., I/O Director New York
University
Diane P. Brennan, Senior Leadership
Client Specialist
Sandra B. Hartog, Ph.D.,PresidentSandra
Hartog Associates
Lynn G. Collins, Ph.D., Executive Vice
President Sandra Hartog Associates
Janis M. Ward, Ph.D., J.M. Ward Consulting
3
Moving From Traditional to Technology-Enhanced
Assessment Centers
Service Provider
ClientEnd-User
Program Designer
Assessor
Strategy
Manage-ability
Design
Evaluation
4
Assessment Center Defined
The 10 Essential Elements of anAssessment Center

• Assessors
• Assessor Training
• Recording Behavior
• Reports
• Data Integration

• Job Analysis
• Behavioral Classification
• Assessment Techniques
• Multiple Assessments
• Simulations

International Task Force on Assessment Center
Guidelines, 2000
5
Assessment Center Continuum
The challenge is to determine the assessment
center design that has the greatest impact for a
given purpose/goal.

• Multiple candidates simultaneously assessed
• Typical duration of1-3 days
• In a designated assessment facility
• Typical assessee-to-assessor ratio of 21

• Utilizes technology to augment the exercises and
  overall simulation
• Shorter duration
• Assessor contact may or may not be face-to-face
• Data are distributed and gathered via technology

• There is no face-to-face interaction all
  activity occurs via phone, internet or through
  other technology

6
Strategy
Service Provider
ClientEnd-User
Program Designer
Assessor
Strategy
Manage-ability
Design
Evaluation
Traditional
Technology-Enhanced
Effective for tech-enhanced roles, technology
forward organizations, leadership roles involving
virtual teams, and global organizations,
simulation effective for lower levelsin
organization
Typically used for leader-ship and management
roles or roles requiring group interactions, can
communicate additional value
Culture
Relies on paper, warehousing of data and
equipment, requires multiple days of dedicated
assessor admin support
Requires technology resources at host and
consultant facilities, instant archiving and
retrieval, completely portable, can be linked to
HRIS system, easier to resource
Resources
Black box of tech development, Complex
implementation requires internal and external
coordination
Faster and known implementation
Build-Out
7
Manageability
Service Provider
ClientEnd-User
Program Designer
Assessor
Strategy
Manage-ability
Design
Evaluation
Traditional
Technology-Enhanced
Opportunity of increasing and danger of reducing
client involvement, reduced social interaction,
opportunity to serve a broader audience,process
often a better representation of org culture
Increased stakeholder involvement and viewpoints,
opportunities for social interaction,
acculturation, and inclusion of other events
Involvement
Greater time commitment for all stakeholdersand
participants (e.g., duration of center, travel)
Shorter duration, just in time delivery, quicker
reportturnaround
Time
Conference-like facilities and related TE,
consult-ing costs tend to be higher
Expensive to implement custo- mized technology,
relatively lower ongoing delivery costs
Cost
8
Design
Service Provider
ClientEnd-User
Program Designer
Assessor
Strategy
Manage-ability
Design
Evaluation
Traditional
Technology-Enhanced
Translate the relevantdimensions and role
components to a realistic simulation that can be
hosted live
Balance the capture of added dimensions with the
loss of others, develop realistic technology that
is fair toall candidates
Job Sample Construction
Administratively heavy prior to assessment,
requires on-site coverage, and dedicated staff
trained in event planning
Different skill set required, dedicated
technology support during assessment
Admin Support
Training in the company, job, dimensions,
simulations, observation, evaluation and feedback
Addition of training on technology
Assessor Training
9
Evaluation
Service Provider
ClientEnd-User
Program Designer
Assessor
Strategy
Manage-ability
Design
Evaluation
Traditional
Technology-Enhanced
Face-to-face observation, depth of integration,
visual cues, cognitive demands in simultaneous
assessments, physically demanding
Harder to get a sense of candidate, no
non-verbals, less extraneous info., rely more on
own assessment, integrate data from virtual
sources
Data Collection Assessment
Limitations due to virtual observations, more
difficultto build rapport, streamlined report
writing
Greater rapport, specificity and detailed
feedback, may have more credibility, leverages
multiple views
Feedback Outcomes
Additional skills related to technology,
rapport, reduced reliance on visuals, less, yet
more, targeted hard data
Solid observational skills, well trained in
assessment
Assessor Qualification
10
Panel
Service Provider
ClientEnd-User
Program Designer
Assessor
Chair
Scott Eggebeen, Ph.D., I/O Director New York
University
Diane P. Brennan, Senior Leadership
Client Specialist
Sandra B. Hartog, Ph.D.,PresidentSandra
Hartog Associates
Lynn G. Collins, Ph.D., Executive Vice
President Sandra Hartog Associates
Janis M. Ward, Ph.D., J.M. Ward Consulting
11
Tradeoffs
Service Provider
Technology-Enhanced
Traditional

• Dependence on Technologist
• Competitive Advantage
• Scalability

• I/O Practice and Comfort
• High End Intervention
• Control over Resources

12
Tradeoffs
Client End-User
Technology-Enhanced
Traditional

• Cost
• Stakeholder Involvement
• Resource Management

• Quality
• Time
• Technology Management

13
Tradeoffs
Program Designer
Technology-Enhanced
Traditional

• Comprehensiveness
• Group Interaction
• Administrative Demands

• Time in Assessment
• Portability/Practicality
• Technology Demands

14
Tradeoffs
Assessor
Technology-Enhanced
Traditional

• Face-to-Face Interactions
• Rich Integration Process
• Event Participation

• Streamlined, Targeted Performance Samples
• Greater Reliance on Own Evaluation Skills
• Volume

15
www.sandrahartogassoc.com/siop2005