Applying Science Towards Understanding Behavior in Organizations

1
Applying Science Towards Understanding Behavior
in Organizations

• Chapters 2 3

2
Research Issues in Organizations

• Approaches to collecting data
• Experimental
• Observational/correlational
• Data collection issues
• Sampling
• How should we select participants?
• What impact does it have on the results?
• Experimental design
• Controlling potential confounds
• Assigning participants to experimental conditions
• Measurement issues
• Describing and interpreting the results

3
Experiments A Review

• Experiments - Do changes in one variable (X)
  cause changes in another variable (Y)?
• Independent Variable (X)
• condition or event that is manipulated by
  experimenter
• Dependent Variable (Y)
• variable that is affected (hopefully) by
  manipulating independent variable
• Extraneous Variable(s)
• any variable other than independent variable that
  may influence dependent variable

4
Experiments Pros and Cons

• Advantage
• Allows conclusions about direct effects of one
  variable on another
• Disadvantages
• Experimental conditions are artificial
• results may not generalize to the real world
• Some questions cant be tested in an experiment
• Require control that is not always available in
  the real world

5
Experimental Design

• Controlling potential confounds
• Goal of experiment is to rule out alternate
  explanations of what affected dependent variable
• Confounds are threats to internal validity
• Can be controlled through appropriate
  experimental design and procedures

6
Validity

• Internal Validity
• History
• Maturation
• Testing
• Instrumentation
• Statistical Regression
• Selection
• Mortality
• Selection-Maturation
• Diffusion of Treatment

• External Validity
• Sample
• Setting (e.g., culture)
• Time (e.g., 60s vs. 90s)
• Replication (lack of)

Do the results of this experiment generalize
(apply) to settings other than the experiment
Is there another reason (other than the
independent variable) that could explain the
results of the experiment.
7
Sampling

• How participants are selected for a study
  influences the extent to which the results can be
  applied to a larger group (external validity).
• A wide variety of techniques are available
• Two Main types of sampling
• Probability
• predetermined chance of any individual in the
  population being selected for the study
• Nonprobability
• Typically nonrandom sampling

8
Sampling Techniques

• Probability Sampling
• Simple random sampling
• Systematic sampling
• Stratified random sampling
• Cluster sampling
• Multistage sampling

• Nonprobability Sampling
• Convenience sampling
• Quota sampling
• Snowball sampling

9
Post with no Control Group
Training
Posttest
10
Pre Post with no Control Group
Pretest
Training
Posttest
11
Control Group with no Pretest
Experimental Group
Training
Posttest
Group Differences
Control Group
Placebo
Posttest
12
Pre Post with Control Group
Pretest
Experimental Training
Posttest
Group Differences
Group Differences
Pretest
Control
Posttest
13
Measurement

• Measurement the process of assigning numbers to
  objects or events according to rules (Linn
  Gronlund, 1995).
• Psychological Measurement concerned with
  evaluating individual differences in
  psychological traits.
• Trait descriptive label applied to a group of
  behaviors (e.g., friendly intelligent)

14
Utilizing Individual Differences

• Psychologists assume that most traits are
  normally distributed in the population.
• e.g., height, intelligence, KSAs
• Psychologists study
• measuring these differences
• using these differences to predict performance
• I/O Psychologists typically primarily rely on
  these as predictors of job performance
• Cognitive abilities
• Personality

15
Types of Assessment

• Biographical Information
• Interviews
• Work Samples
• Letters of Recommendation
• Psychological Tests

16
Biographical Data

• Good questions are about events that are
• historical
• external
• discrete
• controllable (by the individual)
• verifiable
• equal access
• job relevant
• non-invasive
• (Mael, 1991)
• Rationale vs. empirical method

17
Biographical Data

• Strong criterion validity
• drug use, criminal history predicts dysfunctional
  police behavior (Sarchione et al., 1998)
• not redundant with personality (McManus Kelly,
  1999)
• Measurement issues
• Generalizability
• Faking
• Fairness
• Privacy concerns

18
Interviews

• Structured vs. Unstructured
• Info. gathering vs. interpersonal behavior sample
• Situational interview
• How would you handle a circumstance in which you
  needed the help of a person you did not like?
• Measurement issues
• structured has more criterion related validity
• value of unstructured?
• Illusion of validity
• Guidelines for structured interviews

19
Work Samples

• perform a task under standardized conditions
• historically were for blue collar jobs
• e.g. use of tools, demonstrate driving skills
• white collar examples
• speech interview for foreign worker, test of
  basic chemistry knowledge,
• Measurement issues
• high criterion validity if skills are similar to
  job
• costly to administer
• work best with mechanical, rather than
  people-oriented tasks

20
Assessment Centers

• Realistic tasks done in groups
• Assessed by multiple of raters rating multiple
  domains
• Multiple methods
• in basket group exercise
• leaderless group exercise
• Strong criterion validity (e.g., teachers,
  police)
• overall scores predict job performance
• Measurement issues
• costly to administer
• different ratings on a task too highly correlated
• dimension ratings not correlated strongly across
  tasks
• fix? focus on behavior checklists and rater
  training

21
Drug Testing

• opinion?
• People are more accepting of it if job involves
  risks to others (Paronto, et al., 2002)
• Measurement issues
• reliability is very high, but not perfect
• Validity?
• Normands, Salyards, Mahoney (1990)
• over 5000 postal service applicants
• those who tested positive had 59 higher
  absenteeism, 47 more likely to be fired
• no differences in injury or accidents

22
Letters of Recommendation

• ever written a letter of recommendation for
  someone?
• worst criterion validity of all commonly used
  assessment tools
• some use for screening extremely bad candidates
• Measurement issues
• restriction of range
• writer bias/investment

23
Psychological Test Characteristics

• Group vs. individual
• Objective vs. open-ended
• Paper pencil vs. performance
• Power vs. speed

24
Psychological Test Types

• Ability Tests
• Cognitive ability
• Psychomotor ability
• Knowledge and skill or achievement
• Integrity
• Personality
• Emotional Intelligence
• Vocational interest

25
Integrity Tests

• Designed to predict whether employee will engage
  in counterproductive work behavior (CWB)
• overt vs. personality (covert)
• Better at predicting general CWB and performance
  than theft (r .30 -.40)
• Measurement issues
• difficult to measure criteria!
• proprietary issues
• legal and privacy issues
• faking

26
Personality Tests

• measures predispositions toward particular
  feelings and behaviors
• not all tests are based on past research
• many have shown incremental validity
• e.g., predict when controlling for IQ
• Measurement issues
• job relevance
• not easily/often faked or a problem if faked
  (e.g., job faking too)

27
The Big Five Inventory

• Openness
• Highs imaginative, creative, and to seek out
  cultural and educational experiences.
• Lows more down-to-earth, less interest in art
  more practical.
• Conscientiousness
• Highs methodical, well organized and dutiful.
• Lows less careful, less focused more likely to
  be distracted
• Extraversion
• Highs energetic and seek out the company of
  others.
• Lows (introverts) tend to be more quiet and
  reserved.
• Agreeableness
• Highs tend to be trusting, friendly and
  cooperative.
• Lows tend to be more aggressive and less
  cooperative
• Neuroticism
• Highs prone to insecurity and emotional
  distress.
• Lows more relaxed, less emotional and less prone
  to distress.

28
Cognitive Tests

• Have greatest validity
• Often very easy and inexpensive to use
• Wonderlic Personnel Test
• 50 items
• 12 minute time limit
• Sample questions
• Interpreting scores?
• Scores vary as a function of race and ethnicity
• Ethical issues?
• Face validity?

29
(No Transcript)
30
Psychological Test Characteristics

• Group vs. individual
• Objective vs. open-ended
• Paper pencil vs. performance
• Power vs. speed

31
Reliability and Validity

• Reliability
• Test-retest
• Parallel (Alternate) forms
• Internal Consistency
• Validity
• Face
• Content
• Criterion-related
• Construct-related