Kenneth S. Law IACMR Workshop, 20077

1
????
Kenneth S. Law (???)?????? ???IACMR Workshop,
??2007?7?
2
Different types of studies

• Correlational survey studies
• Experimental laboratory studies
• Quasi experiments
• Qualitative studies
• Qualitative reviews
• Meta analysis (Quantitative reviews)

3
The Survey Process

• Idea generation
• Find some hot topics in the literature
• Data collection
• Collect as many related variables as possible
  around a topic in a survey
• Data analysis
• See which pairs of correlations are significant
• Try to massage the data so as to get good results
• Use the most up-to-date analytical tools
• Write up the manuscript
• Try to build up a story based on the significant
  results
• Find a theory related to your results

4
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Multidimensional constructs
• Pilot test
• Convergent Discriminant Validity
• Issues in Questionnaire design
• How to collect data?
• Data analysis

5
1. What is your research question?

• What is your contribution to the literature?
• Is the research question testable?
• Are the constructs well defined?
• Do we have enough validated scales to measure the
  constructs?
• Are the relationships well justified?

6
Contributions

• Theoretical contributions
• New constructs
• New phenomena
• New findings
• New perspectives
• Methodological contributions
• New measures (e.g., new scale development)
• New methods (e.g., cross-level research)

7
What are theoretical contributions?

• A complete theory should contain four elements
• What. Which factors logically should be
  considered as part of the explanation of the
  phenomena? (factor comprehensiveness and
  parsimony)
• How. How are they related?
• Why. What are the underlying psychological,
  economic, or social dynamics that justify the
  selection of factors and the proposed causal
  relationships?
• Who, where, when. These conditions place
  limitations on the propositions generated from a
  theoretical model.

Whetten, D.A. (1989) What constitute a
theoretical contribution. AMR, 1494), 490-495.
8
Job Design
Job Characteristics Model Hackman
Oldham (1976)
MPS
Job Satisfaction
Social Information Processing Model
Salancik Pfeffer (1978)
Job Satisfaction
Perceived job characteristics
9
Why employees want fairness?
Procedural vs. Distributive justice
Instrumental Model
Group Value Model Folger Konovsky (1989)
10
A research question

• Self concept

Downsizing Outsourcing Re-engineering
Productivity
11
Workplace Self Concept
Self Concept
Sociology
Identification
Psychology
WSC
Evaluation
Workplace
12
2. Is the research question testable?

• What is your contribution to the literature?
• Is the research question testable?
• Are the constructs well defined?
• Do we have enough validated scales to measure the
  constructs?
• Are the relationships well justified?

13
???????
14
3. Are the constructs well defined?

• What is your contribution to the literature?
• Is the research question testable?
• Are the constructs well defined?
• Do we have enough validated scales to measure the
  constructs?
• Are the relationships well justified?

15
A research question
Downsizing Outsourcing Re-engineering
Productivity

• Workplace Self Concept (WSC)
• General Self Efficacy
• Organizational-Based Self Esteem
• Core Self Evaluation

16
Construct validity
Self Efficacy
Workplace Self Concept
Discriminant validity
Core Self Evaluation
Convergent Validity
17
Content validity

• Workplace Self Concept include
• Supervisor
• Subordinate
• Colleague
• Employee
• Career

18
4. Measurement Issues

• What is your contribution to the literature?
• Is the research question testable?
• Are the constructs well defined?
• Do we have enough validated scales to measure the
  constructs?
• Are the relationships well justified?

19
Use Established Scales
Self Emotion Appraisal ?????????????????? ????????
?? ???????????? ??????????????????? Regulation of
Emotion ?????,?????????? ??????????? ?????,???????
????????? ???????????????? Use of
Emotion ????????????????????? ???????????????? ???
????????? ????????????? Other's Emotion
Appraisal ??????????????????? ????????????? ??????
??????????? ????????????

• Law, K.S., Wong, C. Song, L.J. (2004). The
  construct and criterion validity of emotional
  intelligence and its potential utility for
  management studies, JAP, 89(3)483-496.
• Wong, C., Law, K.S. (2002). The effects of leader
  and follower emotional intelligence on
  performance and attitude An exploratory study.
  The Leadership Quarterly, 13, 243-274.

20
5. Hypotheses

• What is your contribution to the literature?
• Is the research question testable?
• Are the constructs well defined?
• Do we have enough validated scales to measure the
  constructs?
• Are the relationships well justified?

21
A research question

• Are there existing empirical evidence to support
  the hypothesis?
• Are there un-obvious logical arguments to justify
  your hypothesis?
• Is there a theoretical perspective to justify
  your hypothesis?

22
Three possible arguments

• Bass and Bentler (2001) found that followers who
  followed transformational leaders have a stronger
  vision of where the firm is heading to. As a
  result, we hypothesize that
• A transformational leader leads by creating
  visions for his/her followers. They share their
  visions with their followers and communicate with
  their followers continuous on these visions.
  Since mission and vision is a core component of
  organizational commitment, we hypothesize that
• According to the social exchange theory,
  leader-follower relationship that engages in
  social exchange would expect long term
  reciprocity instead of immediate reward, we
  therefore hypothesized that

23
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs
• Pilot test
• Issues in Questionnaire design
• How to collect data?
• Data analysis

24
3. Measure your construct of interest

• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs

25
3a. Level of Analysis

• Individual level/group level/firm level/industry
  level/cross level
• Example
• The effects of LMX on employee performance.
• On the antecedents and outcomes of group-level
  OCB.
• The effect of HRM practices on firm performance
• The effect of HRM practices on the job
  satisfaction of employees.

26
3a. Level of Analysis
Perceived Organizational Support
Organizational Citizenship Behaviors
Firm Performance
General Manager
Employees
Employees
No. of firms 98 No. of employees per firm 15
27
3. Measure your construct of interest

• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs

28
3b Data source and CMV
Try to solicit data (esp. predictor vs. criterion
variables) from different sources. The problem
of common method variance (CMV)
29
A Method Factor

• Organizational commitment (affective)
  ??? ??
• 1.??????????????????? 1 2 3 4 5
• 2.??????????????????? 1 2 3 4 5
• 3.??????????????????? 1 2 3 4 5
• Turnover Intention
• 7. ???????? 1 2 3 4 5
• 8. ?????????????? 1 2 3 4 5
• 9. ???????,?????????????1 2 3 4 5

30
Rotated Factor Matrix in EFA
Factors Var A B C X1
.29 .60 -.06 X2 .32 .81 .12 X3
.35 .77 .03 X4 .27 .01 .65 X5
.41 .03 .80 X6 .40 .12 .67
Organizational commitment
Turnover intention
31
One Factor Test
Dc2
32
An example
HRM practices of the firm
Degree of social exchange in the organization
Individual performance of employees
Source of information
Middle managers
Top level managers
HR manager
33
Different methods/sources
Organizational commitment
Organizational culture

• Not reported by employee
• rites and ceremonials

• Self reported by employee

reported by employee
reported by supervisor/peer
34
3. Measure your construct of interest

• What is your level of analysis?
• What is your data source?
• Use validated scales if possible.
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs

35
3c Using existing scales

• Adapting measures
• Ratee perceptions was measured by a four-item
  scale adapted from Atwater et al. (2000)
• What has changed? Why?
• Adopting measures
• Moorman (1991) has seven items measuring
  procedural justice
• Procedural justice was measured by three items
  from Moorman (1991)
• Combining measures
• Perception of rater credibility was measured by a
  six-item scale adapted from Kerst (1997) and
  Facteau et al. (1998).
• Perceived demographic similarity was measured
  using four single-item measures based on work by
  Kirchmeyer (1995), Louis (1978), and Riordan
  (1997, 2000).

36
What measure to be used?

• Use full scale of existing validated scales
• Select items only when you have perfect
  justifications
• Use scales that have been validated (esp. cross
  culturally)
• Develop you own measure when you have a strong
  reason that existing measures do not fit or
  there is no good measure of the construct.

37
3c Using existing scales
Measure We developed five items to measure
emotional intelligence in this study. One sample
item is I am able to control my temper most of
the time. Coefficient a of the five items was
.89.

• Problems
• We do not know how the items are developed.
• There is no evidence of validity of the items.
• We do not know whether you have done any item
  trimming or not.
• If yes, we do not know the criteria of item
  selection.

38
3c Using existing scales

• Follow the proper procedure of scale translation.
• The minimum requirement is a forward-backward
  translation.
• It is best to pre-test your (translated) scale
  before use.

39
3. Measure your construct of interest

• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs

40
3d. Developing new scales

• Inductive vs. deductive approach for scale
  development

• Inductive
• Usually behavioral measures of constructs
• E.g., Managers write statements to describe
  behaviors of a transformational leader
• Researcher group all items and sort them into
  various dimensions using systematic
  classification techniques
• Select items to represent each dimension
• Pretesting of the scale

41
Developing new scales

• Deductive
• Start with theory to determine the dimensionality
  of the construct
• For each and every dimension, draft items to
  represent the dimension
• Pretesting of the scale
• Item trimming
• Final validation

42
3e. Formative vs. Reflective indicators
z
Income
Relax
e1
Socio-economic status
Life satisfaction
Happy
e2
Parents income
Positive
e3
Size of apartment
Reflective or effect indicators
Formative or causal indicators
Please give one example of each type of construct
43
3f. Multidimensional constructs
Quality
Aggregate Model
Job Performance
Quantity
On-time
Math
Mental Ability
Latent Model
Verbal
Memory
44
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs
• Pilot test
• Issues in Questionnaire design
• How to collect data?
• Data analysis?
• Convergent Discriminant Validity
• Confirmatory Factor Analysis
• Mediators and moderators
• Cross level analysis

45
4. Pilot Test

• Item trimming (EFA)
• Factor loading gt.4
• Low cross loading
• Item difficulty/Item reliability
• Never trim items based on EFA and then retest
  with a CFA using the same sample
• Cross validation

46
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs
• Pilot test
• Issues in Questionnaire design
• How to collect data?
• Data analysis

47
5. Questionnaire Design

• Question sequencing
• Dependent variables first
• Randomization?
• Grouping of constructs
• Length of questionnaire ( of pages)
• What constructs to include (two papers but not
  too long)

???? ??????????????????????????????,??????????????
?????
48
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs
• Pilot test
• Issues in Questionnaire design
• How to collect data?
• Data analysis

49
6. Data collection

• Minimum N is 15 (one respondent for each item
  within a construct)
• Minimum N gt100 for group level gt200 for
  individual level
• You should be there during data collection.
• Questionnaire distribution the higher the level
  the better

50
Issues in survey design

• What is the research question?
• What are the hypotheses?
• Measure your construct of interest
• What is your level of analysis?
• What is your data source?
• Use validated scales if possible
• The scale development process
• Formative vs. Reflective indicators
• Multidimensional constructs
• Pilot test
• Issues in Questionnaire design
• How to collect data?
• Data analysis

51
7.Data analysis

• Clean your data
• Examine descriptive statistics
• Look at your correlation table
• start with simple analyses
• Test your hypotheses with the appropriate
  analytical tools (i.e., H0) (e.g., mediation)
• Analyze your data at the appropriate level of
  analysis
• Individual level vs. group level vs. firm level
• Dimensional level or construct level
• Do not separate your sample using sub-group
  analysis unless you have no choice (testing
  moderators)

52
7.Data analysis

• Confirmatory factor analysis of all items from
  the same source (no separate CFA)
• When to use CFA vs. EFA?
• Never trim items based on EFA and then retest
  with a CFA using the same sample
• Separate measurement model from structural model
• Using parcels when number of items are large?

53
Forming parcels in CFA
h1
h2
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
h2
54
Data Dependency

• Supervisor 1 Subordinate 1
• Supervisor 1 Subordinate 2
• Supervisor 1 Subordinate 3
• Supervisor 5 Subordinate 1
• Supervisor 5 Subordinate 2
• Supervisor 5 Subordinate 3

55
Sample size in HLM
b13
b03
b12
b02
b11
b01
56
Thank you!