Chapter 27: Reliability

1
Chapter 27 Reliability

• Definitions of reliability
• Theory of reliability
• The interpretation of the reliability coefficient
• The improvement and value of reliability
• Real case

2
Definitions of reliability

• Three approaches to define reliability
• 1. Stability, dependability, predictability
• 2. Lack of distortion
• 3. How much error of measurement(the relative
  absence of error of measurement in a measuring
  instrument)
• Which one deal with the terms of accuracy?
  Reliability? or Validity? or Both? (different
  semantics)
• Reliability accuracy that a measuring instrument
  measures whatever it measures
• Validity accuracy that a measuring instrument
  actually measures whatever it measures

3
Relationship between reliability and validity

• The examples of weight scale and old / new rifle
• Could reliability and validity both exist?
• To have reliability without validity? or to have
  validity without reliability? Which one is
  possible?

Larger variability Less reliable
Reliability, Low validity
Reliable
4
Theory of reliability

• O T E (obtained score true score error)
• VO VT VE

Which one is more reliable? (A)
5
Theory of reliability (cont.)

• Reliability
• Reliability

, proportion of the true variance to the total
obtained variance
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Generalizability Theory
1.across time 2.across other setting 3.across
individual
Cronbach, LJ, Gleser, GC, Nanda, H. and
Rajaratnam, N. (1972). The Dependability of.
Behavior Measurements Theory of Generalizability
for Scores and Profiles, WileyNY.
12
Coefficient Alpha, Cronbach, 1951

• What Is Coefficient Alpha? by Cortina, 1993, JAP

13
What Is Coefficient Alpha?How to interpret the
reliability coefficient?

• A proportion or percentage of the variable shared
  by two variables
• Also a coefficient of determination to tell how
  much variance is true one in the measured
  variable
• The correlation of the true score with the
  observed score is often referred to as the index
  of reliability
• 1. Test-retest reliability
• 2. Equivalent / parallel forms
• 3. Internal consistency (split-half reliability,
  acoefficient, Kuder-Richardson formulas 20 and 21
  (KR-20, KR-21)

14
What Is Coefficient Alpha? An Examination of
Theory and Applications

• Cortina, J. M. (1993)
• Journal of Applied Psychology, 78, pp.98-104.

15
Summary of reliability estimates
16
Forms of reliability

• If error factors associated with the passing of
  time are of interest, test-retest or multiple
  administrations of parallel tests may be used
• If error factors associated with the use of items
  are of interest, coefficient alpha or single
  administration of parallel tests may be used

17
Alpha is the mean of all split-half reliabilities

• Formula for Cronbachs alpha
• Formula for split-half reliability (Spearman and
  Brown)
• Formula for split-half reliability (Flanagan and
  Rulon)

18
Conclusions from table 2

• Alpha can be high in spite of low item
  intercorrelations and multidimensionality.
• af (intercorrelation, no. of items, no. of
  dimensions)
• a is positively correlated with average item
  intercorrelations
• a is positively correlated with number of items
• a is negatively correlated with number of
  dimensions

intercorrelations


no. of items
-
no. of dimensions
19
Precision is measured in terms of the
standard error of intercorrelations
20
(No Transcript)
21
The standard error of the mean and the standard
error of measurement

• standard variance of measurement
• Therefore,
• Then,

22
(No Transcript)
23
The improvement and value of reliability

• Maxmincon principle
• (maximize the variance of the individual
  differences, and minimize the error variance)
• 1. Not unambiguously items ? item analysis
• 2. More items, let
• 3. Clear and standard instruction

24
The value of reliability

• Reliability is a necessary but not a sufficient
  condition for validity
• ????,??????
• ???????????????? ??????
• .7 as a cut point of coefficient of reliability,
  but still depends on how the measure is used

HW ???A????,???????????
25
Real caseYang and Wu, CyberPsychology
Behavior (accepted 2006)
26
Measurement

• Gender and Internet Consumers Decision-Making
• Chyan Yang Chia-Chun Wu
• CyberPsychology Behavior, accepted for
  publication (2006)
• Internet questionnaire
• Forty-item Consumer Style Inventory (CSI) ,
  five-point scale (see as the following)
• Method
• Exploratory factor analysis (EFA) ?online
  shoppers decision-making styles
• Discriminant analysis ?gender differences

27
Reliability

• Internal consistent reliability
• Cronbach a ( gt0.7)
• Example
• Taiwan Online Shoppers Style Characteristics
  Six-Factor Model

28
DATA D1 INPUT a1-a40 CARDS ...(data , 472
observations) PROC CORR DATAD1 ALPHA
NOMISS VAR a9-a11 a13 a15 RUN
29
(No Transcript)
30
Validity

• Content validity
• The questionnaire was adopted from Consumer Style
  Inventory (CSI) which was suggested by Sproles
  (1985) and Sproles and Kendall(1986)

31
(No Transcript)