Assessing Online Privacy Notice Readability

1
Assessing Online Privacy Notice Readability

• George R. Milne
• University of Massachusetts, Amherst

2
Consumer Feedback

• Privacy notices are deliberately made too long
  and verbose. How about the Privacy Notices for
  Dummies version?
• Get real, no one in their right mind WANTS to
  read that mumbo jumbo verbiage
• I wish they would just simplify the basic points
  and have a long wordy version as an option and to
  cover their bums in case of some stupid money
  grubber suing them.
• I dont have a law degree

3
How Effective are Online Privacy Notices?

• Most notices cover Fair Information Principles
• New laws such as HIPAA and GLB make notices
  important
• How readable are noticescan most people
  understand them?

4
Assessing Readability

• Readability has been studied since 1920s
• Readability used in marketing literature to
  assess
• magazine ads (Rossiter 1988)
• product warnings (Riswalder 1989)
• warranties (Moore and Kelley 1993)
• academic journals (Clark and Geisler 1986).

5
Small scale examination of Privacy Notice
Readability

• Hochasuer 200331 hippa notices
• Milne and Culnan 200215 privacy notices
• Anton et. al 200440 online policy documents from
  9 financial institutions
• Jensen and Potts (2004)47 high traffic sites, 17
  health related sites.

6
Research Questions

• How have online privacy notices changed from 2001
  to 2003 in terms of
• Readability?
• Length of notices?
• Use of hyperlinks?
• Are privacy notices that display privacy seals
  more readable?
• Does readability of privacy notices vary across
  industry sector?

7
Methodology

• 483 websites drawn from media metrixs list of
  top websites for Nov. 2000
• All sites with privacy notice linked to home page
  reviewed in Mar. 2001., Mar. 2003
• Final sample, N312
• Data coded by graduate students with instructions
• Policies converted to text file, analyzed using
  Readability Calculations software
• Multiple readability measures used

8
Readability Indices

• Dale Chall (8.0-8.911th,12th grade. 9.0
  college)
• Flesch-Kincaid Grade Level (grade level)
• Fog Index (reading age to understand grade
  5years)
• Smog Index (grade level)

Composite Readability Index created by linear
combination of normed readability indices
9
Measurement Model showed invariance
Fog
Fog
.35
.82
.80
Smog
Smog
Read2001
.98
Read 2003
.99
FKGL
FKGL
.97
.97
.58
Dale Chall
Dale Chall
.53
CFA.913, NFI.908, TLI.856
Errors of fog and dale chall were correlated
10
Readability Declined from 2001 to 2003
Some College is Needed
11
Education Level Needed to Read Website Privacy
Notices
12
Notices became longer, yet more linked
All t-tests plt.05
13
Privacy notices with privacy seals were slightly
more readable
t-test plt.05
14
Notices with Seals were longer and more linked
All t-tests plt.05
15
Readability varies across industry sector
P(F)lt.05
16
Conclusions

• Approximately half the online privacy notices are
  at a higher education level than half the U.S.
  Population
• Notices are growing in lengthon average over 4
  pages of text
• More hyperlinks being used
• Notices with seals are more readable, yet longer
  (6 pages of text)
• Notices vary by sector, effect of legislation not
  conclusive

17
A Short Notice Format is Needed