Trust and Semantic attacks

1
Trust and Semantic attacks
Ponnurangam Kumaraguru (PK) Usable, Privacy, and
Security Mar 17, 2008
2
Who am I?

• Ph.D. candidate in the Computation,
  Organizations, and Society program in the School
  of Computer Science
• Research interests - Privacy, Security, Trust,
  Human Computer Interaction, and Learning Science

3
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

4
What is trust?

• No single definition
• Depends on the situation and the problem
• Many models developed
• Very few models evaluated

5
Trust in literature

• Economics (how trust affects transactions)
• Reputation
• Marketing (how to build trust)
• Persuasion
• HCI (what affects trust)
• Design
• Psychology (positive theory)
• Intimacy

6
Trust Models

• Negative antecedents
• Risk
• Transaction cost
• Uncertainty

• Positive antecedents
• Benevolence
• Comprehensive information
• Credibility
• Familiarity
• Good feedback
• Propensity
• Reliability
• Usability
• Willingness to transact

7
How do users make decisions?

• Interview design, 25 participants (11 - experts
  and 14 - non-experts)
• Measured the strategies and decision process of
  the users in online situations
• Results
• Non-experts wanted advice to help them make
  better trust decisions
• Non-experts used significantly fewer meaningful
  signals compared to experts

P. Kumaraguru, A. Acquisti, and L. Cranor. Trust
modeling for online transactions A phishing
scenario. In Privacy Security Trust, Oct 30 -
Nov 1, 2006, Ontario, Canada.
8
Expert model
Unknown states
Not deliberate states
Signals
States that affect decision
States that affect well-being
Meaningful signals
Misleading signals
Missed signals
9
Non- expert model
Unknown states
Not deliberate states
Signals
States that affect decision
States that affect well-being
Misleading signals
Meaningful signals
Missed signals
10
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

11
Security Attacks Waves

• Physical attack the computers, wires and
  electronics
• E.g. physically cutting the network cable
• Syntactic attack operating logic of the
  computers and networks
• E.g. buffer overflows, DDoS
• Semantic attack the user not the computers
• E.g. Phishing

http//www.schneier.com/essay-035.html
12
Semantic Attacks

• Target the way we, as humans, assign meaning to
  content.
• System and mental model

http//groups.csail.mit.edu/uid/projects/phishing/
proposal.pdf
13
An email that we get
14
Features in the email
Subject eBay Urgent Notification From Billing
Department
15
Features in the email
We regret to inform you that you eBay account
could be suspended if you dont update your
account information.
16
Features in the email
https//signin.ebay.com/ws/eBayISAPI.dll?SignInsi
dverifyco_partnerid2sidteid0
17
Website to collect information
http//www.kusi.org/hcr/eBay/ws23/eBayISAPI.htm
18
What is phishing?

• Phishing is a broadly launched social
  engineering attack in which an electronic
  identity is misrepresented in an attempt to trick
  individuals into revealing personal credentials
  that can be used fraudulently against them.

Financial Services Technology Consortium.
Understanding and countering the phishing threat
A financial service industry perspective. 2005.
19
Phishing Attack Life Cycle
Sourcehttp//www.coopercain.com/User20Data/A20L
eisurely20Lunch20Time20Phishing20Trip-show.ppt
20
A few statistics on phishing

• 73 million US adults received more than 50
  phishing emails each in the year 2005
• Gartner in 2006 found 30 users changed online
  banking behavior because of attacks like phishing
  
• Gartner in 2006 predicted 2.8 billion loss due
  to phishing in that year

21
Why phishing is a hard problem?

• Semantic attacks take advantage of the way humans
  interact with computers
• Phishing is one type of semantic attack
• Phishers make use of the trust that users have on
  legitimate organizations

22
Three strategies for usable privacy and
security

• Invisible strategy
• Regulatory solution
• Detecting and deleting the emails
• User interface based
• Toolbars
• Training users

23
Our Multi-Pronged Approach

• Human side
• Interviews to understand decision-making
• PhishGuru embedded training
• Anti-Phishing Phil game
• Understanding effectiveness of browser warnings
• Computer side
• PILFER email anti-phishing filter
• CANTINA web anti-phishing algorithm

Automate where possible, support where necessary
24
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

25
Why user education is hard?

• Security is a secondary task
• Users not motivated to taking time for education
• Non-existence of an effective method

26
To address the open questions

• Embedded training methodology
• Make the training part of primary task
• Create motivation among users
• Learning science
• Principles for designing training interventions

27
Approaches for training

• Posting articles
• FTC,
• Phishing IQ tests
• Mail Frontier,
• Classroom training
  (Robila et al.)
  
  
• Sending security notices

http//www.ftc.gov/bcp/conline/pubs/alerts/phishin
galrt.htm http//www.sonicwall.com/phishing/ http
//pages.ebay.com/education/spooftutorial/
28
Security notices

• How to spot an email
• How to report spoof email
• Five ways to protect yourself from identity theft

29
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

30
Why learning science?

• Research on how people gain knowledge and learn
  new skills
• ACT-R theory of cognition and learning
• Declarative knowledge (knowing that)
• Procedural knowledge (knowing how)
• Learning science principles

31
Learning science principles

• Learning-by-doing
• More practice better performance
• Story-based agent
• Using agents in a story-based content enhances
  user learning
• Immediate feedback
• Feedback during learning phase results in
  efficient learning

Clark, R.C., and Mayer, R.E. E-Learning and the
science of instruction proven guidelines for
consumers and designers of multimedia learning.
John Wiley Sons, Inc., USA, 2002.
32
Learning science principles

• Conceptual-procedural
• Presenting procedural materials in between
  conceptual materials helps better learning
• Contiguity
• Learning increases when words and pictures are
  presented contiguously than isolated
• Personalization
• Using conversational style rather than formal
  style enhances learning

Clark, R.C., and Mayer, R.E. E-Learning and the
science of instruction proven guidelines for
consumers and designers of multimedia learning.
John Wiley Sons, Inc., USA, 2002.
33
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

34
Design constraints

• People dont proactively read the training
  materials on the web
• People can learn from web-based training
  materials, if only we could get people to read
  them! (Kumaraguru et al.)

P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor,
and J. Hong. Teaching Johnny Not to Fall for
Phish. Tech. rep., Cranegie Mellon University,
2007. http//www.cylab.cmu.edu/files/cmucylab07003
.pdf.
35
Embedded training

• We know people fall for phishing emails
• So make the training available through the
  phishing emails
• Training materials are presented when the users
  actually fall for phishing emails
• Makes training part of primary task
• Creates motivation among users
• Applies learning-by-doing and immediate feedback
  principle

36
Embedded training example
Subject Revision to Your Amazon.com Information
37
Embedded training example
Subject Revision to Your Amazon.com Information
Please login and enter your information
http//www.amazon.com/exec/obidos/sign-in.html
38
Comic strip intervention
39
Design rationale

• What to show in the intervention?
• When to show the intervention?
• Analyzed instructions from most popular websites
• Paper and HTML prototypes, 7 users each
• Lessons learned
• Two designs
• Present the training materials when users click
  on the link

40
Study 1 Evaluation of interventions

• H1 Security notices are an ineffective medium
  for training users
• H2 Users make better decisions when trained by
  embedded methodology compared to security notices

41
Study design

• Think aloud study
• Role play as Bobby Smith, 19 emails including 2
  interventions, and 4 phishing emails
• Three conditions security notices, text /
  graphics intervention, comic strip intervention
• 10 non-expert participants in each condition, 30
  total

P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor,
J. Hong, and E. Nunge. Protecting People from
Phishing The Design and Evaluation of an
Embedded Training Email System. CyLab Technical
Report. CMU-CyLab-06-017, 2006.
http//www.cylab.cmu.edu/default.aspx?id2253 to
be presented at CHI 2007
42
Intervention 1 - Security notices

• How to spot an email
• How to report spoof email
• Five ways to protect yourself from identity theft

43
Intervention 2 - Comic strip
44
Intervention 2 - Comic strip
Applies personalization and story based
principle Presents declarative knowledge
45
Intervention 2 - Comic strip
Applies personalization principle
46
Intervention 2 - Comic strip
Applies contiguity principle
47
Intervention 2 - Comic strip
Applies contiguity and conceptual-procedural
principle Presents procedural knowledge
48
Intervention 3 - Text / graphics
49
User involvement
50
Legitimate
Phish
Training
Spam
51
User study - results

• We treated clicking on link to be falling for
  phishing
• 93 of the users who clicked went ahead and gave
  personal information

52
User study - results
53
User study - results

• Significant difference between security notices
  and the comic strip group (p-value lt
  0.05)
• Significant difference between the comic and the
  text / graphics group (p-value
  lt 0.05)

54
Lessons learned

• H1 Security notices are an ineffective medium
  for training users

Supported

• H2 Users make better decision when trained by
  embedded methodology compared to security notices

Supported
55
Open questions

• Previous studies measured only knowledge gain
• Users have specific knowledge than generalized
  knowledge (Downs et al.)
• What about knowledge retention and transfer?

56
Knowledge retention and transfer

• Knowledge retention (KR)
• The ability to apply the knowledge gained after a
  time period
• Knowledge transfer (KT)
• The ability to transfer the knowledge gained from
  one situation to another situation

57
Study design

• Setup
• Think aloud study
• Role play as Bobby Smith, business administrator
• Respond to Bobbys email
• Experiment
• Part 1 33 emails and one intervention
• Part 2 (after 7 days) 16 emails and no
  intervention
• Conditions
• Control no intervention
• Suspicion an email from a friend
• Non-embedded intervention in the email
• Embedded intervention after clicking on link

58
Sample of emails from study
Email type Sender Subject information
Legitimate-no-link Brandy Anderson Booking hotel rooms for visitors
Legitimate-link Joseph Dicosta Please check PayPal balance
Phishing-no-account Wells Fargo Update your bank information!
Phishing-account eBay Reactivate your eBay account
Spam Eddie Arredondo Fw Re You will want this job
Intervention Amazon Revision to your Amazon.com information
59
Comic strip intervention
60
Hypotheses

• H1 Participants in the embedded condition learn
  more effectively than participants in the
  non-embedded condition, suspicion condition, and
  the control condition
• H2 Participants in the embedded condition retain
  more knowledge about how to avoid phishing
  attacks than participants in the non-embedded
  condition, suspicion condition, and the control
  condition

61
Hypotheses

• H3 Participants in the embedded condition
  transfer more knowledge about how to avoid
  phishing attacks than participants in the
  non-embedded condition, suspicion condition, and
  the control conditions

62
Study results

• We treated clicking on link to be falling for
  phishing
• 89 of the users who clicked went ahead and gave
  personal information

63
Results - Phishing account emails
64
Results - Legitimate link emails
65
Measuring retention

• Training on Amazon.com account revision phish
• Testing a week later on Citibank account revision
  phish
• Significant difference between embedded and other
  groups (p lt 0.01)
• I remember reading last time that thing
  training material said not click and give
  personal information.

66
Measuring transfer

• Training on Amazon.com account revision phish
• Testing a week later on eBay account reactivation
  phish
• Significant difference between embedded and other
  groups (p lt 0.01)
• PhishGuru said not to click on links and give
  personal information, so will not do it, I will
  delete this email.

67
A few observations

• I was more motivated to read the training
  materials since it was presented after me falling
  for the attack.
• Thank you PhishGuru, I will remember that the 5
  instructions given in the training material.
• This image in the email looks like some spam.

68
Outline

• Trust
• Semantic attacks - Phishing
• User education
• Learning science
• Evaluating embedded training
• Ongoing work
• Conclusion

69
Ongoing work

• Test the system in real-world

70
Conclusion

• Educating users about security can be a reality
  rather than just a myth

71
Collect homework
72
Acknowledgements

• Members of Supporting Trust Decision research
  group
• Members of CUPS lab

73
(No Transcript)
74
CMU Usable Privacy and Security
Laboratoryhttp//cups.cs.cmu.edu/
75
Learning-by-doing principle

• Production rules are acquired and strengthened
  through practice
• More practice better performance
• Story-centered curriculum
• Cognitive tutors

76
Immediate feedback principle

• Feedback during knowledge acquisition phase
  results in efficient learning
• Corrects behavior
• Avoids floundering
• LISP tutors
• yes or no or detailed

77
Conceptual-Procedural principle

• A concept is a mental representation or
  prototype of objects or ideas
• A procedure is a series of clearly defined steps
• Presenting procedural materials in between
  conceptual materials helps better learning
• Studies
• Mathematics

78
Contiguity principle

• Learning increases when words and pictures are
  presented contiguously rather than isolated from
  one another
• Human learning process - creating meaningful
  relation between pictures and words
• Studies
• Vehicle braking system
• Geometry cognitive tutor

79
Personalization principle

• Using conversational style rather than formal
  style enhances learning
• To use I, we, me, my, you, and your
  in the instructional materials
• Studies
• Process of lightning formation
• Mathematics

80
Story-based agent principle

• Characters who help in guiding the users through
  the learning process
• Using agents in a story-based content enhances
  user learning
• Stories simulate cognitive process
• Experiments - Herman