Evolution, Deception and Terror

1
Evolution, Deception and Terror

• Ross Anderson
• Cambridge

2
Whats Dependability?

• Were building big complex socio-technical
  systems
• The global card payments system
• The European smart grid
• Facebook
• The NHS database
• What does it take for these systems to be
  dependable?

3
Economics

• What does it even mean for these systems to be
  dependable?
• Payments system who bears the cost of fraud?
• Smart grid meters report to power company, or
  government?
• With many players, you need an equilibrium
  arising out of players incentives
• Approaches include security economics, mechanism
  design,

4
Example Facebook

• Clear conflict of interest
• Facebook wants to sell user data
• Users want feeling of intimacy, small group,
  social control
• Complex access controls 60 settings on 7 pages
• Privacy almost never salient (why?)
• Over 90 of users never change defaults
• This lets Facebook blame the customer when things
  go wrong

5
Privacy

• Most people say they value privacy, but act
  otherwise. Most privacy ventures failed
• Why this privacy gap?
• Odlyzko technology makes price discrimination
  both easier and more attractive
• Acquisti people care about privacy when buying
  clothes, but not cameras
• Loewenstein privacy salience. Do stable privacy
  preferences even exist at all?

6
Social Engineering

• Use a plausible story, or just bully the target
• Whats your PIN so I can cancel your card?
• NYHA case
• Patricia Dunn case
• Kevin Mitnick Art of Deception
• Traditional responses
• mandatory access control
• operational security

7
Social Engineering (2)

• Social psychology
• Solomon Asch, 1951 two-thirds of subjects would
  deny obvious facts to conform to group
• Stanley Milgram, 1964 a similar number will
  administer torture if instructed by an authority
  figure
• Philip Zimbardo, 1971 you dont need authority
  the subjects situation / context is enough
• The Officer Scott case
• And what about users you cant train (customers)?

8
Usability and Psychology

• Why Johnny Cant Encrypt study of encryption
  program PGP showed that 90 of users couldnt
  get it right give 90 minutes
• Private / public, encryption / signing keys, plus
  trust labels was too much people would delete
  private keys, or publish them, or whatever
• Our 1998 study of password advice mnemonics
  best, compliance still patchy
• Security is hard unmotivated users, abstract
  security policies, lack of feedback

9
Phishing

• Started in 2003 with six reported (there had been
  isolated earlier attacks on AOL passwords)
• By 2006, UK banks lost 35m (33m by one bank)
  and US banks maybe 200m
• Early phish crude and greedy but phishermen
  learned fast
• E.g. Thank you for adding a new email address to
  your PayPal account
• The banks make it easy for them e.g. Halifax

10
Phishing (2)

• Banks pay firms to take down phishing sites
• A couple have moved to two-factor authentication
  (CAP) has its own problems
• At present, the phished banks are those with poor
  back-end controls and slow asset recovery
• One gang (Rockphish) is doing half to two-thirds
  of the business
• Mule recruitment seems to be a serious bottleneck

11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Fraud and Phishing Patterns

• Fraudsters do pretty well everything normal
  marketers do
• The IT industry has abandoned manuals people
  learn by doing, and marketers train them in
  unsafe behaviour (click on links)
• Banks approach is blame and train long known
  to not work in safety critical systems
• Their instructions look for the lock, click on
  images not URLs, parse the URL are easily
  turned round, and discriminate against nongeeks

15
(No Transcript)
16
Results

• Ability to detect phishing is correlated with
  SQ-EQ
• It is (independently) correlated with gender
• So the gender HCI issue applies to security too

17
Marketing Psychology

• See, for example, Cialdinis Influence Science
  and Practice
• People make buying decisions with the emotions
  and rationalise afterwards
• Mostly were too busy to research each purchase
  and in the ancestral evolutionary environment we
  had to make flight-or-fight decisions quickly
• The older parts of the brain kept us alive for
  millions of years before we became sentient
• We still use them more than we care to admit!

18
Marketing Psychology (2)

• Mental shortcuts include quality price and
  quality scarcity
• Reciprocation can be used to draw people in
• Then get a commitment and follow through
• Cognitive dissonance people want to be
  consistent (or at least to think that they are)
• Social proof like to do what others do
• People also like to defer to authority
• They want to deal with people they can relate to

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Prospect theory

• Kahneman Tversky, 1970s people value gains and
  losses differently
• Evolutionary logic of risk aversion, status quo
  bias
• Can drive fear marketing, savings, and (some of
  the) irrational behaviour of financial markets

26
Context and Framing

• Framing effects include Was 8.99 now 6.99 and
  the estate agent who shows you a crummy house
  first
• Take along an ugly friend on a double date
• Typical phishing attack user is fixated on task
  completion (e.g. finding why new payee on PayPal
  account)
• Advance fee frauds take this to extreme lengths!
• Risk salience is hugely dependent on context!
  E.g. CMU experiment on privacy

27
Risk Misperception

• Terrorist tactics have evolved over centuries to
  exploit our mental heuristics and biases
• Risk aversion we are oversensitive to
  low-probability, highly-damaging events
• Loewnstein ODonoghue Animal Spirits model
  our objective function by U h(w)M, where U is
  rational utility from deliberative system and M
  is from affective system
• U does Bayesian probability, M just does
  averages, w is willpower
• Explains other stuff (e.g. hyperbolic discounting)

28
Risk Misperception (2)

• Loewenstein-ODonoghue model may give
  quantitative insight into Availability
  heuristic easily-recalled data used to frame
  assessments
• Add extra credence given to images
• Also our behaviour evolved in small social
  groups, and we react against the out-group
• We are also sensitive to agency, and in
  particular to hostile intentions

29
Risk Misperception (3)

• Mortality salience greatly amplifies all this
• Pyszczynski and colleagues the experiment with
  the Tucson judges
• And its not just condemnation of the wicked
• Even taking one group past a graveyard is enough
  of a memento mori
• So what chance has cyber-terrorism got?

30
So What about Terrorism?

• People learn! the lesson from auctions UK/USA
• Politicians learn too! Mueller on attitudes of
  different US presidents, at the time and later
• But whats next will it get ever sneakier and
  nastier, just as marketing does?
• Muellers stats Collier on greed and grievance
• Limits on asymmetry? Network effects? What else?
• How would a capable green terror group operate?