The Economics and Psychology of Security

1
The Economics and Psychology of Security

• Ross Anderson
• Cambridge University

2
Social Science and Security

• The link between economics and security atrophied
  after WW2
• Since 2000, we have started to apply economic
  analysis to IT security and dependability
• Economic analysis often explains failure better
  then technical analysis!
• Infosec mechanisms are used increasingly to
  support business models (DRM, accessory control)
  rather than to manage risk
• Economic analysis is also vital for the public
  policy aspects of security
• Sociology and psychology are now engaged too

3
Traditional View of Infosec

• People used to think that the Internet was
  insecure because of lack of features crypto,
  authentication, filtering
• So engineers worked on providing better, cheaper
  security features AES, PKI, firewalls
• About 1999, we started to realize that this is
  not enough

4
Incentives and Infosec

• Electronic banking UK banks were less liable for
  fraud, so ended up suffering more internal fraud
  and more errors
• Distributed denial of service viruses now dont
  attack the infected machine so much as using it
  to attack others
• Health records hospitals, not patients, buy IT
  systems, so they protect hospitals interests
  rather than patient privacy
• Why is Microsoft software so insecure, despite
  market dominance?

5
New View of Infosec

• Systems are often insecure because the people who
  guard them, or who could fix them, have
  insufficient incentives
• Bank customers suffer when poorly-designed bank
  systems make fraud and phishing easier
• Patients suffer when hospital systems break
  privacy
• Casino websites suffer when infected PCs run DDoS
  attacks on them
• Insecurity is often what economists call an
  externality a side-effect, like environmental
  pollution

6
New Uses of Infosec

• Xerox started using authentication in ink
  cartridges to tie them to the printer and its
  competitors soon followed
• Motorola then started authenticating mobile phone
  batteries to the phone
• Carmakers make chipping harder, and plan to
  authenticate major components
• DRM Apple grabs control of music download, MS
  trying to do the same for HD video content

7
IT Economics (1)

• The first distinguishing characteristic of many
  IT product and service markets is network effects
• Metcalfes law the value of a network is the
  square of the number of users
• Real networks phones, fax, email
• Virtual networks PC architecture versus MAC, or
  Symbian versus WinCE
• Network effects tend to lead to dominant firm
  markets where the winner takes all

8
IT Economics (2)

• Second common feature of IT product and service
  markets is high fixed costs and low marginal
  costs
• Competition can drive down prices to marginal
  cost of production
• This can make it hard to recover capital
  investment, unless stopped by patent, brand,
  compatibility
• These effects can also lead to dominant-firm
  market structures

9
IT Economics (3)

• Third common feature of IT markets is that
  switching from one product or service to another
  is expensive
• E.g. switching from Windows to Linux means
  retraining staff, rewriting apps
• Shapiro-Varian theorem the net present value of
  a software company is the total switching costs
• This is why so much effort goes into managing
  switching costs once you have 3000 worth of
  songs on a 300 iPod, youre locked into iPods

10
IT Economics and Security

• High fixed/low marginal costs, network effects
  and switching costs all tend to lead to
  dominant-firm markets with big first-mover
  advantage
• So time-to-market is critical
• Microsoft philosophy of well ship it Tuesday
  and get it right by version 3 is not perverse
  behaviour by Bill Gates but quite rational
• Whichever company had won in the PC OS business
  would have done the same

11
IT Economics and Security (2)

• When building a network monopoly, you must appeal
  to vendors of complementary products
• Thats application software developers in the
  case of PC versus Apple, or now of Symbian versus
  WinCE, or WinMP versus Real
• Lack of security in earlier versions of Windows
  made it easier to develop applications
• So did the choice of security technologies that
  dump most costs on the user (SSL, PKI, )
• Once youre a monopolist, lock it all down!

12
Why are so many security products ineffective?

• Akerlofs Nobel-prizewinning paper, The Market
  for Lemons introduced asymmetric information
• Suppose a town has 100 used cars for sale 50
  good ones worth 2000 and 50 lemons worth 1000
• What is the equilibrium price of used cars in
  this town?
• If 1500, no good cars will be offered for sale

13
Security and Liability

• Why did digital signatures not take off?
• Industry thought legal uncertainty. So EU passed
  electronic signature law
• But customers and merchants resist transfer of
  liability by bankers for disputed transactions
• If youre a customer, best stick with credit
  cards, so fraud remains largely the banks problem

14
Privacy

• Most people say they value privacy, but act
  otherwise. Most privacy technology firms failed
• Acquisti people care about privacy when buying
  clothes, but not cameras (data relating to body
  or image are more privacy sensitive)
• Issue for mobile phone industry phone viruses
  worse for image than PC viruses
• Varian you can maybe fix privacy by giving
  people property rights in personal information
• Odlyzko technology makes price discrimination
  both easier and more attractive

15
Why Bill wasnt interested in security

• While Microsoft was growing, the two critical
  factors were speed, and appeal to application
  developers
• Security markets were over-hyped and driven by
  artificial factors
• Issues like privacy and liability were more
  complex than they seemed
• The public couldnt tell good security from bad
  anyway

16
Why is Bill now changing his mind?

• Security can help lock customers in, and extend
  power from one market to another
• Information Rights Management changes ownership
  of a file from the machine owner to the file
  creator
• Remember value of software company total
  switching costs. And once documents cant be
  converted without creators permission, the
  switching cost is much higher
• And will WMP/Vista let Microsoft do to high
  definition movies what Apple did for music?

17
Open versus Closed?

• Are open-source systems more dependable? Its
  easier for the attackers to find vulnerabilities,
  but also easier for the defenders to find and fix
  them
• Theory openness helps both equally if bugs are
  random and standard dependability model
  assumptions apply
• Statistics bugs are correlated in a number of
  real systems (Milk or Wine?)
• Trade-off the gains from this, versus the risks
  to systems whose owners dont patch

18
How Much to Spend?

• How much should the average company spend on
  information security?
• Governments, vendors say much much more than at
  present!
• But theyve been saying this for 20 years!
• Measurements of security return-on-investment
  suggest about 20 p.a. overall
• So the total expenditure may be about right

19
Skewed Incentives

• Why do large companies spend too much on security
  and small companies too little?
• Research shows theres an adverse selection
  effect
• Corporate security managers tend to be
  risk-averse people, often from accounting /
  finance
• More risk-loving people may become sales or
  engineering staff, or small-firm entrepreneurs
• Theres also due-diligence, government
  regulation, and insurance to think of

20
Skewed Incentives (2)

• If you are DirNSA and have a nice new hack on XP
  and Vista, do you tell Bill?
• Tell protect 300m Americans
• Dont tell be able to hack 400m Europeans,
  1000m Chinese,
• If the Chinese hack US systems, they keep quiet.
  If you hack their systems, you can brag about it
  to the President
• So offence can be favoured over defence

21
Large Project Failure

• Maybe 30 of large projects fail
• But we build much bigger failures nowadays than
  30 years ago so
• Why do more public-sector projects fail?
• Consider what the incentives are on project
  managers versus ministers and what sort of
  people will become successful project managers
  versus ministers!

22
Security and Sociology

• Theres a lot of interest recently in using
  social networks to analyse interactions and
  systems
• Barabási and Albert showed that a scale-free
  network could be attacked efficiently by
  targeting its high-order nodes
• Think rulers target Saxon landlords / Ukrainian
  kulaks / Tutsi schoolteachers /
• Can we use evolutionary game theory ideas to
  figure out how networks evolve?
• Idea run many simulations between different
  attack / defence strategies

23
Security and Sociology (2)

• Vertex-order attacks with
• Black normal (scale-free) node replenishment
• Green defenders replace high-order nodes with
  rings
• Cyan they use cliques (c.f. system biology )

24
Psychology and Security

• Fastest growing online crime is phishing it
  only started in 2004, but by 2006 it cost the UK
  35m and the USA perhaps 200m
• Pretexting always existed (see Mitnicks book),
  but phishing industrializes it
• In a company you can train the staff in
  operational security (though many dont). Its
  harder when the target is your users!
• Maybe more secure machines would inevitably drive
  the bad guys to target the people instead
• What can security folks learn from psychology?

25
Psychology and Security (2)

• Security usability research is fairly new and the
  results are pessimistic most security products
  dont work well or at all
• Over half of all SSL certificates are wrong
• No problem we train people to keep on clicking
  OK until they can get their work done
• Banks react to phishing by blame and train
  efforts towards customers but we know from the
  safety-critical world that this doesnt work
• Systems designed by geeks discriminate against
  women, the elderly and the less educated

26
Psychology and Security (3)

• Social psychology has long been relevant to us!
• Solomon Asch showed most people would deny the
  evidence of their eyes to conform to a group
• Stanley Milgram showed that 60 of people will do
  downright immoral things if ordered to
• Philip Zimbardos Stanford Prisoner Experiment
  showed roles and group dynamics were enough
• The disturbing case of Officer Scott
• How can systems resist abuse of authority?
• Why do people need enemies?
• Why does terrorism work?

27
Psychology and Security (4)

• Evolutionary psychology may eventually explain
  cognitive biases. It is based on the massive
  modularity hypothesis and the use of FMRI to
  track brain function
• Simon Baron-Cohens work on autism suggests a
  theory of mind module central to empathy for
  others mental states
• This is how we differ from the great apes
• It helps us lie, and to detect lies told by
  others
• So are we really homo sapiens sapiens or homo
  sapiens deceptor?

28
The Information Society

• More and more goods contain software
• More and more industries are starting to become
  like the software industry
• The good flexibility, rapid response
• The bad frustration, poor service
• The ugly monopolies
• How will society evolve to cope?

29
The Research Agenda

• We need to figure out how to balance competing
  social goals, as we have in the physical world
• Security economics gives us tools to understand
  whats going on and to analyse policy options
• Sociology also gives some useful insights
• And security psychology is not just a side
  discipline relevant to usability and phishing
  it has the potential to bring us fundamental
  insights, just as security economics has

30
More

• Economics and Security Resource Page
  www.cl.cam.ac.uk/rja14/econsec.html (or follow
  link from www.ross-anderson.com)
• WEIS Annual Workshop on Economics and
  Information Security next at CMU, June 78 2006
• Foundation for Information Policy Research
  www.fipr.org