Adversaries and Information Leaks

1
Adversaries and Information Leaks

• Geoffrey Smith
• Florida International University
• November 7, 2007

TGC 2007 Workshop on the Interplay of Programming
Languages and Cryptography
2
Motivation

• Suppose that a program c processes some sensitive
  information.
• How do we know that c will not leak the
  information, either accidentally or maliciously?
• How can we ensure that c is trustworthy ?
• Secure information flow analysis Do a static
  analysis (e.g. type checking) on c prior to
  executing it.

3
Two Adversaries

• The program c
• Has direct access to the sensitive information (H
  variables)
• Behavior is constrained by the static analysis
• The observer O of cs public output
• Has direct access only to cs public output
  (final values of L variables, etc.)
• Behavior is unconstrained (except for
  computational resource bounds)

4
The Denning Restrictions

• Classification An expression is H if it contains
  any H variables otherwise it is L.
• Explicit flows A H expression cannot be assigned
  to a L variable.
• Implicit flows A guarded command with a H guard
  cannot assign to L variables.

if ((secret 2) 0) leak 0 else
leak 1
H guard
assignment to L variable
5
Noninterference

• If c satisfies the Denning Restrictions, then
  (assuming c always terminates) the final values
  of L variables are independent of the initial
  values of H variables.
• So observer O can deduce nothing about the
  initial values of H variables.
• Major practical challenge How can we relax
  noninterference to allow small information
  leaks, while still preserving security?

6
Talk Outline

• Secure information flow for a language with
  encryption FMSE06
• Termination leaks in probabilistic programs
  PLAS07
• Foundations for quantitative information flow
• Joint work with Rafael Alpízar.

7
I. Secure information flow for a language with
encryption

• Suppose that E and D denote encryption and
  decryption with a suitably-chosen shared key K.
• Programs can call E and D but cannot access K
  directly.
• Intuitively, we want the following rules
• If e is H, then E(e) is L.
• If e is either L or H, then D(e) is H.
• But is this sound ? Note that it violates
  noninterference, since E(e) depends on e.

8
It is unsound if encryption is deterministic!

• Assume secret H, mask L, leak L.
• leak 0
• mask 2n-1
• while mask ? 0 do (
• if E(secret mask) E(secret) then
• leak leak mask
• mask mask gtgt 1
• )

9
Symmetric Encryption Schemes

• SE with security parameter k is a triple (K,E,D)
  where
• K is a randomized key-generation algorithm
• We write K lt- K.
• E is a randomized encryption algorithm
• We write C lt- EK(M).
• D is a deterministic decryption algorithm
• We write M DK(C).

10
IND-CPA Security
M1
M2
b?
EK(LR(,,b))
M1 and M2 must have equal length.
A
C

• The box contains key K and selection bit b.
• If b0, the left strings are encrypted.
• If b1, the right strings are encrypted.
• The adversary A wants to guess b.

11
IND-CPA advantage

• Experiment Expind-cpa-b(A)
• K lt- K
• d lt- AEK(LR(,,b))
• return d
• Advind-cpa(A) PrExpind-cpa-1(A) 1
  PrExpind-cpa-0(A) 1.
• SE is IND-CPA secure if no polynomial-time
  adversary A can achieve a non-negligible IND-CPA
  advantage.

12
Our programming language

• e x n e1e2 D(e1,e2)
• c x e
• x lt- R
• (x,y) lt- E(e)
• skip
• if e then c1 else c2
• while e do c
• c1c2
• Note n-bit values, 2n-bit ciphertexts.

random assignment according to distribution R
13
Our type system

• Each variable is classified as H or L.
• We just enforce the Denning restrictions, with
  modifications for the new constructs.
• E(e) is L, even if e is H.
• D(e1,e2) is H, even if e1 and e2 are L.
• R (a random value) is L.

14
Leaking adversary B

• B has a H variable h and a L variable l, and
  other variables typed arbitrarily.
• h is initialized to 0 or 1, each with probability
  ½.
• B can call E() and D().
• B tries to copy the initial value of h into l.

15
Leaking advantage of B

• Experiment Expleak(B)
• K lt- K
• h0 lt- 0,1
• h h0
• initialize other variables to 0
• run BEK(),DK()
• if l h0 then return 1 else return 0
• Advleak(B) 2 PrExpleak(B) 1 - 1

16
Soundness via Reduction

• For now, drop D() from the language.
• Theorem Given a well-typed leaking adversary B
  that runs in time p(k), there exists an IND-CPA
  adversary A that runs in time O(p(k)) and such
  that
• Advind-cpa(A) ½Advleak(B).
• Corollary If SE is IND-CPA secure, then no
  polynomial-time, well-typed leaking adversary B
  achieves non-negligible advantage.

17
Proof of Theorem

• Given B, construct A that runs B with a
  randomly-chosen 1-bit value of h.
• Whenever B calls E(e), A passes (0n, e) to its
  oracle EK(LR(,,b)).
• So if b 1, E(e) returns EK(e).
• And if b 0, E(e) returns EK(0n), a random
  number that has nothing to do with e!
• If B terminates within p(k) steps and succeeds in
  leaking h to l, then A guesses 1.
• Otherwise A guesses 0.

18
What is As IND-CPA advantage?

• If b 1, B is run faithfully.
• Hence
• PrExpind-cpa-1(A) 1
• PrExpleak(B) 1
• ½Advleak(B) ½
• If b 0, B is not run faithfully.
• Here B is just a well-typed program with random
  assignment but no encryption.

19
More on the b 0 case

• In this case, we expect the type system to
  prevent B from leaking h to l.
• However, when B is run unfaithfully, it might
  fail to terminate!
• Some careful analysis is needed to deal with this
  possibility Part II of talk!
• But in the end we get
• PrExpind-cpa-0(A) 1 ½
• So Advind-cpa(A) ½Advleak(B), as claimed. ?

20
Can we get a result more like noninterference?

• Let c be a well-typed, polynomial-time program.
• Let memories µ and ? agree on L variables.
• Run c under either µ or ?, each with probability
  ½.
• A noninterference adversary O is given the final
  values of the L variables of c.
• O tries to guess which initial memory was used.

21
A computational noninterference result

• Theorem No polynomial-time adversary O (for c,
  µ, and ?) can achieve a non-negligible
  noninterference advantage.
• Proof idea Given O, we can construct a
  well-typed leaking adversary B.
• Note that O cannot be assumed to be well typed!
• But because O sees only the L variables of c, it
  is automatically well typed under our typing
  rules.

22
Construction of B

• initialize L variables of c according to µ and ?
• if h 0 then
• initialize H variables of c according to µ
• else
• initialize H variables of c according to ?
• c
• O // O puts its guess into g
• l g

23
Related work

• Laud and Vene FCT 2005.
• Work on cryptographic protocols Backes and
  Pfitzmann Oakland 2005, Laud CCS 2005,
• Askarov, Hedin, Sabelfeld SAS06
• Laud POPL08
• Vaughan and Zdancewic Oakland07

24
II. Termination leaks in probabilistic programs

• In Part I, we assumed that all adversaries run in
  time polynomial in k, the key size.
• This might seem to be without loss of
  generality (practically speaking) since
  otherwise the adversary takes too long.
• But what if program c either terminates quickly
  or else goes into an infinite loop?
• In that case, observer O might quickly be able to
  observe whether c terminates.

25
The Denning Restrictions and Nontermination

• The Denning Restrictions allow H variables to
  affect nontermination
• If c satisfies the Denning Restrictions, then
  (assuming c always terminates) the final values
  of L variables are independent of the initial
  values of H variables.
• Can we quantify such termination leaks?

while secret 0 do skip leak 1
26
Probabilistic Noninterference

• Consider probabilistic programs with random
  assignment but no encryption.
• Such programs are modeled as Markov chains of
  configurations (c,µ).
• And noninterference becomes
• The final probability distribution on L variables
  is independent of the initial values of H
  variables.

27
A program that violates probabilistic
noninterference

• If h 0, terminates with l 0 with probability
  ½ and loops with probability ½.
• If h 1, terminates with l 1 with probability
  ½ and loops with probability ½.

t lt- 0,1 if t 0 then while h 1 do
skip l 0 else while h 0 do skip
l 1
randomly assign 0 or 1 to t
t L h H l L
28
Approximate probabilistic noninterference

• Theorem If c satisfies the Denning restrictions
  and loops with probability at most p, then cs
  deviation from probabilistic noninterference is
  at most 2p.
• In our example, p ½, and the deviation is ½
  - 0 0 ½ 1 2p.

probability that l 1 when h 0 and when h 1
probability that l 0 when h 0 and when h 1
29
Stripped program ?c?

• Replace all subcommands that dont assign to L
  variables with skip.
• Note ?c? contains no H variables.

t lt- 0,1 if t 0 then while h 1 do
skip l 0 else while h 0 do skip
l 1
t lt- 0,1 if t 0 then skip l
0 else skip l 1
30
The Bucket Property
cs buckets
loop
l 0
l 1
l 2
Pour water from loop bucket into other buckets.
?c?s buckets
loop
l 0
l 1
l 2
31
Proof technique

• In prior work on secure information flow,
  probabilistic bisimulation has often been useful.
• Here we use a probabilistic simulation Jonsson
  and Larson 1991 instead.
• We define a modification of the weak simulation
  considered by Baier, Katoen, Hermanns, Wolf
  2005.

32
Fast Simulation on a Markov chain (S,P)

• A fast simulation R is a binary relation on S
  such that if s1 R s2 then the states reachable in
  one step from s1 can be partitioned into U and V
  such that
• v R s2 for every v ? V
• letting K Su?UP(s1,u), if K gt 0 then there
  exists a function ? S x S -gt 0,1 with
• ?(u,w) gt 0 implies that u R w
• P(s1,u)/K Sw?S?(u,w) for all u ? U
• P(s2,w) Su?U?(u,w) for all w ? S.

33
Proving the Bucket Property

• Given R, a set T is upwards closed if s?T and
  sRs implies s?T.
• Pr(s,n,T) is the probability of going from s to a
  state in T in at most n steps.
• Theorem If R is a fast simulation, T is upwards
  closed, and s1 R s2, then Pr(s1,n,T)
  Pr(s2,n,T) for every n.
• We can define a fast simulation RL such that
  (c,µ) RL (?c?,µ), for any well-typed c.

34
Approximate noninterference
(c,µ)
at most p
loop
l 0
l 1
l 2
(?c?,µ) (?c?,?)
µ and ? agree on L variables
loop
l 0
l 1
l 2
(c,?)
at most p
loop
l 0
l 1
l 2
35
Remarks

• Observer Os ability to distinguish µ and ? by
  statistical hypothesis testing could be bounded
  as in Di Pierro, Hankin, Wiklicky 2002.
• The Bucket Property is also crucial to the
  soundness proof of the type system considered in
  Part I of this talk.

36
III. Foundations for quantitative information flow

• To allow small information leaks, we need a
  quantitative theory of information.
• Quite a lot of recent work
• Clark, Hunt, Malacaria 2002, 2005, 2007
• Köpf and Basin CCS07
• Clarkson, Myers, Schneider CSFW05
• Lowe CSFW02
• Di Pierro, Hankin, Wiklicky CSFW02

37
Research Steps

• Define a quantitative notion of information flow.
• Show that the notion gives appropriate security
  guarantees.
• Devise static analyses to enforce a given
  quantitative flow policy.
• Prove the soundness of the analyses.
• Here well discuss only steps 1 and 2.

38
Our Conceptual Framework

• Rather than trying to tackle the general problem,
  lets consider important special cases to better
  see whats going on.
• Assume that secret h is chosen from some space S
  with some a priori distribution.
• Assume that c is a program that has only h as
  input and (maybe) leaks information about h to
  its unique public output l.
• Assume that c is deterministic and total.

39
Then Köpf Basin 07

• There exists a function f such that l f(h).
• f induces an equivalence relation on S.
• h1 h2 iff f(h1) f(h2)
• So c partitions S into equivalence classes

f-1(l3)
f-1(l1)
f-1(l2)
40
What is leaked?

• The observer O sees the final value of l.
• This tells O which equivalence class h belonged
  to.
• How bad is that?
• Extreme 1 If f is a constant function, then
  there is just one equivalence class, and
  noninterference holds.
• Extreme 2 If f is one-to-one, then the
  equivalence classes are singletons, and we have
  total leakage of h (in principle).

41
Quantitative Measures

• Consider a discrete random variable X whose
  values have probabilities p1, p2, p3,
• Assume pi pi1
• Shannon Entropy H(X) S pi log (1/pi)
• uncertainty about X
• expected number of bits to transmit X
• Guessing Entropy G(X) S i pi
• expected number of guesses to guess X

42
Shannon Entropy applied to the partitions induced
by c

• Lets assume that
• S n and h is uniformly distributed
• The partition has r equivalence classes C1,
  C2, , Cr and Ci ni
• H(h) log n
• initial uncertainty about h
• H(l) ? (ni/n) log (n/ni)
• Plausibly, amount of information leaked
• Extreme 1 H(l) 0
• Extreme 2 H(l) log n

43
How much uncertainty about h remains after the
attack?

• This can be calculated as a conditional Shannon
  entropy
• H(hl) ? (ni/n) H(Ci) ? (ni/n) log ni
• Extreme 1 H(hl) log n
• Extreme 2 H(hl) 0
• There is a pretty equation relating these!
• H(h) H(l) H(hl)

initial uncertainty
information leaked
remaining uncertainty
44
So is Step 1 finished?

• 1. Define a quantitative notion of information
  flow
• In the special case that we are considering, it
  seems promising to define the amount of
  information leaked to be H(l), and the remaining
  uncertainty to be H(hl).
• This seems to be the literature consensus
• Clarke, Hunt, Malacaria
• Köpf and Basin also use G(l) and G(hl)
• Clarkson, Myers, Schneider (?) Sec. 4.4, when
  the attackers belief matches the a priori
  distribution

45
What about Step 2?

• 2. Show that the notion gives appropriate
  security guarantees.
• Leakage 0 iff noninterference holds
• Good, but this establishes only that the
  zero/nonzero distinction is meaningful!
• More interesting is the Fano Inequality.
• But this gives extremely weak bounds.
• Does the value of H(hl) accurately reflect the
  threat to h?

46
An Attack

• Copy 1/10 of the bits of h into l.
• l h 0177777
• Gives 2.1 log n n.1 equivalence classes, each
  of size 2.9 log n n.9.
• H(l) .1 log n
• H(hl) .9 log n
• After this attack, 9/10 of the bits are
  completely unknown.

47
Another Attack

• Put 90 of the possible values of h into one big
  equivalence class, and put each of the remaining
  10 into singleton classes
• if (h lt n/10) l h else l -1
• H(l) .9 log (1/.9) .1 log n .1 log n .14
• H(hl) .9 log (.9n) .9 log n .14
• Essentially the same as the previous attack!
• But now O can guess h with probability 1/10.

48
A New Measure

• With H(hl), we cant do Step 2 well!
• Why not use Step 2 to guide Step 1?
• Define V(hl), the vulnerability of h given l, to
  be the probability that O can guess h correctly
  in one try, given l.

49
Calculating V(hl) when h is uniformly distributed

• As before, assume that
• S n and h is uniformly distributed
• The partition has r equivalence classes C1,
  C2, , Cr and Ci ni
• V(hl) S (ni/n) (1/ni) r/n
• So all that matters is the number of equivalence
  classes, not their sizes!

r
i1
50
Examples

• Noninterference case r 1, V(hl) 1/n
• Total leakage case r n, V(hl) 1
• Copy 1/10 of bits r n.1, V(hl) 1/n.9
• Put 1/10 of hs values into singleton classes r
  1 n/10, V(hl) 1/10
• Put hs values into classes, each of size 10 r
  n/10, V(hl) 1/10
• Password checker r 2, V(hl) 2/n

51
Conclusion

• Maybe V(hl) is a better foundation for
  quantitative information flow.
• Using a single number is crude.
• Compare examples d, e.
• V(hl) is not so good with respect to
  compositionality.