The Relationship Between Sampling and Counting, and Simulated Annealing

1
The Relationship Between Sampling and
Counting,and Simulated Annealing

• Anat Halpern

2
The Relationship Between Sampling and Counting

• Lecture notes Eric Vigoda
• Lecture notes Alistair Sinclair

3
Fully Polynomial Randomized Approximation Scheme
(FPRAS)

• In time polynomial in x, 1/e, log(1/d)

4
Fully Polynomial Almost Uniform Sampler (FPAUS)

• In time polynomial in x, log(1/d)

5
Self-reducibility - Intuition
6
Self-reducibility

• We define a language L to be
• self-reducible if there exists an oracle
  polynomial time TM M that on inputs of length n
  only asks its oracle questions of length less
  than n and that accepts a string w iff w ? L.
• (Dan Spielman)

7
Counting-Sampling Equivalences
8
Running Example - Matching

• A matching on a graph is a set of edges of such
  that no two of them share a vertex in common.

9
Exact Counter ? Exact Sampler

• Lemma (Vigoda)
• Given an algorithm which exactly computes the
  number of matchings of an arbitrary graph G
  (V,E) in time polynomial in V, we can then
  construct an algorithm which outputs a
  (uniformly) random matching of an arbitrary graph
  G (V,E) in time polynomial in V.

10
Exact Counter ? Exact Sampler

• Choose an arbitrary edge e.
• Let G1(V,E\e)
• Let G2induced graph on V\u,v
• For a matching M of G, either M is a matching of
  G1, or M\e is a matching of G2.

e(u,v)
G1
G2
11
Exact Counter ? Exact Sampler

• M(G) set of matchings of G.
• M(G) M(G1) M(G2)
• Sampling constructing R recursively using Pr

12
Generating a Sampling - example

• Choose an arbitrary edge e.
• Let G1(V,E\e)
• Let G2induced graph on V\u,v

• Calculate Pr, given M(G1), M(G2)
• Choose e with probability Pr.
• Continue with GG2

e(u,v)
And so on
G1
G2
13
Exact Sampler ? FPRAS Counter

• Lemma (Vigoda)
• Given an algorithm which for an arbitrary graph
• G (V,E) generates a random matching in time
  polynomial in V, then we can construct an FPRAS
  for estimating M(G).

14
Simulated Annealing

• Write the desired quantity as a telescoping
  product
• Where
• is trivial to compute
• Each ratio is bounded small number of samples
  is needed

15
Exact Sampler ? FPRAS Counter

• Arbitrarily order the edges

16
Exact Sampler ? FPRAS Counter

• Boundaries on pi
• ? very few samples needed to estimate pi

17
Exact Sampler ? FPRAS Counter

• Boundaries on pi explanation

18
Exact Sampler ? FPRAS Counter

• Counting
• Draw s random samples from
• Count the samples that are also in
• Estimate pi
• Result

19
Counting-Sampling In General

• Self-reducibility tree
• Example SAT
• FPAUS Sampler ? FPRAS Counter

20
Accelerating Simulated Annealing for the
Permanent and Combinatorial Counting Problems

• Bezáková, tefankovic, Vazirani, Vigoda

21
Accelerating Simulated Annealing

• The problem
• FPAUS Sampler ? FPRAS Counter
• Running example - graph coloring
• O?O(G) set of all k-colorings of G
• Input parameters e,d

22
Traditional reduction
23
Continuous version

• Additional parameters
• activity ?
• Configuration (coloring) - s
• No. of illegal (monochromatic) edges M(s)
• configuration space -
• Motivation fewer random-sampling problems
• instead of m

24
Partition Function

• Intuition counting weighted configurations

25
Partition Function cont.

• Z - polynomial in ?
• Constant coefficient O
• For kgt?,
• ? For
• Allows to obtain an efficient estimator for O

26
Partition Function cont.

• Proof

27
Counting Colorings

• Activity sequence
• Boundaries
• Number of k-colorings

28
Former ApproachUniform Cooling Schedule

• Z polynomial of degree m
• Sufficient to define
• Rate of decrease

29
Improved ApproachAccelerating Schedule

• Decrease rate depends on Z
• Recall
• Constant coefficient
• Intuition

30
Accelerating Schedule

• As ? decreases, the rate of Z will be bounded by
  the rate of polynomials
• When the polynomial dominates, we can
  decrease ? by a factor of

31
Applying Improved Schedule

• Approximate
• Distribution
• Random labeling
• Unbiased estimator

32
Unbiased estimator

• Proof

33
Counting at last

• Given an algorithm for generating colorings
• such that
• Draw samples of