Information-Theoretic Approaches to Branching in Search

1
Information-Theoretic Approaches to Branching in
Search

• Andrew Gilpin and Tuomas Sandholm
• Carnegie Mellon University
• Computer Science Department

2
Introduction

• Deciding which question to branch on is a key
  element of search algorithms
• We present four families of branching strategies
• Each shows promising results over existing
  methods
• Each technique is information-theoretically
  motivated
• Start of search Most uncertainty
• End of search Zero uncertainty

3
Quantifying uncertainty in 0-1 MIP

• Main idea treat relaxed variables as independent
  probabilities
• Entropy is used to measure amount of uncertainty
  in a variable e(x) -x log2 x (1-x) log2
  (1-x)
• Entropy is additive for independent variables

4
Entropic lookahead for variable selection

• Entropic Branching (EB)
• Let x be the LP solution at the current node
• For each fractional xi
• Let xl be solution of LP with xil 0
• Let xu be solution of LP with xiu 1
• entropyi (1-xi) entropy(xl) xi
  entropy(xu)
• Return argmini entropyi
• (This algorithm generalizes to general integer
  programs)

5
Entropic lookahead for variable selection

• Entropic branching is similar to strong branching
• Require same amount of computation
• Can be combined in hybrid heuristics
• Entropic branching can be generalized to
• multi-variable branches
• more than one-step lookahead

6
Experimental results for lookahead-based
branching strategies

• On MIPLIB, entropic and strong branching are
  dominated by branching on most-fractional
  variable
• Entropic and strong branching perform comparably
• On real-world procurement optimization problems
• Strong branching outperforms most-fractional
  variable branching by 27
• Entropic branching outperforms strong branching
  by 29.5
• Thus, entropic branching performs as well or
  better than strong branching
• Even though entropic branching ignores objective
  info

7
Combinatorial procurement auction with max
winners constraint

• Suppliers submit bids on bundles of items
• Buyer specifies a maximum number of winning
  suppliers
• Buyer wishes to choose an allocation such that
  cost is minimized and the max winners constraint
  is satisfied
• This problem is NP-complete (even if bids are on
  single items only)
• The max winners constraint is the main driver of
  hardness, and this class of problems has been
  observed to be difficult in practice
• The MIP formulation has a binary variable for
  each bid, and a binary indicator variable for
  each supplier

8
Lookahead-free branching strategy for procurement
optimization

• Indicator entropic branching (IEB)
• Let yj be the value of supplier js indicator
  variable in the current nodes LP solution
• For each j where yj is fractional
• Let entropyj be the sum of the entropies of
  supplier js bids
• Branch on yk, where k argminj entropyj

Suppliers Bids Items Max CPLEX IEB
20 18000 100 5 25.63 11.81
30 60750 225 8 5755.92 551.82
40 144000 400 10 37.05 30.57
9
Multi-variable branching

• We can generalize entropic branching to branching
  on sums of variables
• Given a set X x1,, xn, let k floor(x1
  xn)
• We can use the following branches
• x1 xn k and x1 xn k
  1
• For each branch, we compute entropy using
  lookahead to select the least uncertain branch
• Prop. Using multi-variable branches does not
  increase the search space
• Experimental results indicate that the search
  tree is reduced, but an efficient heuristic for
  selecting the candidate set X is required

10
Conclusion

• Introduced new paradigm for branch selection
  based on an information-theoretic approach
• Developed four families of search strategies
• Lookahead entropic branching
• Hybrid entropic and strong branching
• Lookahead-free entropic branching (for problems
  with indicator variables)
• Multi-variable entropic branching
• Experimental results show significant improvement
  over existing branching strategies