Solve Your Problem Faster - by changing the model

1
Solve Your Problem Faster - by changing the model

• Barbara Smith

2
Background Assumptions

• A constraint programming tool providing
• a systematic search algorithm
• combined with constraint propagation
• a set of pre-defined constraints
• A problem that can be represented as a finite
  domain constraint satisfaction or optimization
  problem
• Caveat the experiences described are based on
  ILOG Solver

3
What do I mean by Faster?

• e.g.Peaceable Armies of Queens problem
• place m black m white queens on a chessboard so
  that the black queens dont attack the white
  queens, and maximize m
• Optimal solution for an 11 x
  11 board

4
Results for 8 x 8 board
Backtracks to find optimal Backtracks to prove optimality Time (sec.)
Basic model 5270 12,002,608 2100
Eliminate symmetry 5270 938,652 240
New model 4676 478,012 72
Variable order-ing heuristic 2973 44,276 12
Just changing the model makes a huge difference
to the time to solve the problem
5
What are the options?

• Given a CSP representation of the problem
• if there is symmetry in the CSP, eliminate it
• find a related representation to use instead/as
  well
• add variables to express different aspects of
  the problem
• add constraints
• to relate new variables to old
• to prune dead-ends earlier
• change the search strategy
• (other possibilities wont be considered)

6
Symmetry in the CSP

• A symmetry transforms any (full or partial)
  assignment into another so that
  consistency/inconsistency is preserved
• Symmetry causes wasted search effort after
  exploring choices that dont lead to a solution,
  symmetrically equivalent choices will be explored
• Especially difficult if there is no solution, or
  if all solutions are wanted, or in optimization
  problems

7
Symmetry Breaking During Search (SBDS)

• See Gent Smith, ECAI2000

g(var! val) for any unbroken symmetry g,
i.e. if g(A) is (or will be) true

On backtracking to a choice point, add symmetry
breaking constraints to the 2nd branch explored
8
Symmetries of armies of queens

• Variable si,j represents the square i, j
• The values are b, w or e (black, white or empty)
  
• 7 chess board symmetries x (horizontal
  reflection), y, d1, d2, r90, r180, r270 bw
  (swap black white queens) 7 combinations
• x(si,jv) ? sn-i1,jv
• bw(si,j v) ? si,j v , (v b if v w,
  w if v b, e if v e )
  
• bw_x(si,jv) ? sn-i1,jv , etc.
• 15 symmetry functions are needed in all
• each takes a constraint as input, e.g. si,jv,
  and returns the symmetric constraint, e.g. si,j
  v

9
Armies of Queens 8 x 8 Board
x if s8,1w then s8,2!w , etc.
10
Results for 8 x 8 board
Fails to find optimal Fails to prove optimality Time (sec.)
Basic model 5270 12,002,608 2100
Eliminate symmetry 5270 938,652 240
11
Optimizing SONET Rings

• See Sherali Smith, Improving Discrete Model
  Representations via Symmetry Considerations
  (no relation)
• Transmission over optical fibre networks
• Known traffic demands between pairs of client
  nodes
• A node is installed on a SONET ring using an ADM
  (add-drop multiplexer)
• If there is traffic demand between 2 nodes, there
  must be a ring that they are both on
• Rings have capacity limits (number of ADMs, i.e.
  nodes, traffic)
• Satisfy demands using the minimum number of ADMs

12
Example Optimal Solution

• Up to 7 rings, maximum capacity 5 ADMs
• Ignore traffic capacity (for now)

13
A CSP Model

• Variables xij 1 if node i is on ring j
• At most 5 nodes on any ring
  
• If there is a demand between nodes k and l
• Minimize

14
Symmetry in the SONET CSP

• The rings are indistinguishable (only numbered
  for the purposes of the CSP model)
• We can eliminate the symmetry using SBDS just by
  describing all transpositions of pairs of rings
• e.g. r12(xi,j v) ? xi,2v if j 1
  xi,1v if j 2 xi,jv otherwise
• Thats all we can forget about symmetry (e.g.
  when choosing the variable ordering)

15
Three Alternative Models

• Whether a given node is on a given ring
• xij 1 if node i is on ring j
• Which ring(s) each node is on
• Ni set of rings node i is on
• Which nodes are on each ring
• Rj set of nodes on ring j
• In principle, any of these 3 sets of variables
  could be the basis of a complete CSP model

16
Dual Models from Boolean Variables

• Given a CSP with Boolean variables xijkl we can
  form a new set of variables with one less
  subscript
• e.g. xijkl 1 corresponds to yjkl i (an
  integer variable) or i ? Yjkl (a set variable),
  depending on whether one or several possible
  values i are associated with each combination of
  j,k,l,
• if the Boolean variables have n subscripts, we
  can derive n sets of dual variables

17
Which Model to Choose?

• We dont need to choose just one set of variables
  we can use them all at once
• We then need new channelling constraints to link
  the sets of variables
• (xij 1) (i ? Rj) (j ? Ni)
• (see Cheng, Choi, Lee Wu, Constraints, 1999)
• But we should not combine all 3 complete models
• Adding variables channelling constraints
  doesnt cost much duplicated constraints do

18
Why add more variables?

• Express each problem constraint in whichever way
  is easiest /most natural/ propagates best
• e.g. gives better results than
• (Often easiest /most natural/ propagates best are
  the same thing)
• We can observe effects of search on different
  aspects of the model, express them
• develop implied constraints, search strategies,
  etc.

19
Possible Variables in the SONET Problem

• Whether a given node is on a given ring ?
• Which ring(s) each node is on ?
• Which nodes are on each ring ?
• How many nodes are on each ring ?
• How many rings each node is on ?
• The total number of nodes on all rings ?
• The total of the number of rings each node is on
  ?
• Which demand pairs are on each ring
• Which ring(s) each demand pair is on
• How many rings are used

20
Choosing the Search Variables

• We need to choose a set of variables such that an
  assignment to each one, satisfying the
  constraints, is a complete solution to the
  problem
• Assume we pass the search variables to the search
  algorithm in a list or array
• the order determines a static variable ordering

21
Possible Choices

• Use just one set of variables, e.g. xij the
  others are just for constraint propagation
• Use two (or more) sets of variables (of the same
  type) e.g. Rj ,Ni
• interleave them in a sensible (static) order
• or use a dynamic ordering applied to both sets of
  variables
• Use an incomplete set of variables first, to
  reduce the search space before assigning a
  complete set
• e.g. decide how many rings each node is on
  (search variables Ni) and then which rings each
  node is on (xij )
• All three possibilities are useful the first
  will be used in the SONET problem (and later the
  third)

22
Implied Constraints

• Constraints which can be derived from the
  existing constraints, and so dont eliminate any
  solutions
• We only want useful implied constraints
• they reduce search i.e. at some point during
  search, a partial assignment must be tried which
  is inconsistent with the implied constraint but
  would otherwise not fail immediately
• they reduce running time the overhead of
  propagating extra constraints must be less than
  the savings in search

23
How to Find Useful Implied Constraints

• Identify obviously wrong partial assignments that
  may/do occur during search
• Try to predict them by contemplation/intuition
• Observe the search in progress
• Having many variables in the model enables
  observing/thinking about many possible aspects of
  the search
• But we still need to check empirically that new
  constraints do reduce both search and running time

24
Implied Constraints SONET

• A node with degree in the demand graph gt 4 must
  be on more than 1 ring (Ni gt 1)
• If a pair of connected nodes have more than 3
  neighbours in total, at least one of the pair
  must be on more than 1 ring (NkNl gt 2)

25
Optimality Constraints

• In optimizing, if we know that for any solution
  with a particular characteristic, there must be
  another solution at least as good, we can add
  constraints forbidding it
• e.g.
• no ring should have just one node on it
• any two rings must have more than 5 nodes in
  total (otherwise we could merge them)
• Derive these in the same way as implied
  constraints

26
Variable Ordering Heuristics

• Armies of Queens
• Place a white queen next where it will attack
  fewest additional squares
• SONET problem
• add a node to a ring with spare capacity choose
  the node connected to most nodes already on the
  ring of those, the node connected to most nodes
  still to be placed

27
Finding a Good Solution v. Proving Optimality

• Armies of Queens
• the heuristic finds an optimal solution for every
  case where this is known
• but its hopeless for proving optimality
• the anti-heuristic (place a white queen where
  it will attack most additional squares) is much
  better!

28
Finding a Good Solution v. Proving Optimality
SONET

• The heuristic finds near-optimal solutions quite
  quickly, but is no good for proving optimality
• How could we prove this solution is optimal
  2,3,4,9,12, 1,3,7,8,10, 4,5,6,7,10,
  1,8,11,12,13 ?
• e.g. show that we cannot reduce the number of
  times that any of 3, 4, 7, 8, 12 appear without
  having another node appear twice instead
• Introduce variables ni Ni i.e. the number of
  rings that node i is on
• ? Two-stage search find a good solution, then
  start search again, assigning ni variables first
  and then xij variables

29
What do I mean by Faster?

• The basic model (just the xij variables, no
  symmetry-breaking) can only solve small problems
  (7 nodes, 8 demand pairs)
• The final model can solve the problems in the
  Sherali Smith paper (13 nodes, 24 demand pairs)
• 3 full sets of variables others
• SBDS
• implied constraints
• variable ordering heuristic
• two-stage search process

30
Some Advice

• Eliminate symmetry
• Use lots of variables, with channelling
  constraints
• But dont express the same problem constraints
  twice
• Add constraints that make explicit what you know
  about satisfying/optimal solutions
• But only if they reduce search and running time
• Learn from solving the problem by hand and
  observing the search
• If finding good solutions is easy and proving
  optimality is hard, consider using a different
  strategies for each stage

31
Conclusions

• Can we list 10 Steps to Successful Modelling?
• New problems still often lead to new ideas about
  modelling
• But some patterns do recur frequently
• e.g. models representing dual viewpoints
• We are beginning to automate some aspects of
  modelling
• e.g. symmetry, implied constraints
• Still a long way to go before building a good
  model of a problem is straightforward
• e.g. we often cant tell if model A is better
  than model B without trying them both
• More research is still needed...

32
Selected References

• Symmetry Breaking During Search in Constraint
  Programming, I. P. Gent and B. M. Smith,
  Proceedings ECAI'2000, pp. 599-603, 2000.
• Models and Symmetry Breaking for Peaceable
  Armies of Queens, K.E Petrie, B. M. Smith I.
  P. Gent, APES Report APES-50-2002, May 2002.
• Improving Discrete Model Representations via
  Symmetry Considerations, H. D. Sherali J. C.
  Smith, Man.Sci. (47) pp. 1396-1407, 2001.
• Increasing Constraint Propagation by Redundant
  Modeling an Experience Report, B. M. W. Cheng,
  K. M. F. Choi, J. H. M. Lee J. C. K. Wu,
  Constraints (4) pp. 167-192, 1999.