Chapter 5. Constraint Satisfaction Problems

1
Chapter 5.Constraint Satisfaction Problems

• CS570, Artificial Intelligence
• Group 11 Jin H. Park, Mi H. Koo, Chon H. Lee

2
Contents

• What is CSP?
• Solving CSP by search
• Heuristics for CSP
• Summary

3
DefinitionWhat is CSP?

• A special kind of problem
• States are defined by values of a fixed set of
  variables
• variables X1, X2, , Xn
• Domain according to a variable Xi
  (Di)
• Goal test is defined by constraints on variable
  values
• constraints C1, C2, Cn
• Problem structure is represented by constraint
  graph
• A problem where one is given (i) a (finite) set
  of variables, (ii) a function which maps every
  variable to a domain, and (iii) a set of
  constraints. Each constraint restricts the
  combination of values that a set of variables may
  take simultaneously.http//cswww.essex.ac.uk/CSP/
  tutorial.html

4
TerminologiesWhat is CSP?

• Consistent (or legal) assignment
• An assignment that does not violate any
  constraints
• Complete assignment
• An assignment in which every variable is
  mentioned
• Solution
• A complete and consistent assignment
• A complete assignment that satisfies all the
  constraints
• Constraint graph
• Node variable
• Arc constraint

5
Incremental formulation as standard search
problemWhat is CSP?

• States
• Set of value assignments to some or all of the
  variables
• Initial state
• The empty assignment
• Successor function
• Value assignment to any unassigned variable
  without conflicts with previously assigned
  variables
• Goal test
• Finding out the current assignment is complete
• Path Cost
• A constant cost

6
Benefits from treating a problem as a CSPWhat is
CSP?

• The successor function and goal test can be
  written in a generic way that applies to all
  CSPs.
• We can develop effective, generic heuristics that
  require no additional, domain-specific expertise.
• The structure of the constraint graph can be used
  to simplify the solution process.

7
Examples of CSP(1/2) 8-queen problemWhat is
CSP?

• Variables
• Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8
• Domain
• Q1, , Q8 ? (1,1), (1,2), , (8,8)
• Constraint set
• No queen attacks any other

8
Examples of CSP(2/2) map-coloring problemWhat
is CSP?

• Variables
• WA, NT, Q, NSW, V, SA,T
• Domain
• WA, NT, , SA ? red, green, blue
• Constraints
• Neighboring regions must have distinct colors.
• ( WA ? NT, WA ? SA, )

9
The classification of CSP - by variables domain
What is CSP?

• Discrete variables
• Finite domains
• 8-queen problem, map-coloring problem
• Boolean CSP
• Includes 3SAT(NP-complete)
• gt Dont expect that in the worst case, we can
  solve finite-domain CSP in less
  than exponential time.
• Infinite domains
• Job scheduling
• Continuous variables
• Linear programming problem

10
The classification of CSP - by constraints
characteristic What is CSP?

• Absolute Constraint
• Unary Constraint
• Ex. SA ? green
• Binary Constraint
• Ex. SA ? WA
• Higher-order Constraint
• Ex. Crypt-arithmetic column constraints
• Preference Constraint
• ex) Prof. X might prefer teaching in the morning.
• Many real-world CSP
• Solved with optimization search methods

11
Backtracking SearchSolving CSP by search(1)

• BFS vs. DFS
• BFS ? terrible!
• A tree with n!dn leaves (nd)((n-1)d)((n-2)d)
  (1d) n!dn
• Reduction by commutativity of CSP
• A solution is not in the permutations but in
  combinations.
• A tree with dn leaves
• DFS
• Used popularly
• Every solution must be a complete assignment and
  therefore appears at depth n if there are n
  variables
• The search tree extends only to depth n.
• A variant of DFS Backtracking search
• Chooses values for one variable at a time
• Backtracts when failed even before reaching a
  leaf.
• Better than BFS due to backtracking, but still
  inefficient!!

12
Backtracking SearchSolving CSP by search(2)
function BACKTRACKING-SEARCH (csp) returns a
solution, or failure return
RECURSIVE-BACKTRACKING(, csp) function
RECURSIVE-BACKTRACKING(assignment, csp) returns a
solution, or failure if assignment is complete
then return assignment var ?
SELECT-UNASSIGNED-VARIABLE(VARIABLEScsp,
assignment, csp) for each value in
ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according
to CONSTRAINTScsp then add
varvalue to assignment result ?
RECURSIVE-BACKTRACKING(assignment, csp)
if result ! failure then return result
remove var value from assignment
return failure
? BACKTRACKING OCCURS HERE!!
13
Backtracking SearchImproving backtracking
efficiency

• Which variable should be assigned next?
• MRV heuristic
• In what order should its values be tried?
• LCV heuristic
• Can we detect inevitable failure early?
• Forward checking
• Constraint propagation (Arc Consistency)
• When a path fails, can the search avoid repeating
  this failure?
• Backjumping
• Can we take advantage of problem structure?
• Tree-structured CSP

14
Backtracking SearchImproving backtracking
efficiency
function BACKTRACKING-SEARCH (csp) returns a
solution, or failure return
RECURSIVE-BACKTRACKING(, csp) function
RECURSIVE-BACKTRACKING(assignment, csp) returns a
solution, or failure if assignment is complete
then return assignment var ?
SELECT-UNASSIGNED-VARIABLE(VARIABLEScsp,
assignment, csp) for each value in
ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according
to CONSTRAINTScsp then add
varvalue to assignment result ?
RECURSIVE-BACKTRACKING(assignment, csp)
if result ! failure then return result
remove var value from assignment
return failure
15
MRV LCVHeuristics for variable value ordering

• Variable selection
• MRV(minimum remaining values) heuristic
• Choose the variable with the fewest legal values.
• Degree heuristic
• In first variable selection, MRV heuristic
  doesnt help at all.
• Use degree heuristic (selects the variable with
  the largest degree.) ? Tie-breaker !!
• Value ordering
• Least-Constraining-value heuristic
• ? not to rule out neighbors

16
MRV LCV an example for degree heuristic
Heuristics for variable value ordering
NT
NT
Q
SA
SA
SA
NT R, G, B,3 Q R, G, B,3 WA R, G,
B,2 SA R, G, B,5 NSW R, G, B,3 V R,
G, B,2 T R, G, B,0
NT R, G Q R, G WA R, G SA B NSW
R, G V R, G T R, G, B
NT G Q R WA R SA G NSW R, G V
R, G T R, G, B
3 3 2 5 3 2 0
17
MRV LCV an example for MRV Heuristics for
variable value ordering
NT
NT
Q
SA
SA
SA
NT R, G, B Q R, G, B WA R, G, B SA
R, G, B NSW R, G, B V R, G, B T R,
G, B
NT G, B Q R, G, B WA R SA G, B NSW
R, G, B V R, G, B T R, G, B
NT G Q R, B WA R SA B NSW R, G,
B V R, G, B T R, G, B
18
MRV LCV an example for LCVHeuristics for
variable value ordering
Red in Q is least constraint value
NT
Q
WA
NT
WA
WA
Q
NT
WA
NT G, B Q R, G, B WA R SA G, B NSW
R, G, B V R, G, B T R, G, B
NT R, G, B Q R, G, B WA R, G, B SA
R, G, B NSW R, G, B V R, G, B T R,
G, B
NT G Q R, B WA R SA B NSW R, G,
B V R, G, B T R, G, B
NT G Q R WA R SA B NSW G, B V R,
G, B T R, G, B
NT G Q B WA R SA ? NSW R,G V R, G,
B T R, G, B
19
Forwarding checking(1/4)Heuristics for early
failure-detection

• How can we find out which variable is minimum
  remaining?
• What does eliminate the values from domain?

NT red, Q green WA red, green, blue SA
red, green, blue NSW red, green, blue V
red, green, blue T red, green, blue
?
?
?
?
20
Forwarding checking(2/4)Heuristics for early
failure-detection

• Forward checking
• A variable X is assigned.
• Looks at each unassigned variable Y connected to
  X by a constraint
• Deletes any values of Ys domain that is
  inconsistent with X
• If there is any Y with empty domain, undo this
  assigning.
• ? Backtrack!!
• ? If not, MRV !!
• Forward checking is an obvious partner of MRV
  heuristic !!
• ? MRV heuristic is activated after forward
  checking.

21
Forwarding checking(3/4) an exampleHeuristics
for early failure-detection
NT
Q
WA
SA
NSW
V
T
22
Forwarding checking(4/4) - Heuristic
flowHeuristics for early failure-detection
Forward Checking
Yes
Tie?
No
Degree heuristic
Select the MRV
Forward Checking
Select the LCV
Goal?
23
Constraint propagation arc consistency
Heuristics for early failure-detection

• The (directed) arc (X?Y) is consistent
• For all x ? X, there is some y ? Y.
• Detects an inconsistency that is not detected by
  pure F/C.
• Makes CSP possibly with reduced domain.
• Provides a fast method of constraint propagation
• For preprocessing or propagation step
• O(n2d3)
• n2 the maximum number of arcs
• d the maximum number of insertion for one node
• d2 the number of forward checking

24
Constraint propagation arc consistency
algorithmHeuristics for early failure-detection
function AC-3(csp) returns the CSP, possibly with
reduced domains inputs csp, a binary CSP with
variables X1, X2, , Xn local variables
queue, a queue of arcs, initially all the arcs in
csp while queue is not empty do
(Xi,Xj) ? REMOVE-FIRST(queue) if
REMOVE-INCONSISTENT-VALUES(Xi,Xj) then
for each Xk in NEIGHBORSXi do
add(Xk,Xi) to queue function
REMOVE-INCONSISTENT-VALUES() returns true iff we
remove a value removed ? false for each x
in DOMAINXi do if no value y in
DOMAINXj allows (x,y) to satisfy the constraint
between Xi and Xj then delete x
from DOMAINXi removed ? true return removed
25
Special constraints Alldiff Heuristics for
early failure-detection

• Problem-dependent constraints
• Alldiff constraint
• If there are m variables involved in the
  constraint and n possible distinct values, m gt n
  ? inconsistent for Alldiff constraint
• An example
• 3 variables NT,SA,Q
• NT, SA, Q ? green, blue
• m3 gt n2
• inconsistent

WA
NSW
26
Special constraints AtmostHeuristics for early
failure-detection

• Resource constraint
• Atmost constraint(Use limited resource)
• An example
• Totally, no more than 10 personnel should be
  assigned to jobs.
• Atmost(10, PA1, PA2, PA3, PA4)
• Job14 ? 3,4,5,6 ( min value 3 )
• 333 3 gt 10 (minimum resource 12 but available
  resource 10)
• inconsistent
• Job14 ? 2, 3,4,5,6
• 2222 lt10 (consistent)
• 2225(or 6) gt 10(inconsistent)
• Delete 5,6 from domain

27
BackjumpingHeuristics for early failure-detection
1

• The weakness of Backtracking
• If failed in SA, backtrack to T.
• But, T is not relevant to SA.
• Backjumping
• Backtracks to the most recent variable in the
  conflict set.
• Conflict set the set of variables that caused
  the failure
• If failed in SA, backjump to V.

Q
NSW
2
5
V
3
Next variable
T
4
28
Local Search(1/2)Min-conflicts heuristic

• The initial state assigns a value to every
  variable, and successor function usually works by
  changing the value of one variable at a time.
• Select the value that results in the minimum of
  conflicts with other variables
• Min-conflicts is effective for many CSPs,
  particularly when given a reasonable initial
  state.
• Ex. n-queen problem, Hubble space telescope
  scheduling
• Specially important in scheduling problems
• Airline schedule (when the schedule is changed)

29
Local Search(2/2)Min-conflicts heuristic

• Example of min-conflict heuristic

30
GeneralHeuristics for complexity reduction

• The only way we can possibly hope to deal with
  the real world is to decompose it into many
  sub-problems.
• Independent sub-problems
• Solve them respectively ? if assignment Si is a
  solution of CSPi, then UiSi is a solution of
  UiCSPi
• Why important?
• Each CSPi has c variables from the total of n
  variables ? n/c subproblems
• dc work for each sub-problem, so total work is
  O(dcn/c) ? linear in n !!
• Without decomposition, O(dn) ? exponential in n
  !!
• Ex. n(node)80, d(domain)2, c(variables in
  subproblem)20
• 280 vs. 4220( 4 billion years vs. 0.4 sec)
• Ex) The mainland and Tasmania in previous
  map-coloring.
• But, rare!!!

31
Tree-structured CSPHeuristics for complexity
reduction

• If constraint graph is a tree, we solve the CSP
  in time linear in the number of variables
• 1. Choose a variable as root
• 2. Order variables from root to leaves such that
  every nodes parent precedes it in the ordering
  ?Makes constraint graph arc-consistent
• 3. For j from n down to 2, apply arc consistency
  to the arc(Xi, Xj)
• Xi parent of Xj
• ?Just finds a consistent value.(no backtrack
  because of arc-consistency)
• 4. For j from 1 to n, assign Xj consistently
  with Parent(Xj)
• Time complexity O(nd2)

32
Tree-structured CSP an linear ordering example
Heuristics for complexity reduction
33
Transforming general CSP to tree-structured
CSP(1/4)Heuristics for complexity reduction

• Cutset conditioning
• Remove some variables S
  (Assign values to
  some variables)
• S is called Cycle cutset when constraint graph is
  a tree after the removal of S
• Time Complexity O(dc(n-c)d2)
• c cutset size
• dc the of possible cutset assignment
• Time complexity in tree-structred CSP (n-c)d2

34
Transforming general CSP to tree-structured
CSP(2/4)Heuristics for complexity reduction
35
Transforming general CSP to tree-structured
CSP(3/4)Heuristics for complexity reduction

• Tree decomposition of constraint graph
• Each sub-problem is solved independently, and the
  resulting solutions are combined.
• Subprogram should not be too large.
• Requirements for tree decomposition
• Every variable in the original problem appears in
  at least on of the sub-problems.
• If two variables are connected by a constraint in
  the original problem, they must appear together
  in at least one of the sub-problems.
• If a variable appears in two sub-problems in the
  tree, it must appear in every sub-problem along
  the path connecting those sub-problems.
• Divides graph to many components (sub-tree)
• If components are regarded as nodes, it will be a
  tree.
• Applies tree-structured algorithm
• In arc consistency, check the common variables in
  the two component

36
Transforming general CSP to tree-structured
CSP(4/4)Heuristics for complexity reduction

• Time Complexity O(ndw1)
• of possible value assignment of last ordering
  component dw
• of components n/w
• Time complexity for arc consistency
• Max of common values in two components w
• w d

37
Summary

• CSPs are a special kind of problem
• States defined by values of a fixed set of
  variables
• Goal test defined by constraints on variable
  values
• are represented by constraint graph
• Backtracking a DFS with one variable assigned
  per node
• Variable ordering value selection heuristics
  help significantly
• Forward checking prevents assignments that
  guarantee later failure
• Constraint propagation does additional work to
  constrain values and detect inconsistencies
• The CSP representation allows analysis of problem
  structure
• Tree-structured CSPs can be solved in linear time
• Cutset conditioning can reduce a general CSP to a
  tree structured on
• If cutset is small, cutset conditioning can be
  very efficient
• Tree decomposition techniques transform the CSP
  into a tree of subprograms
• Tree width should be small for tree decomposition
  technique to be efficient