Escaping from the CSP Straitjacket

1
Escaping from the CSP Straitjacket

• Mark Wallace

2
Aims

• Constraint Research why do it?
• Exciting challenges
• Constraint programming why do it this way?
• A relevant approach
• IC-Parc and Parc Technologies
• The buzz of solving applications

3
Logic The Vision Thing

• Formalisation of Mathematics
• Until Goedel This formula cannot be proven
• Guaranteed Correct Systems
• Failure of runnable specifications
• General Problem Solver
• Unscalability of AI

4
Logic Restoring the Holy Grail

• The New GPS
• Tackle a specific (wide) class of problems
• NP
• Separate model from algorithm
• Algorithm Logic Control

5
Steps towards the Vision

• Start bottom up
• Routing
• Partitioning
• Scheduling
• Collect and analyse algorithms
• Constraint propagation
• Linear constraint solving
• Complete search
• Stochastic search
• Devise (modelling) languages
• For expressing algorithms
• For combining algorithms

6
Robert Browning

• Constraint Programming
• That low man goes on adding one to one
• His hundreds soon hit

• CSP
• This high man aiming at a million
• Misses a unit

7
Modelling

• Separate model from algorithm
• Algorithm Logic Control
• Devise (modelling) languages
• For expressing algorithms
• For combining algorithms

Conceptual model
Design model
8
Modelling Science or Art?

• Functions
• A functional style of programming is natural
• Entities/Relationships
• The concept of Entity/Relationship is a simple
  but very natural and powerful concept for
  modelling our perceptions of the real world
• Objects
• An object-oriented program more closely
  resembles the world it is modelling
• Agents
• The ability of agents to plancooperate,
  coordinate and negotiate will lead to
  significant improvements in the quality and
  sophistication of the software systems that can
  be conceived and implemented

9
Constraints Modelling

• Bertrand
• A constraint language can make it easier for a
  human problem solver to describe a problem to a
  computer
• OCL
• We need the precision of mathematics, but the
  ease of use of natural language
• AMPL
• Before solving, it is necessary to formulate
  the underlying model, and generate the requisite
  data structures. AMPL is designed to make these
  steps easier and less error-prone

10
Modelling Formalisms
11
Constraint Satisfaction Problem (CSP)

• Modeling Formalism for Combinatorial Problems
• Simple
• Powerful
• Well-researched

12
Constraint Satisfaction Problem

• Set of Variables - V
• Set of Domains - D
• Set of Constraints - C

Fr,b
V
N,G,B,F
c
Gr,y
D
r,y,b,r,y,r,b
c
c
C
c cltr,ygt,ltr,bgt,ltb,rgt,ltb,ygt,lty,rgt,lty,b
gt
c
Nr,y,b
Br,b
c
13
Weaknesses of the CSP Formalism

• Set of Variables V
• Fixed size
• Set of Domains D
• Redundant
• Set of Constraints C
• Unstructured and fixed arity

14
The Maze A Problem Naturally Modelled with an
Unspecified Number of Variables
start
Solution start,b1,b2,b3,centre
d2
b1
b2
b3
maze(Path) - fromto(start,Here,There,centre),
fromto(,Sofar,HereSofar,Path) do
link(Here,There), not
member(There,Sofar). link(start,b1).
centre
b4
d1
15
Weaknesses of the CSP Formalism

• Set of Variables V
• Fixed size
• Set of Domains D
• Redundant
• Set of Constraints C
• Unstructured and fixed arity

16
Constraint Propagation by tightening domains

• State
• S D1 x D2 x x Dn
• Constraint Behaviour
• fc S -gt S
• Propagation Behaviour
• Fixpoint of fc1,,fcm

17
Constraint Behaviour
X
Forward checking
Y
Z
X1..10
Y1..10
Label X1
Label Y1
X
Arc-consistency
Y
Z
Label X1
Z1..10
Label Y2
18
Hybrid Constraint Behaviour
- passive check
X1..10
Y1..10
- forward checking
- bounds
X
Y
Z
Label X1
Label Y2
Z1..10
19
Constraint Reasoning

• Constraint Simplification
• Constraint Rewriting
• Substitution
• Cutting Planes
• integers x,y
• Generalised Propagation

20
Weaknesses of the CSP Formalism

• Set of Variables V
• Fixed size
• Set of Domains D
• Redundant
• Set of Constraints C
• Unstructured and fixed arity

21
A Constraint
Resource Taska
Resource Taskb
Start Taska
Start Taskb
22
What useful information can be extracted?

• Constraint tightness
• The fraction of possible n-tuples that are
  allowed
• Constraint Density (for binary CSPs)
• The fraction of variable pairs that are
  constrained

23
Constraint Classes

• Linear Constraints
• Simplex
• Functions
• AC5
• Disjunctive Constraints
• Search
• Temporal/Resource Constraints
• Heuristics

24
What sort of constraint is this?
Resource Taska
Resource Taskb
Start Taska
Start Taskb
25
A Natural Formulation
26
Global Constraints have no Fixed Arity

• Constraint Definition all_distinct
• No member of List occurs more than once
• Constraint Use
• all_distinct(X,Y,Z).
• all_distinct(A,B,C,D).
• all_distinct(X,3,Y).
• Global Constraints Capture Problem Structure
• Pigeonhole problems exponentially hard to solve
  using resolution
• Pigeonhole problems are easy using specialised
  constraint behaviour

27
Logic Restoring the Holy Grail

• The New GPS
• Tackle a specific (wide) class of problems
• NP
• Separate model from algorithm
• Algorithm Logic Control

28
The Ultimate Problem
to think creatively in essence, to solve the
problem of how to solve problems
29
Solving a Constraint Problem
Specify the Model
M Specify the Constraint Behaviour(s)
C Specify the Search Behaviour S
Dependencies
M C S
30
The Meta-Problem (GPS)

• Meta-Variables and Possible Values
• Model (transformation)
• Constraint behaviours
• Search Behaviour
• Meta-Constraints
• How problem model/constraints/search behaviours
  interact
• Meta-level search
• The search for the best model and behaviour
• e.g. Hill Climbing (Minton)
• Local search (Caseau)

Mt1,t2,tn1 Cp1,p2,pn2 Ss1,s2,sn3
Solution Mt2, Cp4, Ss1
31
GPS from the CSP Perspective

• Transform the model
• CSP -gt CSP
• Specify a complete set of transformations
• Propagate Constraints
• State -gt State
• Specify a complete set of propagation behaviours
• Search for a solution
• Label variables
• Specify a complete set of labelling heuristics

32
CSPs and Search Behaviour

• Search space D1 x D2 x x Dn
• Solution complete labelling of variables
• Search step add/change variable label

33
Generic Search Heuristic

• Constrainedness measure
• Size of search space (Domains)
• Number of solutions (Constraint tightness and
  density)
• Variable Choice
• minimise constrainedness of the remaining
  subproblem
• Estimating Constrainedness of Subproblem
• Reduce domains of linked variables
• Remove variable and all its constraints
• Apply constrainedness measure

34
Why not tackle the GPS Meta-Problem?

• No Complete Set of Model transformations
• Not just CSP models
• No Complete Set of Constraint Behaviours
• Not just state-gtstate functions
• No Complete Set of Search Methods
• Not just variable labelling

35
Weaknesses of the CSP Formalism

• Set of Variables V
• Fixed size
• Set of Domains D
• Redundant
• Set of Constraints C
• Unstructured and fixed arity
• Influence over Solving techniques
• Domain propagation
• Search by labelling
• Optimisation
• CSP with a constraint on the cost

36
Non-CSP Transformations

• Types of Transformation
• Problem implementation
• Model -gt Data Structures
• Problem decomposition
• Variable in one subproblem represents tuple in
  another
• Transformation During Search
• Splitting disjuncts
• (replace non-overlaps with precedes or
  follows
• Removing symmetries
• (domainused values1 new value)
• Dropping variables
• (exiting from a CLP clause)

37
Non-CSP forms of constraint reasoning

• Global Constraints
• Tailored to a class of problems
• Controlling constraint behaviours
• CHRs
• communicating constraint solvers
• adapt to search behaviour
• Adding constraints, and no goods during search

38
Search but not labelling

• Split a disjunctive constraint
• R1 R2 R1 \ R2
• Add a bound
• Xlt 4 X gt 5
• Order two temporal variables
• S1 gt S25 S2 gt S17
• Add a constraint
• C1 C2 where

39
Non-CSP Forms of Search

• Probing
• Solve relaxed problem
• Introduce a constraint to fix a violation
• CLP(X)
• Unfold a clause from the definition of a
  predicate
• Just add constraints
• Introduce global cuts until the convex hull is
  integer-valued
• Just add Variables
• Column generation

40
Weaknesses of the CSP Formalism

• Set of Variables V
• Fixed size
• Set of Domains D
• Redundant
• Set of Constraints C
• Unstructured and fixed arity
• Influence over Solving techniques
• Domain propagation
• Search by labelling
• Optimisation
• CSP with a constraint on the cost

41
Constraint Satisfaction and Optimisation Problem
(CSOP)

• Sequence of CSPs
• Constrain cost to be better than solution of
  previous CSP
• Granularity of a CSOP
• Fine if solutions with any given cost is small
• Coarse if solution with any given cost is large
• Phase Transition
• CSPs are hard if the expected number of solutions
  is one
• All fine grained CSOPs are hard

42
Optimisation Problems

• Value ordering matters
• Balance objectives
• Feasibility
• Cost
• Algorithms
• Local Improvement
• Examples
• Simplex
• Column Generation

43
Decomposition Master and Slave Problem
Master Problem - optimisation
Slave Problem
44
Cutting Stock
Requirements 51 X 47 X 80 X
C
B
A
Raw Material Board
A cutting
45
Cutting Stock - Solving

• Fix max no. of boards
• Yields set of variables
• Find all ways of cutting a board
• Yields domain for each variable
• Label Variables
• Constraints ensure the requirements are met
• Empty cutting for boards not needed
• Weakness
• There are often a huge number of ways of cutting
  a board

46
Cutting Stock by Column Generation

• Find an initial solution
• Search for a cutting which could improve the
  current solution the slave problem
• Find a better solution, using the new cutting
  the master problem
• If success, return to 2), else stop.

47
Cutting Stock Slave Problem

• Solution (a cutting) Cut
• Number of pieces of size A NACut
• Number of pieces of size B NBCut
• Number of pieces of size B NCCut
• Cost CostCut
• Constraints
• Physical constraints on possible cuttings, eg

Pricing Constraint (NACut SPASol) (NBCut
SPBSol) (NCCut SPCSol) gt CostCut
48
Cutting Stock Master Problem

• Solution Sol
• Variables
• NCut1 Number of boards with cutting Cut1
• NCut2 Number of boards with cutting Cut2
• Overall Cost
• NCut1 CostCut1 NCut2 CostCut2
• (CostCut is the cost returned from the slave
  problem)
• Constraints (Linear)
• NCut1NACut1 NCut2NACut2 gt 51
• (NACut is the number of pieces of size A)
• NCut1NBCut1 NCut2NBCut2 gt 47
• (NBCut is the number of pieces of size B)
• NCut1NCCut1 NCut2NCCut2 gt 80
• (NCCut is the number of pieces of size C)

(Shadow price SPASol)
(Shadow price SPBSol)
(Shadow price SPCSol)
49
Analysis of Column Generation

• Domains grow during search
• to include newly found cuttings
• Or
• Variables increase during search
• Each cutting is a variable

50
That Holy Grail Again

• The GPS Meta-Problem - Prerequisites
• The Meta-Domains?
• Problem Representation
• Constraint Behaviours language
• Search behaviour language
• The Meta-Constraints
• How the model, constraint and search behaviours
  constrain each other
• The Meta-Problem Solver
• Learning the right model, constraint and search
  behaviours for the problem at hand

51
Constraint Programming Research Aims - Algorithms

• Map the whole world of algorithms
• Identify an orthogonal set of primitive
  behaviours
• Develop a generic language for combining
• Problem representations
• Transformations
• Constraint behaviours
• Search methods

52
CSP and Constraint Programming -Language Aims

• Teaching MBAs
• Familiar
• Simple
• Closed
• Developing Advanced Algorithms
• Powerful
• Integrated
• Reconfigurable

53
IC-Parc Research Aims the ECLiPSe Language

• Separate modelling from behaviour
• Support an extensive toolset
• Representations
• Algorithms
• Support experimentation
• Representations
• Algorithms
• Hybrids

54
IC-Parc and Parc Technologies - Applications

• Transportation
• Internet Support
• Logistics