An Overview of Constraint Programming

1
An Overview of Constraint Programming

• APORS 2000 Tutorial
• July 7, 2000
• Martin Henz, National University of Singapore

2
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

3
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

4
Constraint Programming in a Nutshell

• SEND MORE MONEY

5
Constraint Programming in a Nutshell

• SEND MORE MONEY

6
SEND MORE MONEY

• Assign distinct digits to the letters
• S, E, N, D, M, O, R, Y
• such that S E N D
• M O R E
• M O N E Y
• holds.

7
SEND MORE MONEY

• Assign distinct digits to the letters
• S, E, N, D, M, O, R, Y
• such that S E N D
• M O R E
• M O N E Y
• holds.

Solution 9 5 6 7 1 0 8
5 1 0 6 5 2
8
Modeling

• Formalize the problem as a constraint problem
• number of variables n
• constraints c1,,cm ? ?n
• problem Find a (v1,,vn)? ?n such
• that a ? ci , for all 1 ? i ?
  m

9
A Model for MONEY

• number of variables 8
• constraints
• c1 (S,E,N,D,M,O,R,Y)? ?8 0 ? S,,Y ? 9
• c2 (S,E,N,D,M,O,R,Y)? ?8
• 1000S 100E 10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

10
A Model for MONEY (continued)

• more constraints
• c3 (S,E,N,D,M,O,R,Y)? ?8 S ? 0
• c4 (S,E,N,D,M,O,R,Y)? ?8 M ? 0
• c5 (S,E,N,D,M,O,R,Y)? ?8 SY all different

11
Solution for MONEY

• c1 (S,E,N,D,M,O,R,Y)? ?8 0?S,,Y?9
• c2 (S,E,N,D,M,O,R,Y)? ?8
• 1000S 100E 10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y
• c3 (S,E,N,D,M,O,R,Y)? ?8 S ? 0
• c4 (S,E,N,D,M,O,R,Y)? ?8 M ? 0
• c5 (S,E,N,D,M,O,R,Y)? ?8 SY all
  different
• Solution (9,5,6,7,1,0,8,2)? ?8

12
Elements of Constraint Programming

• Exploiting constraints during tree search
• Use propagation algorithms for constraints
• Employ branching algorithm
• Execute exploration algorithm

13
S E N D M O R E M O N E Y
S ? ? E ? ? N ? ? D ? ? M ? ? O ? ? R ? ? Y ? ?

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

14
S E N D M O R E M O N E Y
Propagate
S ? 0..9 E ? 0..9 N ? 0..9 D ? 0..9 M ?
0..9 O ? 0..9 R ? 0..9 Y ? 0..9

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

15
S E N D M O R E M O N E Y
Propagate
S ? 1..9 E ? 0..9 N ? 0..9 D ? 0..9 M ?
1..9 O ? 0..9 R ? 0..9 Y ? 0..9

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

16
S E N D M O R E M O N E Y
Propagate
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

17
S E N D M O R E M O N E Y
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
Branch
E ? 4
E 4
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

18
S E N D M O R E M O N E Y
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
Propagate
E ? 4
E 4
S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8

• 0?S,,Y?9
• S ? 0
• M ? 0
• SY all different
• 1000S 100E
  10N D
• 1000M 100O 10R E
• 10000M 1000O 100N 10E Y

19
S E N D M O R E M O N E Y
Branch
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E ? 4
E 4

S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 5
E ? 5
S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
20
S E N D M O R E M O N E Y
Propagate
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E ? 4
E 4

S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 5
E ? 5
S ? 9 E ? 5 N ? 6 D ? 7 M ? 1 O ? 0 R
? 8 Y ? 2
S ? 9 E ? 6..7 N ? 7..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
21
S E N D M O R E M O N E Y
Complete Search Tree
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 4
E ? 4

S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 5
E ? 5
S ? 9 E ? 5 N ? 6 D ? 7 M ? 1 O ? 0 R
? 8 Y ? 2
S ? 9 E ? 6..7 N ? 7..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E ? 6
E 6


22
Constraint Programming Systems

• Role support elements of
  constraint programming
• Provide propagation algorithms for constraints
• all different (e.g. wait for fixing)
• summation (e.g. interval consistency)
• Allow choice of branching algorithm
• (e.g. first-fail)
• Allow choice of exploration algorithm
• (e.g. depth-first search)

23
Using Oz
proc Money Root S E N D M O R Y in
Rootsol(sS eE nN dD mM oO rR yY) Root
09 FD.distinct Root S \ 0 M \
0 1000S 100E 10N D
1000M 100O 10R E 10000M
1000O 100N 10E Y FD.distribute ff
Root end ExploreAll Money
Modeling
Propagation
Branching
Exploration
MONEY Demo
24
The Art of Constraint Programming

• Choose model
• Choose propagation algorithms
• Choose branching algorithm
• Choose exploration algorithm

25
Programming Systems for Finite Domain Constraint
Programming

• Finite domain constraint programming library
• PECOS (Puget 1992)
• ILOG Solver (Puget 1993)
• Finite domain constraint programming languages
• CHIP (Dincbas, Hentenryck, Simonis, Aggoun 1988)
• SICStus Prolog (Haridi, Carlson 1995)
• Oz (Smolka and others 1995)
• OPL (van Hentenryck 1998)

26
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

27
Constraint Problems

• A finite domain constraint problem consists of
• number of variables n
• constraints c1,,cm ? ?n
• The problem is to find
• a (v1,,vn)? ?n such that
• a ? ci , for all 1 ? i ? m

28
Constraint Solving

• Given a satisfiable constraint C and a new
  constraint C.
• Constraint solving means deciding whether C ? C
  is satisfiable.
• Example
• C n gt 2
• C an bn cn

29
Constraint Solving

• Constraint solving is not possible for general
  constraints.
• Constraint programming separates constraints into
• basic constraints complete constraint solving
• non-basic constraints propagation (incomplete)
  search needed

30
Basic Constraints in Finite Domain Constraint
Programming

• Basic constraints are conjunctions of constraints
  of the form X ? S, where S is a finite set of
  integers.
• Constraint solving is done by intersecting
  domains.
• Example
• C ( X?1..10 ? Y?9..20 )
• C ( X?9..15 ? Y?14..30 )
• In practice, we keep a solved form, storing the
  current domain of every variable.

31
Basic Constraints and Propagators
1000S 100E 10N D
1000M 100O 10R E 10000M 1000O
100N 10E Y
all_different(S,E,N,D, M,O,R,Y)
S ? 1..9 E ? 0..9 N ? 0..9 D ? 0..9 M ?
1..9 O ? 0..9 R ? 0..9 Y ? 0..9
32
Basic Constraints and Propagators
1000S 100E 10N D
1000M 100O 10R E 10000M 1000O
100N 10E Y
all different(S,E,N,D, M,O,R,Y)
S ? 1..9 E ? 0..9 N ? 0..9 D ? 0..9 M ?
1 O ? 0..9 R ? 0..9 Y ? 0..9
33
Basic Constraints and Propagators
1000S 100E 10N D
1000M 100O 10R E 10000M 1000O
100N 10E Y
all different(S,E,N,D, M,O,R,Y)
S ? 2..9 E ? 0,2..9 N ? 0,2..9 D ?
0,2..9 M ? 1 O ? 0,2..9 R ? 0,2..9 Y ?
0,2..9
34
Basic Constraints and Propagators
1000S 100E 10N D
1000M 100O 10R E 10000M 1000O
100N 10E Y
all different(S,E,N,D, M,O,R,Y)
S ? 2..9 E ? 0,2..9 N ? 0,2..9 D ?
0,2..9 M ? 1 O ? 0 R ? 0,2..9 Y ? 0,2..9
and so on and so on
35
Basic Constraints and Propagators
1000S 100E 10N D
1000M 100O 10R E 10000M 1000O
100N 10E Y
all different(S,E,N,D, M,O,R,Y)
S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
36
Issues in Propagation

• Expressivity What kind of information can be
  expressed as propagators?
• Completeness What behavior can be expected from
  propagation?
• Efficiency How much computational resources does
  propagation consume?

37
Completeness of Propagation

• Given Basic constraint C and propagator P.
• Propagation is complete, if for every variable x
  and every value v in the domain of x, there is an
  assignment in which xv that satisfies C and P.
• Complete propagation is also called
• domain-consistency or arc-consistency.

38
Completeness of Propagation

• General arithmetic constraints are undecidable
  (Hilberts Tenth Problem).
• Propagation may not exhibit all inconsistencies.
• Example
• c1 n gt 2
• c2 an bn cn

39
Example Complete All Different

• C w ? 1,2,3,4
• x ? 2,3,4
• y ? 2,3,4
• z ? 2,3,4
• P all_different(w,x,y,z)

40
Example Complete All Different

• C w ? 1,2,3,4
• x ? 2,3,4
• y ? 2,3,4
• z ? 2,3,4
• P all_different(w,x,y,z)
• Most efficient known algorithm O(X2 dmax2)
• Regin 1994, using graph matching

41
Basic Constraints vs. Propagators

• Basic constraints
• are conjunctions of constraints of the form
  X ? S, where S is a finite set of integers
• enjoy complete constraint solving
• Propagators
• can be arbitrarily expressive (arithmetic,
  symbolic)
• implementation typically fast but incomplete

42
Propagation vs Branching

• Obvious trade-off
• Complex propagation algorithms result in
  fewer, but more expensive nodes in the search
  tree.
• Example MONEY with

alldiff and sum only test fixed assignment
alldiff wait for fixed variables sum interval
cons.
alldiff and sum domain consistency
43
Some Propagator Classes

• Symbolic propagators
• Arithmetic propagators
• Scheduling propagators
• Reification

44
Symbolic PropagatorsExample The Element
Propagator

• Oz FD.element I Vector X
• Meaning X is the Ith element of Array
• Example A 5 6 7 8
• I ? 0,9 X ? 0,9
• Propagation in both directions
• I ? 1,3 ? X ? 5,7
• X ? 7 ? I ? 4

45
Arithmetic Propagators

• General arithmetic equations
• I1X11X1m1 InXn1Xnmn 0
• lt gt lt gt \

46
Scheduling Propagators

• Details later.
• Scheduling constraint classes
• resource constraints
• precedence constraints

47
Reified Constraints

• Reflecting the validity of a constraint in a
• 0/1 variable
• Example Reified arithmetic equations
• X (I1X11X1m1 InXn1Xnmn 0)
• lt gt lt gt \

48
Excursion From Propagation to Problem Solving
49
Branching Algorithms

• Constraint programming systems provide
• libraries of predefined branching algorithms
• programming support for user-defined branching
  algorithms

50
Branching for MONEY
proc Money Root S E N D M O R Y in
Rootsol(sS eE nN dD mM oO rR yY) Root
09 FD.distinct Root S \ 0 M \
0 1000S 100E 10N D
1000M 100O 10R E 10000M
1000O 100N 10E Y FD.distribute ff
Root end ExploreAll Money
Modeling
Propagation
Branching
Exploration
51
Basic Choice Points

• choice x lt y x gt y end

x lt y
x gt y
52
Choice Point Sequences

• choice x lt y x gt y end
• choice z 1 z 2 end

x lt y
x gt y
z 1
z 2
z 1
z 2
53
Examples of Branching Algorithms

• Enumeration Choose variable, choose value
• naive enumeration
  choose variables and values in
  a fixed sequence
• first-fail enumeration
  choose a variable with minimal
  domain size
• Domain-splitting
• choice X lt Mid X gt Mid end
• Task sequencing for scheduling (later)

54
Excursion Branching Algorithms in Oz
55
Exploration Algorithms

• Combinations of aspects such as
• Order of exploration
• Interaction
• Optimization
• Visualization

56
Exploration for MONEY
proc Money Root S E N D M O R Y in
Rootsol(sS eE nN dD mM oO rR yY) Root
09 FD.distinct Root S \ 0 M \
0 1000S 100E 10N D
1000M 100O 10R E 10000M
1000O 100N 10E Y FD.distribute ff
Root end ExploreAll Money
Modeling
Propagation
Branching
Exploration
57
Order of Exploration

• Depth-first search
• Iterative Deepening
• Limited discrepancy search Harvey/Ginsberg 95

58
Interaction

• First-solution search
• All solution search
• Last solution search
• Search with user interaction

59
Optimization

• Branch-and-bound
• Restart optimization

60
Visualization

• Example Oz Explorer Schulte 1997.
• Oz Explorer combines
• visualization
• first/all solution / user interaction
• branch-and-bound optimization
• depth-first search

61
Excursion Exploration Algorithms in Oz
62
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

63
ACC 1997/98 A Success Story of Constraint
Programming

• Integer programming enumeration, 24 hours
• Nemhauser, Trick Scheduling a Major College
  Basketball Conference, Operations Research, 1998,
  46(1)
• Constraint programming, less than 1 minute.
• Henz Scheduling a Major College Basketball
  Conference - Revisited, Operations Research, to
  appear

64
Round Robin Tournament Planning Problems

• n teams, each playing a fixed number of times r
  against every other team
• r 1 single, r 2 double round robin.
• Each match is home match for one and away match
  for the other
• Dense round robin
• At each date, each team plays at most once.
• The number of dates is minimal.

65
The ACC 1997/98 Problem

• 9 teams participate in tournament
• Dense double round robin
• there are 2 9 dates
• at each date, each team plays either home, away
  or has a bye
• Alternating weekday and weekend matches

66
The ACC 1997/98 Problem (contd)

• No team can play away on both last dates.
• No team may have more than two away matches in a
  row.
• No team may have more than two home matches in a
  row.
• No team may have more than three away matches or
  byes in a row.
• No team may have more than four home matches or
  byes in a row.

67
The ACC 1997/98 Problem (contd)

• Of the weekends, each team plays four at home,
  four away, and one bye.
• Each team must have home matches or byes at least
  on two of the first five weekends.
• Every team except FSU has a traditional rival.
  The rival pairs are Clem-GT, Duke-UNC, UMD-UVA
  and NCSt-Wake. In the last date, every team
  except FSU plays against its rival, unless it
  plays against FSU or has a bye.

68
The ACC 1997/98 Problem (contd)

• The following pairings must occur at least once
  in dates 11 to 18 Duke-GT, Duke-Wake, GT-UNC,
  UNC-Wake.
• No team plays in two consecutive dates away
  against Duke and UNC. No team plays in three
  consecutive dates against Duke UNC and Wake.
• UNC plays Duke in last date and date 11.
• UNC plays Clem in the second date.
• Duke has bye in the first date 16.

69
The ACC 1997/98 Problem (contd)

• Wake does not play home in date 17.
• Wake has a bye in the first date.
• Clem, Duke, UMD and Wake do not play away in the
  last date.
• Clem, FSU, GT and Wake do not play away in the
  fist date.
• Neither FSU nor NCSt have a bye in the last date.
  
• UNC does not have a bye in the first date.

70
Nemhauser/Trick Solution

• Enumerate home/away/bye patterns
• explicit enumeration (very fast)
• Compute pattern sets
• integer programming (below 1 minute)
• Compute abstract schedules
• integer programming (several minutes)
• Compute concrete schedules
• explicit enumeration (approx. 24 hours)
• Schreuder, Combinatorial Aspects of Construction
  of Competition Dutch Football Leagues, Discr.
  Appl. Math, 35301-312, 1992.

71
Modeling ACC 97/98 as Constraint Satisfaction
Problem

• Variables
• 9 18 variables taking values from 0,1 that
  express which team plays home when. Example
  HUNC, 51 means UNC plays home on date 5.
• away, bye similar, e.g. AUNC, 5 or BUNC, 5
• 9 18 variables taking values from 0,1,...,9
  that express against which team which other team
  plays. Example ?UNC, 5 1 means UNC plays team 1
  (Clem) on date 5

72
Modeling ACC 97/98 as Constraint Satisfaction
Problem (contd)

• Constraints
• Example No team plays away on both last
  dates.
• AClem,17 AClem,18 lt 2, ADuke,17
  ADuke,18 lt 2, ...
• All constraints can be easily formalized in
  this manner.
• But Search leads to huge search trees!
• No solutions can be found within
  reasonable time!

73
First Step Back to Nemhauser/Trick!

• Constraint programming for generating all
  patterns.
• CSP representation straightforward.
• computing time below 1 second (Pentium II,
  233MHz)
• Constraint programming for generating all pattern
  sets.
• CSP representation straightforward.
• computing time 3.1 seconds

74
Back to Schreuder

• Constraint programming for abstract schedules
• Introduce variable matrix for abstract
  opponents similar to ? in naïve model
• there are many abstract schedules
• runtime over 20 minutes
• Constraint programming for concrete schedules
• model somewhat complicated, using two levels of
  the element constraint
• runtime several minutes

75
Cains Model

• Alternative to last two phases of Nemhauser/Trick
• Assign teams to patterns in a given pattern set.
• Assign opponent teams for each team and date.
• W.O. Cain, Jr, The computer-assisted heuristic
  approach used to schedule the major league
  baseball clubs, Optimal Strategies in Sports,
  North-Holland, 1977

76
Cain 1977
Schreuder 1992
77
Cains Model in CP Main Idea

• Remember matrices H, A, B represent patterns and
  matrix ? represents opponents
• Given matrices H, A, B of 0/1 values
  representing given pattern set.
• Vector p of variables ranging from 1 to n pGT
  identifies the pattern for team GT.
• Team GT plays according to pattern indicated by p
  on date 2 HpGT,2HGT,2
• Implemented element(pGT,H2 , HGT,2 )

78
Using Cains Model in CP

• Model for Cain simpler than model for Schreuder
• Runtimes
• patterns to teams 33 seconds
• opponent team assignment 20.7 seconds
• overall runtime for all 179 solutions 57.1
  seconds

79
Friar Tuck

• Constraint programming tool for sport scheduling,
  ACC 97/98 just one instance
• Convenient entry of constraints through GUI
• Friar Tuck is distributed under GPL
• Friar Tuck 1.1 available for Unix and Windows
  95/98 (www.comp.nus.edu.sg/henz/projects/FriarTuc
  k)
• Implementation language Oz using Mozart (see
  www.mozart-oz.org)

80
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

81
Constraint Programming Techniques

• Symmetry Breaking
• Redundant Constraints
• Parameterized Problems

82
Symmetry Breaking

• Often, the most efficient model admits
• many different solutions that are essentially
• the same (symmetric to each other).
• Symmetry breaking tries to improve the
• performance of search by eliminating
• such symmetries.

83
Redundant Constraints

• Pruning of original model is often not sufficient
• Adding redundant constraints sometimes helps

84
Example Grocery Puzzle

• A kid goes into a grocery store and buys four
  items. The cashier charges 7.11. The kid pays
  and is about to leave when the cashier calls the
  kid back, and says Hold on, I multiplied the
  four items instead of adding them Ill try
  again Gosh, with adding them the price still
  comes to 7.11! What were the prices of the four
  items?

85
Model for Grocery Puzzle

• Variables A, B, C, D represent prices of items in
  cents.
• Constraints
• A B C D 711
• A B C D 711 100 100 100

86
Additional Constraints

• Redundant constraint
• Symmetries

79 is prime factor of 711. Thus without loss of
generality A divisible by 79
B ? C ? D
87
Solution to Grocery Puzzle
Grocery
Grocery plus Redundancy
Grocery plus Redundancy plus Symmetry
88
Example Fractions

• Find distinct non-zero digits such that the
  following equation holds
• A D G
• 1
• B C E F H I

89
Model for Fractions

• One variable for each letter, similar to MONEY
• Constraint
• AEFHI DBCHI GBCEF
• BCEFHI

90
Additional Constraints
A D G ? ? B C E F H I

• Symmetries
• Redundant constraints

A 3 ? 1 B C
G 3 ? 1 H I
91
Fractions
Fractions plus Symmetries
Fractions plus Symmetries plus Redundancies

• Constraint
• AEFHI
• DBCHI
• GBCEF BCEFHI
• Symmetries
• AEF gt DBC
• DHI gt GEF
• Redundant Constraints
• 3A gt BC
• 3G lt HI

92
Redundant Constraints

• Adding redundant constraints sometimes results
  in dramatic performance improvements.

93
Performance of Symmetry Breaking

• All solution search Symmetry breaking usually
  improves performance often dramatically
• One solution search Symmetry breaking may or may
  not improve performance

94
Parameterized Problems

• Modeling facilities in Oz are embedded in
  advanced programming language.
• Problem scripts can be result of computation.
• Applications
• Embedding problem solving in complex applications
• Parameterized problems

Excursion N Queens
95
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling

96
Review

• Constraint programming is a framework for
  integrating three families of algorithms
• Propagation algorithms
• Branching algorithms
• Exploration algorithms

97
S E N D M O R E M O N E Y
S ? 9 E ? 4..7 N ? 5..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 4
E ? 4

S ? 9 E ? 5..7 N ? 6..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E 5
E ? 5
S ? 9 E ? 5 N ? 6 D ? 7 M ? 1 O ? 0 R
? 8 Y ? 2
S ? 9 E ? 6..7 N ? 7..8 D ? 2..8 M ?
1 O ? 0 R ? 2..8 Y ? 2..8
E ? 6
E 6


98
Optimization

• Modeling define optimization function
• Propagation algorithms identify propagation
  algorithms for optimization function
• Branching algorithms identify branching
  algorithms that lead to good solutions early
• Exploration algorithms extend existing
  exploration algorithms to achieve optimization

99
Optimization Example

• SEND MOST MONEY

100
SEND MOST MONEY

• Assign distinct digits to the letters
• S, E, N, D, M, O, T, Y
• such that
• S E N D
• M O S T
• M O N E Y holds and
• M O N E Y is maximal.

101
Modeling

• Formalize the problem as a constraint
  optimization problem
• Number of variables n
• Constraints c1,,cm ? ?n
• Optimization constraints d1,,dm ? ?n ?2?n Given
  a solution a, and an optimization constraint di ,
  the constraint di(a)? ?n contains only those
  assignments b for which b is better than a.

102
A Model for MONEY

• number of variables 8
• constraints
• c1 (S,E,N,D,M,O,T,Y)? ?8 0 ? S,,Y ? 9
• c2 (S,E,N,D,M,O,T,Y)? ?8
• 1000S 100E 10N D
• 1000M 100O 10S T
• 10000M 1000O 100N 10E Y

103
A Model for MONEY (continued)

• more constraints
• c3 (S,E,N,D,M,O,T,Y)? ?8 S ? 0
• c4 (S,E,N,D,M,O,T,Y)? ?8 M ? 0
• c5 (S,E,N,D,M,O,T,Y)? ?8 SY all
  different
• optimization constraint
  
• d (s,e,n,d,m,o,t,y) ?
• (S,E,N,D,M,O,T,Y)? ?8
• 10000m 1000o 100n 10e y
• lt 10000M 1000O 100N 10E Y

104
Propagation Algorithms

• Identify a propagation algorithm to implement the
  optimization constraints
• Example SEND MOST MONEY
• d (s,e,n,d,m,o,t,y) ?
• (S,E,N,D,M,O,T,Y))? ?8??8
• 10000m 1000o 100n 10e y
• lt 10000M 1000O 100N 10E Y
• Given a solution a, choose propagation algorithm
  for d(a).

105
Branching Algorithms

• Identify a branching algorithm that finds good
  solutions early.
• Example SEND MOST MONEY
• Idea Naïve enumeration in the order M, O,
  N, E, Y.
• Try highest values first.

106
Exploration Algorithms

• Modify exploration such that for each solution a,
  corresponding optimization constraints are added.
• Two strategies
• branch-and-bound continue as in original
  exploration
• restart optimization after each solution, start
  from the root.

107
Using Oz
proc Money Root S E N D M O R Y in
Rootsol(sS eE nN dD mM oO rR yY) Root
09 FD.distinct Root S \ 0 M \
0 1000S 100E 10N D
1000M 100O 10R E 10000M
1000O 100N 10E Y FD.distribute ff
Root end proc OptimizationConstraint Old New
sol(sS1 eE1 nN1 dD1 mM1 oO1 tT1 yY1)
Old sol(sS2 eE2 nN2 dD2 mM2 oO2 tT2
yY2) New in 10000M1 1000O1 100N1
10E1 Y1 lt 10000M2 1000O2 100N2
10E2 Y2 end Explorer.object script(Money
OptimizationConstraint)
108
Experiments with MONEY
MONEY Search
MONEY Optimization
109
Scheduling

• Scheduling Problems
• Propagation Algorithms for Resource Constraints
• Branching Algorithms
• Other Constraints
• Exploration Algorithms
• Literature

110
Scheduling Problems

• Assign starting times (and sometimes durations)
  to tasks, subject to
• resource constraints,
• precedence constraints,
• idiosyncratic and other constraints, and
• an optimization function, typically to minimize
  overall schedule duration.

111
Example Building a Bridge
Show Problem
Animate Solution
Gantt Chart

• Resource constraints
• Example a1 and a2 use excavator, cannot overlap
  in time
• Precedence constraints
• Example p1 requires a3, a3 must end before p1
  starts

112
Modeling

• Indices
• Tasks beg a1 a2 a3 a4 a5 a6 p1
• Constants duration of tasks
• Duration d(beg0 a14 a22...)
• Variables represent each task with a finite
  domain variable representing its starting time.
• Start FD.record start Tasks 0MaxTime
• Start.t is finite domain variable representing
  the starting time of task t

113
Precedence Constraints

• For each two tasks t1, t2, where t1 must precede
  t2, introduce a constraint
• Start.t1 Duration.t1 ? Start.t2

114
Resource Constraints

• For each two tasks t1, t2 that require a unary
  resource r, we have the constraint
• Start.t1 Duration.t1 ? Start.t2
• ?
• Start.t2 Duration.t2 ? Start.t1
• But many constraints and weak propagation.
• Thus, introduce global resource constraints.

115
Global Resource Constraints in Oz

• Schedule.serializedDisj
• a1 a2 a3 a4 a5 a6 excavator
• p1 p2 pile driver
• ...
• Start Duration
• User chooses among Schedule.serialized
• Schedule.serializedDisj
• Schedule.taskIntervals

116
Propagation Using Disjunction

• For all tasks t1, t2 using the same resource
• Start.t1 Duration.t1 ? Start.t2
• ?
• Start.t2 Duration.t2 ? Start.t1
• Weakest but fastest form of propagation for unary
  resources.

117
Propagation Edge finding

• For a given task t and set of tasks S, all
  sharing the same unary resource , find out
  whether t can occur
• before all tasks in S,
• after all tasks in S,
• between two tasks in S,
• and infer corresponding basic constraints.

118
Propagation Task Intervals

• Notation est(t) earliest starting time
• lct(t) latest completion time
• For two tasks t1 and t2 where est(t1)?est(t2) and
  lct(t1) ? lct(t2), the task interval I(t1,t2) is
  defined
• I(t1,t2) t est(t1) ? est(t) and
• lct(t) ? lct(t2)
• Perform edge finding on all task intervals.

119
Task Intervals Example
A
B
C
D
I(A,B) A,B,C I(C,D) B,C,D
120
Which Edge Finder?

• As usual, trade-off between run-time of
  propagation algorithm and strength of
  propagation. Edge finding can be expensive
  depending on which tasks are considered.
• Recent edge finders have complexity n2 for one
  propagation step, where n is the number of tasks
  using the resource.
• Edge finding based on task intervals can be
  stronger than these, but typically have
  complexity n3 .

121
Demo Propagation for Unary Resources
Bridge
122
Branching Algorithms Serializers

• Simple enumeration techniques such as first-fail
  are hopeless for scheduling.
• Use unary resources to guide branching.
• For two tasks t1, t2 sharing the same resource,
  use the constraints
• Start.t1 Duration.t1 ? Start.t2
• and
• Start.t2 Duration.t2 ? Start.t1
• for branching.

123
Which Tasks To Serialize?

• Resource-oriented serialization Serialize all
  tasks of one resource completely, before tasks of
  another resource are serialized.
• Most-used-resource serialization
• Global slack
• Local slack
• Task-oriented serialization Choose two suitable
  tasks at a time, regardless of resources.
• Slack-based task serialization (see later)

124
Most-used-resource Serialization

• Most-used-resource serialization serialize
  first the resource that is used the most.
• Let T be a set of tasks running on resource r.
• demand(T) ?t?T Duration.t
• Let S be the set of all tasks using resource r.
• demand(r) demand(S),
• Serialize the resource r with maximal demand(r)
  first.

125
Slack-based Resource Serialization

• Let T be a set of tasks running on resource r.
• supply(T) lct(T) - est(T)
• demand(T) ?t?T Duration.t
• slack(T) supply(T) - demand(T)
• Let S be the set of all tasks using resource r.
• slack(r) slack(S), S is set of all task
  running on r.
• Global slack serialization Serialize resource
  with smallest slack first

126
Local-Slack Resource Serialization

• Let Ir be all task intervals on r. The local
  slack is defined as min slack(I) I ? Ir
• Local slack serialization Serialize resource
  with smallest local slack first. Use global slack
  for tie-breaking.

127
Which Tasks Are Serialized?

• Ideas
• Use edge finding to look for possible first tasks
  (and last tasks)
• Choose tasks according to their est, lst
• 
  (lct, ect).

128
Task-oriented Serialization

• Among all tasks using all resources, select a
  pair of tasks according to local/global slack
  criteria and other considerations, regardless
  what resources are serialized already.
• Provides more fine-grained control at the expense
  of runtime for finding candidate task pairs among
  all tasks.

129
Demo Serialization Algorithms
Bridge
130
Exploration Algorithms

• Usually branch-and-bound using minimization of
  the starting time of a last task
• minimize aend.start subject to ...
• Lower-bounding Prove the non-existence of a
  solution with Start.last ? n, add constraint
  Start.last gt n increase n by ?.
• Upper-bounding Find solution with Start.last
  n, add constraint Start.last lt n decrease n by
  ?.

131
Literature on CP Scheduling

• Van Hentenryck The OPL optimization programming
  language, 1999
• Constraint Programming Tutorial of Mozart, 2000
  (www.mozart-oz.org)
• Applegate, Cook A computational study of the
  job-shop scheduling problem, 1991
• Carlier, Pinson An algorithm for solving the
  job-shop scheduling problem, 1989
• Various papers by Laburthe, Caseau, Baptiste, Le
  Pape, Nuijten, see Overview paper

132
Overview

• Constraint Programming in a Nutshell
• Elements of Constraint Programming
• Case study ACC 97/98 Basketball
• Constraint Programming Techniques
• Constraint-based Scheduling
• ???

133
Assessment Dont Use It!

• Dont use constraint programming for
• Problems for which there are known efficient
  algorithms or heuristics. Example Traveling
  salesman.
• Problems for which integer programming works
  well. Example Many discrete assignment problems.
• Problems with weak constraints and a complex
  optimization function. Example Timetabling
  problems.

134
Assessment Do Use It!

• Use constraint programming for
• Problems for which integer programming does not
  work (linear models too large).
• Problems for which there are no efficient
  solutions available.
• Problems with tight constraints, where
  propagation can be employed. Example ACC 97/98.
• Problems for which strong branching algorithms
  exist. Example Scheduling with unary resources.

135
Myths Debunked

• A fad! Constraint programming has been used
  successfully in a number of application areas,
  most spectacularly in scheduling
• Universal! More failure stories than success
  stories.
• All new! Many ideas come from AI search and
  Operations Research. In particular, the important
  ideas for scheduling come from OR.
• Artificial Intelligence! Mostly quite earthly
  algorithms. Constraint programming systems
  provide software framework for specialized
  algorithms to interact.

136
The Future

• Constraint programming will become a standard
  technique in OR for solving combinatorial
  problems, along with local search and integer
  programming.
• Constraint programming techniques will be tightly
  integrated with integer programming and local
  search.