Declarative Programming Techniques

About This Presentation

Title:

Declarative Programming Techniques

Description:

– PowerPoint PPT presentation

Number of Views:65

Avg rating:3.0/5.0

Slides: 175

Provided by: infoU

Category:

more less

Transcript and Presenter's Notes

Title: Declarative Programming Techniques

1
Declarative Programming Techniques

Seif Haridi
KTH
Peter Van Roy
UCL

2
Overview

What is declarativeness?
Classification, advantages for large and small
programs
Iterative and recursive programs
Programming with lists and trees
Lists, accumulators, difference lists, trees,
parsing, drawing trees
Reasoning about efficiency
Time and space complexity, big-oh notation,
recurrence equations
Higher-order programming
Basic operations, loops, data-driven techniques,
laziness, currying
User-defined data types
Dictionary, word frequencies
Making types secure abstract data types
The real world
File and window I/O, large-scale program
structure, more on efficiency
Limitations and extensions of declarative
programming

3
Declarative operations (1)

An operation is declarative if whenever it is
called with the same arguments, it returns the
same results independent of any other computation
state
A declarative operation is
Independent (depends only on its arguments,
nothing else)
Stateless (no internal state is remembered
between calls)
Deterministic (call with same operations always
give same results)
Declarative operations can be composed together
to yield other declarative components
All basic operations of the declarative model are
declarative and combining them always gives
declarative components

4
Declarative operations (2)
rest of computation
5
Why declarative components (1)

There are two reasons why they are important
(Programming in the large) A declarative
component can be written, tested, and proved
correct independent of other components and of
its own past history.
The complexity (reasoning complexity) of a
program composed of declarative components is the
sum of the complexity of the components
In general the reasoning complexity of programs
that are composed of nondeclarative components
explodes because of the intimate interaction
between components
(Programming in the small) Programs written in
the declarative model are much easier to reason
about than programs written in more expressive
models (e.g., an object-oriented model).
Simple algebraic and logical reasoning techniques
can be used

6
Why declarative components (2)

Since declarative components are mathematical
functions, algebraic reasoning is possible i.e.
substituting equals for equals
The declarative model of chapter 4 guarantees
that all programs written are declarative
Declarative components can be written in models
that allow stateful data types, but there is no
guarantee

7
Classification ofdeclarative programming
Descriptive
Declarativeprogramming
Observational
Functional programming
Programmable
Declarative model
Deterministiclogic programming
Definitional

The word declarative means many things to many
people. Lets try to eliminate the confusion.
The basic intuition is to program by defining the
what without explaining the how

Nondeterministiclogic programming
8
Descriptive language
?s? skip
empty statement ?x? ?y?

variable-variable binding
?x?
?record? variable-value binding

?s1? ?s2? sequential composition local
?x? in ?s1? end declaration
Other descriptive languages include HTML and XML
9
Descriptive language
ltperson id 530101-xxxgt ltnamegt Seif
lt/namegt ltagegt 48 lt/agegt lt/persongt
Other descriptive languages include HTML and XML
10
Kernel language
The following defines the syntax of a statement,
?s? denotes a statement
?s? skip
empty statement ?x? ?y?

variable-variable binding
?x?
?v? variable-value binding
?s1?
?s2? sequential composition local ?x?
in ?s1? end declaration proc ?x? ?y1?
?yn? ?s1? end procedure introduction if
?x? then ?s1? else ?s2? end conditional
?x? ?y1? ?yn? procedure
application case ?x? of ?pattern? then ?s1?
else ?s2? end pattern matching
11
Why the KL is declarative

All basic operations are declarative
Given the components (substatements) are
declarative,
sequential composition
local statement
procedure definition
procedure call
if statement
try statement
are all declarative

12
Structure of this chapter

Iterative computation
Recursive computation
Thinking inductively
Lists and trees

data abstraction
procedural abstraction

Control abstraction
Higher-order programming

User-defined data types
Secure abstract data types

Modularity
Functors and modules

Time and space complexity
Nondeclarative needs
Limits of declarative programming

13
Iterative computation

An iterative computation is a one whose execution
stack is bounded by a constant, independent of
the length of the computation
Iterative computation starts with an initial
state S0, and transforms the state in a number of
steps until a final state Sfinal is reached

14
The general scheme

fun Iterate Si
if IsDone Si then Si
else Si1 in
Si1 Transform Si
Iterate Si1
end
end
IsDone and Transform are problem dependent

15
The computation model

STACK RIterate S0
STACK S1 Transform S0, RIterate S1
STACK RIterate S1
STACK Si1 Transform Si, RIterate
Si1
STACK RIterate Si1

16
Newtons method for thesquare root of a positive
real number

Given a real number x, start with a guess g, and
improve this guess iteratively until it is
accurate enough
The improved guess g is the average of g and x/g

17
Newtons method for thesquare root of a positive
real number

Given a real number x, start with a guess g, and
improve this guess iteratively until it is
accurate enough
The improved guess g is the average of g and
x/g
Accurate enough is defined as x g2 / x
lt 0.00001

18
SqrtIter

fun SqrtIter Guess X
if GoodEnough Guess X then Guess
else Guess1 Improve Guess X in
SqrtIter Guess1 X
end
end
Compare to the general scheme
The state is the pair Guess and X
IsDone is implemented by the procedure GoodEnough
Transform is implemented by the procedure Improve

19
The program version 1

fun Sqrt X
Guess 1.0
in SqrtIter Guess X
end
fun SqrtIter Guess X
if GoodEnough Guess X then Guess
else
SqrtIter Improve Guess X X
end
end

fun Improve Guess X (Guess
X/Guess)/2.0 end fun GoodEnough Guess X Abs
X - GuessGuess/X lt 0.00001 end
20
Using local procedures

The main procedure Sqrt uses the helper
procedures SqrtIter, GoodEnough, Improve, and
Abs
SqrtIter is only needed inside Sqrt
GoodEnough and Improve are only needed inside
SqrtIter
Abs (absolute value) is a general utility
The general idea is that helper procedures should
not be visible globally, but only locally

21
Sqrt version 2

local
fun SqrtIter Guess X
if GoodEnough Guess X then Guess
else SqrtIter Improve Guess X X end
end
fun Improve Guess X
(Guess X/Guess)/2.0
end
fun GoodEnough Guess X
Abs X - GuessGuess/X lt 0.000001
end
in
fun Sqrt X
Guess 1.0
in SqrtIter Guess X end
end

22
Sqrt version 3

Define GoodEnough and Improve inside SqrtIter
local
fun SqrtIter Guess X
fun Improve
(Guess X/Guess)/2.0
end
fun GoodEnough
Abs X - GuessGuess/X lt 0.000001
end
in
if GoodEnough then Guess
else SqrtIter Improve X end
end
in fun Sqrt X
Guess 1.0 in
SqrtIter Guess X
end
end

23
Sqrt version 3

Define GoodEnough and Improve inside SqrtIter
local
fun SqrtIter Guess X
fun Improve
(Guess X/Guess)/2.0
end
fun GoodEnough
Abs X - GuessGuess/X lt 0.000001
end
in
if GoodEnough then Guess
else SqrtIter Improve X end
end
in fun Sqrt X
Guess 1.0 in
SqrtIter Guess X
end
end

The program has a single drawback on each
iteration two procedure values are created, one
for Improve and one for GoodEnough
24
Sqrt final version

fun Sqrt X
fun Improve Guess
(Guess X/Guess)/2.0
end
fun GoodEnough Guess
Abs X - GuessGuess/X lt 0.000001
end
fun SqrtIter Guess
if GoodEnough Guess then Guess
else SqrtIter Improve Guess end
end
Guess 1.0
in SqrtIter Guess
end

The final version is a compromise
between abstraction and efficiency
25
From a general schemeto a control abstraction (1)

fun Iterate Si
if IsDone Si then Si
else Si1 in
Si1 Transform Si
Iterate Si1
end
end
IsDone and Transform are problem dependent

26
From a general schemeto a control abstraction (2)

fun Iterate S IsDone Transform
if IsDone S then S
else S1 in
S1 Transform S
Iterate S1
end
end

fun Iterate Si if IsDone Si then Si else
Si1 in Si1 Transform Si Iterate
Si1 end end
27
Sqrt using the Iterate abstraction

fun Sqrt X
fun Improve Guess
(Guess X/Guess)/2.0
end
fun GoodEnough Guess
Abs X - GuessGuess/X lt 0.000001
end
Guess 1.0
in
Iterate Guess GoodEnough Improve
end

28
Sqrt using the Iterate abstraction

fun Sqrt X
Iterate
1.0
fun G Abs X - GG/X lt 0.000001 end
fun G (G X/G)/2.0 end
end

This could become a linguistic abstraction
29
Recursive computations

Recursive computation is one whose stack size
grows linear to the size of some input data
Consider a secure version of the factorial
function
fun Fact N
if N0 then 1
elseif Ngt0 then NFact N-1
else raise domainError end
end
end
This is similar to the definition we saw before,
but guarded against domain errors (and looping)
by raising an exception

30
Recursive computation

proc Fact N R
if N0 then R1
elseif Ngt0 then R1 in
Fact N-1 R
R NR1
else raise domainError end
end
end

31
Execution stack

Fact 5 r0
Fact 4 r1 , r05 r1
Fact 3 r2, r14 r2 , r05 r1
Fact 2 r3, r23 r3 , r14 r2 , r05 r1
Fact 1 r4, r32 r4 , r23 r3 , r14 r2 ,
r05 r1
Fact 0 r5, r41 r5, r32 r4 , r23 r3 ,
r14 r2 , r05 r1
r51, r41 r5, r32 r4 , r23 r3 , r14 r2 ,
r05 r1
r41 1, r32 r4 , r23 r3 , r14 r2 , r05
r1
r32 1 , r23 r3 , r14 r2 , r05 r1
r23 2 , r14 r2 , r05 r1

32
Substitution-basedabstract machine

The abstract machine we saw in Chapter 4 is based
on environments
It is nice for a computer, but awkward for hand
calculation
We make a slight change to the abstract machine
so that it is easier for a human to calculate
with
Use substitutions instead of environments
substitute identifiers by their store entities
Identifiers go away when execution starts we
manipulate store variables directly

33
Iterative Factorial

State Fact N
(0,Fact 0) ?? (1,Fact 1) ??? ??? (N,Fact
N)
In general (I,Fact I)
Termination condition is I equal to Nfun
IsDone I FI I N end
Transformation (I,Fact I) ??? (I1,
(I1)Fact I) proc Transform I FI I1
FI1 I1 I1 FI1 I1FIend

34
Iterative Factorial

State Fact N
(0,Fact 0) ?? (1,Fact 1) ??? ??? (N,Fact
N)
Transformation (I,Fact I) ??? (I1,
(I1)Fact I)
fun Fact N fun FactIter I FI if IN then
FI else FactIter I1 (I1)FI
end end FactIter 0 1end

35
Iterative Factorial

State Fact N
(0,Fact 0) ?? (1,Fact 1) ??? ??? (N,Fact
N)
Transformation (I,Fact I) ??? (I1,
(I1)Fact I)
fun Fact N Iterate t(0 1) fun t(I
IF) I N end fun t(I IF) J I1 in t(J
JIF) end end

36
Iterative Factorial

State Fact N
(1,5) ?? (15,4) ??? ??? (Fact N,0)
In general (I,J)
Invariant IFact J (IJ)Fact J-1 Fact
N
Termination condition is J equal to 0fun
IsDone I J I 0 end
Transformation (I,J) ??? (IJ, J-1) proc
Transform I J I1 J1 I1 IJ J1
J1-1end

37
Programmingwith lists and trees

Defining types
Simple list functions
Converting recursive to iterative functions
Deriving list functions from type specifications
State and accumulators
Difference lists
Trees
Drawing trees

38
User defined data types

A list is defined as a special subset of the
record datatype
A list Xs is either
XXr where Xr is a list, or
nil
Other subsets of the record datatype are also
useful, for example one may define a binary tree
(btree) to be
node(keyK valueV leftLT rightRT) where LT and
BT are binary trees, or
leaf
This begs for a notation to define concisely
subtypes of records

39
Defining types

?list? ?value? ?list? nil
defines a list type where the elements can be of
any type
?list T? T ?list? nil
defines a type function that given the type of
the parameter T returns a type, e.g. ?list ?int?
?
?btree T? node(key ?literal? valueT left
?btree T? right ?btree T?)
leaf(key ?literal? valueT)
Procedure types are denoted by procT1 Tn
Function types are denoted by funT1 TnT and
is equivalent to procT1 Tn T
Examples fun?list? ?list? ?list?

40
Lists

General Lists have the following definition?list
T? T ?list? nil
The most useful elementary procedures on lists
can be found in the Base module List of the
Mozart system
Induction method on lists, assume we want to
prove a property P(Xs) for all lists Xs
The Basis prove P(Xs) for Xs equals to nil, X,
and X Y
The Induction step Assume P(Xs) hold, and prove
P(XXs) for arbitrary X of type T

41
Constructive method forprograms on lists

General Lists have the following definition?list
T? T ?list T? nil
The task is to write a program Task Xs1 Xsn
Select one or more of the arguments Xsi
Construct the task for Xsi equals to nil, X,
and X Y
The recursive step assume Task Xsi is
constructed, and design the program for Task
XXsi for arbitrary X of type T

42
Simple functions on lists

Some of these functions exist in the library
module List
Nth Xs N, returns the Nth element of Xs
Append Xs Ys, returns a list which is the
concatenation of Xs followed by Ys
Reverse Xs returns the elements of Xs in a
reverse order, e.g. Reverse 1 2 3 is 3 2 1
Sorting lists, MergeSort
Generic operations of lists, e.g. performing an
operation on all the elements of a list,
filtering a list with respect to a predicate P

43
The Nth function

Define a function that gets the Nth element of a
list
Nth is of type fun ?list T? ?int??T? ,
Reasoning select N, two cases N1, and Ngt1
N1 Nth Xs 1 ? Xs.1
Ngt1 assume we have the solution for Nth Xr N-1
for a smaller list Xr, then Nth XXr N ? Nth
Xr N-1
fun Nth Xs NXXr Xs in
if N1 then X
elseif Ngt1 then Nth Xr N-1
end
end

44
The Nth function

fun Nth Xs NXXr Xs in
if N1 then X
elseif Ngt1 then Nth Xr N-1
end
end

fun Nth Xs Nif N1 then Xs.1
elseif Ngt1 then Nth Xs.2 N-1
end
end

45
The Nth function

Define a function that gets the Nth element of a
list
Nth is of type fun ?list T? ?int??T? ,
fun Nth Xs N
if N1 then Xs.1
elseif Ngt1 then Nth Xs.2 N-1
end
end
There are two situations where the program fails
N gt length of Xs, (we get a situation where Xs is
nil) or
N is not positive, (we get a missing else
condition)
Getting the nth element takes time proportional
to n

46
The Member function

Member is of type fun ?value? ?list
?value???bool? ,
fun Member E Xs
case Xs
of nil then false
XXr then
if XE then true else Member E Xr end
end
end
XE orelse Member E Xr is equivalent to
if XE then true else Member E Xr end
In the worst case, the whole list Xs is
traversed, i.e., worst case behavior is the
length of Xs, and on average half of the list

47
The Append function

fun Append Xs Ys
case Xs
of nil then Ys
XXr then XAppend Xr Ys
end
end
The inductive reasoning is on the first argument
Xs
Appending Xs and Ys is proportional to the length
of the first list
declare Xs0 1 2 Ys a b Zs0 Append
Xs0 Ys
Observe that Xs0, Ys0 and Zs0 exist after Append
A new copy of Xs0, call it Xs0, is constructed
with an unbound variable attached to the end
12X, thereafter X is bound to Ys

48
The Append function

proc Append Xs Ys Zs
case Xs
of nil then Zs Ys
XXr then Zr in Zs XZr
Append Xr Ys Zr
end
end
declare Xs0 1 2 Ys a b Zs0 Append
Xs0 Ys
Observe that Xs0, Ys and Zs0 exist after Append
A new copy of Xs0, call it Xs0, is constructed
with an unbound variable attached to the end
12X, thereafter X is bound to Ys

49
Append execution (overview)

Stack Append 12nil a b zs0 Store zs0,
...
Stack Append 2nil a b zs1 Store zs0
1zs1, zs1, ...
Stack Append nil a b zs2 Store zs0
1zs1, zs12zs2, zs2, ...
Stack zs2 a b Store zs0 1zs1,
zs12zs2, zs2, ...
Stack Store zs0 1zs1, zs12zs2,
zs2 abnil, ...

50
Reverse

fun Reverse Xs
case Xs
of nil then nil
XXr then Append Reverse Xr X
end
end
Xs0 1 2 3 4

reverse of Xs1
4 3 2
Xs1
append 4 3 2 and 1
51
Length

fun Length Xs
case Xs
of nil then 0
XXr then 1Length Xr
end
end
Inductive reasoning on Xs

52
Merging two sorted lists

Merging two sorted lists
Merge 3 5 10 2 5 6 ? 2 3 5 5 6 10
funMerge ?list T? ?list T? ?list T? , where T
is either ?int?, ?float?, or ?atom?
fun Merge Xs Ys
case Xs Ys
of nil Ys then Ys
Xs nil then Xs
(XXr) (YYr) then
if X lt Y then XMerge Xr Ys
else YMerge Xs Yr endend
end

53
Sorting with Mergesort

MergeSort ?list T? ?list T? T is either
?int?, ?float?, ?atom?
MergeSort uses a divide-and-conquer strategy
Split the list into two smaller lists of roughly
equal size
Use MergeSort (recursively) to sort the two
smaller lists
Merge the two sorted lists to get the final result

54
Sorting with Mergesort
L11
S11
L1
S1
split
merge
L12
S12
L
split
S
merge
L21
S21
L2
split
merge
S2
L22
S22
55
Sorting with Mergesort

MergeSort ?list T? ?list T? T is either
?int?, ?float?, ?atom?
MergeSort uses a divide-and-conquer strategy
Split the list into two smaller lists of roughly
equal size
Use MergeSort (recursively) to sort the two
smaller lists
Merge the two sorted lists to get the final
result
fun MergeSort Xs
case Xs of nil then nil X then Xselse Ys
Zs in
Split Xs Ys Zs Merge MergeSort Ys
MergeSort Zsend
end

56
Split

proc Split Xs Ys Zs
case Xs
of nil then Ys nil Zs nil X then Ys
Xs Zs nil X1X2Xr then Yr Zr in
Ys X1Yr Zs X2Zr Split Xr Yr Zrend
end

57
Exercise

The merge sort, described above is an example of
divide and conquer problem-solving strategy.
Another simpler (but less efficient) is insertion
sort. This sorting algorithm is defined as
follows
To sort a list Xs of zero or one element, return
Xs
To sort a list Xs of the form X1X2Xr,
sort X2Xr to get Ys
insert X1 in the right position in Ys, to get the
result

58
Converting recursiveinto iterative computation

Consider the view of state transformation
instead S0? S1? S2?...?Si ? ... ? Sf
At S0 no elements are counted
At S1 one elements is counted
At Si i elements are counted
At Sn where n is final state all elements are
counted
The state is in general a pair (I, Ys)
I is the length of the initial prefix of the list
Xs that is counted, and Ys is the suffix of Xs
that remains to count

Consider Length fun ltlist T gtltintgt fun
Length Xs case Xs of nil then 0
XXr then 1Length Xr end end
59
Converting recursiveinto iterative computation

The state is in general a pair (I, Ys)
I is the length of the initial prefix of the list
Xs that is counted, and Ys is the suffix of Xs
that remains to count

fun IterLength I Xs case Xs of nil then
I XXr then IterLength I1 Xr
end end fun Length Xs IterLength 0 Xs end
Xs
x1 x2 ... xi xi1 ... xn
Ys

Assume Ys is YYr
(I, Y Yr) ? (I1, Yr)
Length Xs I Length Y Yr (I1)
Length Yr ... NLength nil

60
State invariants

Given a state SI that is (I, Ys) the property
P(SI) defined as Length Xs I Length Yr is
called state invariant

fun IterLength I Xs case Xs of nil then
I XXr then IterLength I1 Xr
end end fun Length Xs IterLength 0 Xs end
Xs
x1 x2 ... xi xi1 ... xn
Ys

Induction using state invariants
prove P(S0) (0, Xs)
assume P(SI) , prove P(SI1)
conclusion for all I, P(SI) holds (in particular
P(Sfinal) holds

61
Reverse

fun Reverse Xs
case Xs
of nil then nil
XXr then Append Reverse Xr X
end
end
This program does a recursive computation
Let us define another program that is iterative
(Initial list is Xs)
(nil , x1 x2 ... xi xi1 ... xn nil)
(x1nil , x2 ... xi xi1 ... xn)
(x2 x1nil , ... xi xi1 ... xn)

62
Reverse

fun IterReverse Rs Xs
case Xs
of nil then Rs
XXr then IterReverse XRs Xr
end
end
fun Reverse Xs IterReverse nil Xs end

63
Nested lists

?nestedList T? nil T
?nestedList T? ?nestedList T?
?nestedList T?
we also add the condition that T ? nil, and T ?
XXr for some X and Xr
Given an input Ls of type ?nestedList T? count
the number of elements in Ls
Induction of the type ?nestedList T?

64
Nested lists

fun IsCons Xs case Xs of XXr then true else
false end end
fun IsLeaf X Not IsCons X andthen X \ nil
end
fun LengthL Xs
case Xs
of nil then 0
XXr andthen IsLeaf X then
1 LengthL Xr
XXr andthen Not IsLeaf X then
LengthL X LengthL Xr
end
end

65
Nested lists

fun Flatten Xs
case Xs
of nil then nil
XXr andthen IsLeaf X then
XFlatten Xr
XXr andthen Not IsLeaf X then
Append Flatten X Flatten Xr
end
end

66
Accumulators

Accumulator programming is a way to handle state
in declarative programs. It is a programming
technique that uses arguments to carry state,
transform the state, and pass it to the next
procedure.
Assume that the state S consists of a number of
components to be transformed individually
S (X,Y,Z,...)
For each procedure P, each state component is
made into a pair, the first component is the
input state and the second components is the
output state after P has terminated
S is represented as (Xin, Xout,Yin, Yout,Zin,
Zout,...)

67
Accumulators

Assume that the state S consists of a number of
components to be transformed individually
S (X,Y,Z,)
Assume P1 to Pn are procedures

accumulator

proc P X0 X Y0 Y Z0 Z P1 X0 X1 Y0 Y1 Z0
Z1 P2 X1 X2 Y1 Y2 Z1 Z2 Pn Xn-1 X Yn-1
Y Zn-1 Z end
The procedural syntax is easier to use if there
is more than one accumulator

68
Example

Consider a variant of MergeSort with accumulator
proc MergeSort1 N S0 S Xs
N is an integer,
S0 is an input list to be sorted
S is the remainder of S0 after the first N
elements are sorted
Xs is the sorted first N elements of S0
The pair (S0, S) is an accumulator
The definition is in a procedural syntax because
it has two outputs S and Xs

69
Example (2)

fun MergeSort Xs
MergeSort1 Length X Xs _ Ys
Ys
end

proc MergeSort1 N S0 S Xs if N0 then S
S0 Xs nil elseif N 1 then XS S0 X
else N gt 1 S1 Xs1 Xs2 NL N div 2
NR N - NL MergeSort1 NL S0 S1 Xs1
MergeSort1 NR S1 S Xs2 Xs Merge Xs1
Xs2 end end
70
Multiple accumulators

Consider a stack machine for evaluating
arithmetic expressions
Example (14)-3
The machine executes the following instructions
push(1)push(4)pluspush(3)minus

1
5
5
2
4
3
71
Multiple accumulators (2)

Example (14)-3
The arithmetic expressions are represented as
trees
minus(plus(1 4) 3)
Write a procedure that takes arithmetic
expressions represented as trees and output a
list of stack machine instructions and counts the
number of instructions
proc ExprCode Expr Cin Cout Nin Nout
Cin initial list of instructions
Cout final list of instructions
Nin initial count
Nout final count

72
Multiple accumulators (3)

proc ExprCode Expr C0 C N0 N
case Expr
of plus(Expr1 Expr2) then C1 N1 in
C1 plusC0
N1 N0 1
SeqCode Expr2 Expr1 C1 C N1 N
minus(Expr1 Expr2) then C1 N1 in
C1 minusC0
N1 N0 1
SeqCode Expr2 Expr1 C1 C N1 N
I andthen IsInt I then
C push(I)C0
N N0 1
end
end

73
Multiple accumulators (4)

proc ExprCode Expr C0 C N0 N
case Expr
of plus(Expr1 Expr2) then C1 N1 in
C1 plusC0
N1 N0 1
SeqCode Expr2 Expr1 C1 C N1 N
minus(Expr1 Expr2) then C1 N1 in
C1 minusC0
N1 N0 1
SeqCode Expr2 Expr1 C1 C N1 N
I andthen IsInt I then
C push(I)C0
N N0 1
end
end

proc SeqCode Es C0 C N0 N case Es of nil
then C C0 N N0 EEr then N1 C1 in
ExprCode E C0 C1 N0 N1 SeqCode Er C1 C
N1 N end end
74
Shorter version (4)

proc ExprCode Expr C0 C N0 N
case Expr
of plus(Expr1 Expr2) then
SeqCode Expr2 Expr1 plusC0 C N0 1 N
minus(Expr1 Expr2) then
SeqCode Expr2 Expr1 minusC0 C N0 1 N
I andthen IsInt I then
C push(I)C0
N N0 1
end
end

proc SeqCode Es C0 C N0 N case Es of nil
then C C0 N N0 EEr then N1 C1 in
ExprCode E C0 C1 N0 N1 SeqCode Er C1 C
N1 N end end
75
Functional style (4)

fun ExprCode Expr t(C0 N0)
case Expr
of plus(Expr1 Expr2) then
SeqCode Expr2 Expr1 t(plusC0 N0 1)
minus(Expr1 Expr2) then
SeqCode Expr2 Expr1 t(minusC0 N0 1)
I andthen IsInt I then
t(push(I)C0 N0 1)
end
end

fun SeqCode Es T case Es of nil then T
EEr then T1 ExprCode E T in
SeqCode Er T1 end end
76
The lecture

Based on Chapter 5 (updated version exist on the
web page)
Difference lists
Programming with trees
Higher order programming
Modularity and program structure
Stand-alone applications

77
Difference lists (1)

A difference list is a pair of lists, each might
have an unbound tail, with the invariant that the
one can get the second list by removing zero or
more elements from the first list
X X Represent the empty list
nil nil idem
a a idem
(abcX) X Represents a b c
a b c d d idem

78
Difference lists (2)

When the second list is unbound, an append
operation with another difference list takes
constant time
fun AppendD D1 D2 S1 E1 D1 S2 E2
D1in E1 S2 S1 E2end
local X Y in Browse AppendD (123X)X
(45Y)Y end
Displays (12345Y)Y

79
A FIFO queuewith difference lists (1)

A FIFO queue is a sequence of elements with an
insert and a delete operation.
Insert adds an element to one end and delete
removes it from the other end
Queues can be implemented with lists. If L
represents the queue content, then inserting X
gives XL and deleting X gives ButLast L X (all
elements but the last).
Delete is inefficient it takes time proportional
to the number of queue elements
With difference lists we can implement a queue
with constant-time insert and delete operations
The queue content is represented as q(N S E),
where N is the number of elements and SE is a
difference list representing the elements

80
A FIFO queue with difference lists (2)

Inserting b
In q(1 aT T)
Out q(2 abU U)
Deleting X
In q(2 abU U)
Out q(1 bU U) and Xa
Difference list allows operations at both ends
N is needed to keep track of the number of queue
elements

fun NewQueue X in q(0 X X) end fun Insert Q
X case Q of q(N S E) then E1 in EXE1 q(N1 S
E1) end end proc Delete Q Q1 X case Q of q(N
S E) then S1 in XS1S Q1 q(N-1 S1 E)
end end fun EmptyQueue case Q of q(N S E) then
N0 end end
81
Flatten (revisited)

fun Flatten Xs
case Xs
of nil then nil
XXr andthen IsLeaf X then
XFlatten Xr
XXr andthen Not IsLeaf X then
Append Flatten X Flatten Xr
end
end

Remember the Flatten function we wrote before?
Let us replace lists by difference lists and see
what happens.
82
Flatten with difference lists (1)

Flatten of nil is XX
Flatten of XXr is Y1Y where
flatten of X is Y1Y2
flatten of Xr is Y3Y
equate Y2 and Y3
Flatten of a leaf X is (XY)Y

Here is what it looks like as text
83
Flatten with difference lists (2)

proc FlattenD Xs Ds
case Xs
of nil then Y in Ds YY
XXr then Y0 Y1 Y2 in Ds Y0Y2
Flatten X Y0Y1 Flatten Xr Y1Y2
X andthen IsLeaf X then Y in (XY)Y
end
end
fun Flatten Xs Y in FlattenD Xs Ynil Y end

Here is the new program. It is much more
efficient than the first version.
84
Reverse (revisited)

Here is our recursive reverse
Rewrite this with difference lists
Reverse of nil is XX
Reverse of XXs is Y1Y, where
reverse of Xs is Y1Y2, and
equate Y2 and XY

fun Reverse Xs case Xs of nil then nil
XXr then Append Reverse Xr X end end
85
Reverse with difference lists (1)

The naive version takes time proportional to the
square of the input length
Using difference lists in the naive version makes
it linear time
We use two arguments Y1 and Y instead of Y1Y
With a minor change we can make it iterative as
well

fun ReverseD Xsproc ReverseD Xs Y1 Y case
Xs of nil then Y1Y XXs then Y2 in
ReverseD Xr Y1 Y2 Y2
XY endendR in ReverseD Xs R nilR end
86
Reverse with difference lists (2)

fun ReverseD Xsproc ReverseD Xs Y1 Y case
Xs of nil then Y1Y XXs then
ReverseD Xr Y1 XY endendR in
ReverseD Xs R nilR
end

87
Difference lists summary

Difference lists are a way to represent lists in
the declarative model such that one append
operation can be done in constant time
A function that builds a big list by
concatenating together lots of little lists can
usually be written efficiently with difference
lists
The function can be written naively, using
difference lists and append, and will be
efficient when the append is expanded out
Difference lists are declarative, yet have some
of the power of destructive assignment
Because of the single-assignment property of the
dataflow variable
Difference lists originate from Prolog

88
Trees

Next to lists, trees are the most important
inductive data structure
A tree is either a leaf node or a node containing
one or more subtrees
While a list has a linear structure, a tree can
have a branching structure
There are an enormous number of different kinds
of trees. Here we will focus on one kind,
ordered binary trees. Later on we will see other
kinds of trees.

89
Ordered binary trees

?btree T1? tree(keyT1 value ?value? ?btree
T1? ?btree T1? ) leaf
Binary each non-leaf node has two subtrees
Ordered keys of left subtree lt key of node lt
keys of right subtree

key3 valuex
key1 valuey
key5 valuez
leaf
leaf
leaf
leaf
90
Search trees

Search tree A tree used for looking up,
inserting, and deleting information
Let us define three operations
Lookup X T returns the value corresponding to
key X
Insert X V T returns a new tree containing the
pair (X,V)
Delete X T returns a new tree that does not
contain key X

91
Looking up information
fun Lookup X T case T of leaf then
notfound tree(keyY valueV T1 T2) andthen
XY then found(V) tree(keyY valueV T1
T2) andthen XltY then Lookup X T1
tree(keyY valueV T1 T2) andthen XgtY
then Lookup X T2 end end

There are four cases
X is not found
X is found
X might be in the left subtree
X might be in the right subtree

92
Inserting information

There are four cases
(X,V) is inserted immediately
(X,V) replaces an existing node with same key
(X,V) is inserted in the left subtree
(X,V) is inserted in the right subtree

fun Insert X V T case T of leaf then
tree(keyX valueV leaf leaf) tree(keyY
valueW T1 T2) andthen XY then tree(keyX
valueV T1 T2) tree(keyY valueW T1 T2)
andthen XltY then tree(keyY valueW Insert X V
T1 T2) tree(keyY valueW T1 T2) andthen
XgtY then tree(keyY valueW T1 Insert X V T2)
end end
93
Deleting information (1)

There are four cases
(X,V) is not in the tree
(X,V) is deleted immediately
(X,V) is deleted from the left subtree
(X,V) is deleted from the right subtree
Right? Wrong!

fun Delete X T case T of leaf then
leaf tree(keyY valueW T1 T2) andthen
XY then leaf tree(keyY valueW T1 T2)
andthen XltY then tree(keyY valueW Delete X
T1 T2) tree(keyY valueW T1 T2) andthen
XgtY then tree(keyY valueW T1 Delete X T2)
end end
94
Binary tree deletion
A
A
95
Binary tree deletion
A
A
X
?
Deleting X from a binary tree. The problem is how
to patch up the tree after X is removed
96
Binary tree deletion
remove X
transfer Y
X
Y
Y
Filling the gap after removal of X
97
Deleting information (2)
fun Delete X T case T of leaf then
leaf tree(keyY valueW T1 T2) andthen
XY then case RemoveSmallest T2 of none then
T1 YpWpTp then tree(keyYp
valueWp T1 Tp) end tree(keyY valueW T1
T2) andthen XltY then tree(keyY valueW Delete
X T1 T2) tree(keyY valueW T1 T2)
andthen XgtY then tree(keyY valueW T1 Delete X
T2) end end

The problem with the previous program is that it
does not correctly delete a non-leaf node
To do it right, the tree has to be reorganized
A new element has to take the place of the
deleted one
It can be the smallest of the right subtree or
the largest of the left subtree

98
Deleting information (3)

To remove the root node Y, there are two
possibilities
One subtree is a leaf. Result is the other
subtree.
Neither subtree is a leaf. Result is obtained by
removing an element from one of the subtrees.

Y
T1
T1
leaf
Y
Yp
T1
T2
T1
Tp
99
Deleting information (4)
fun RemoveSmallest T case T of leaf
then none tree(keyX valueV T1 T2) then
case RemoveSmallest T2 of
none then XVT2 XpVpTp then
XpVptree(keyX valueV T1 Tp) end
end end

The function RemoveSmallest T removes the
smallest element from T and returns the triple
XpVpTp, where (Xp,Vp) is the smallest element
and Tp is the remaining tree

100
Deleting information (5)

Why is deletion complicated?
It is because the tree satisfies a global
condition, namely that it is ordered
The deletion operation has to work to maintain
the truth of this condition
Many tree algorithms rely on global conditions
and have to work hard to maintain them

101
Depth-first traversal

An important operation for trees is visiting all
the nodes
There are many orders in which the nodes can be
visited
The simplest is depth-first traversal, in which
one subtree is completely visited before the
other is started
Variants are infix, prefix, and postfix traversals

fun DFS T case T of leaf then skip
tree(keyX valueV T1 T2) then DFS
T1 Browse XV DFS T2 end end
102
Breadth-first traversal
fun BFS T fun TreeInsert Q T
if T\leaf then Insert Q T else Q end end
proc BFSQueue Q1 if EmptyQueue
Q1 then skip else X Q2Delete Q1 X
tree(keyK valueV L R)X
in Browse KV
BFSQueue TreeInsert TreeInsert Q2 R L
end end in BFSQueue TreeInsert
NewQueue T end

A second basic traversal order is breadth-first
In this order, all nodes of depth n are visited
before nodes of depth n1
The depth of a node is the distance to the root,
in number of hops
This is harder to implement than depth-first it
uses a queue of subtrees

103
Balanced trees

We look at ordered binary trees that are
approximately balanced
A tree T with n nodes is approximately balanced
if the height of T is at most c.log2(n1), for
some constant c
Insertion and deletion operations are in the
order of O(log n)
Red-black trees preserve this property the
height of a RB-tree is at most 2.log2(n1) where
n is the number of nodes in the tree

104
Red Black trees

Red Black trees have the following properties
Every node is either red or black
If a node is red, then both its children are
black
Each path from a node to all descendant leaves
contains the same number of black nodes
It follows that the shortest path from the root
to a leaf has all black nodes and is of length
log(n1) for a tree of n nodes
The longest path has alternating black-red nodes
and of length 2.log(n1)
Insertion and deletion algorithms preserve the
balanced tree property

105
Example
26
41
17
47
30
21
14
19
10
28
16
23
38
7
12
11
106
Insertion at leaf
10
10
or
10
?
5
12
10
10
or
10
?
12
5
Has to be fixed
107
Insertion at internal node (left)
Insertion may lead to violation of red-black
property
12
12
5
10
10
6
??
108
Insertion at internal node (left)
Correction is done by rotation (to the right in
this case)
Right Rotate
Z
Y
Y
X
Z
T4
X
T1
T2
T3
T3
T4
T1
T2
109
Insertion at internal node (left)
Correction is done by rotation (to the right in
this case)
Right Rotate
Z
Y
X
T4
X
Z
T1
Y
T1
T2
T3
T4
T2
T3
110
Insertion at internal node (right)
Left Rotate
X
Y
T1
Y
X
Z
Z
T2
T1
T2
T3
T4
T3
T4
111
Insertion at internal node (right)
Left Rotate
X
Y
T1
Z
X
Z
Y
T4
T1
T2
T3
T4
T3
T2
112
Program

ltcolorgt red black
lttreegt nil node(ltcolorgt ltintgt lttreegt
lttreegt)
declare
fun Insert X T
node(C Y T1 T2) InsertRot X T
in node(black Y T1 T2)
end

113
Program

fun InsertRot X T
case T
of nil then node(red X nil nil)
node(C Y T1 T2) andthen X Y then T
node(C Y T1 T2) andthen X lt Y then
T3 InsertRot X T1 in
RotateRight node(C Y T3 T2)
node(C Y T1 T2) andthen X gt Y then
T3 InsertRot X T2 in
RotateLeft node(C Y T1 T3)
end
end

114
Insertion at internal node (left)
fun RotateRight T node(C Z LT T4) T in
case LT of node(red Y node(red X T1 T2) T3)
then node(red Y node(black X T1 T2)
node(black Z T3 T4)) node(red X T1 node(red
Y T2 T3)) then node(red Y node(black X
T1 T2) node(black Z T3 T4)) else T end end
Right Rotate
Z
Y
Y
T4
X
Z
X
T3
T3
T4
T1
T2
T1
T2
115
Insertion at internal node (right)
fun RotateLeft T node(C X T1 RT) T in
case RT of node(red Z node(red Y T2 T3) T4)
then node(red Y node(black X T1 T2)
node(black Z T3 T4)) node(red Y T2 node(red
Z T3 T4)) then node(red Y node(black X
T1 T2) node(black Z T3 T4)) else T end end
Left Rotate
X
Y
Z
T1
X
Z
Y
T4
T3
T4
T1
T2
T3
T2
116
Drawing trees
117
Reasoning about efficiency

Computational efficiency is two things
Execution time needed (e.g., in seconds)
Memory space used (e.g., in memory words)
The kernel language its semantics allow us to
calculate the execution time up to a constant
factor
For example, execution time is proportional to
n2, if input size is n
To find the constant factor, it is necessary to
measure what happens in the implementation (e.g.,
wall-clock time)
Measuring is the only way that really works
there are so many optimizations in the hardware
(caching, super-scalar execution, virtual memory,
...) that calculating exact time is almost
impossible
Let us first see how to calculate the execution
time up to a constant factor

118
Big-oh notation

We will give the computational efficiency of a
program in terms of the big-oh notation
O(f(n))
Let T(n) be a function that is the execution time
of some program, measured in the size of the
input n
Let f(n) be some function defined on nonnegative
integers
T(n) is of O(f(n)) if
T(n) ? c.f(n) for some positive constant
c,except for some small values of n ? n0
Sometimes this is written T(n)O(f(n)). Be
careful!
If g(n)O(f(n)) and h(n)O(f(n)) then it is not
true that g(n)h(n)!

119
Calculating execution time

We will use the kernel language as a guide to
calculate the time
Each kernel operation has an execution time (that
may be a function of the size of its
arguments)
To calculate the execution time of a program
Start with the function definitions written in
the kernel language
Calculate the time of each function in terms of
its definition
Assume each functions time is a function of the
size of its input arguments
This gives a series of recurrence equations
There may be more than one equation for a given
function, e.g., to handle the base case of a
recursion
Solving these recurrence equations gives the
execution time complexity of each function

120
Kernel language execution time
Execution time T(s) of each kernel language
operation ?s?
?s? skip
k ?x? ?y?
k ?x? ?v?
k ?s1? ?s2? T(s1) T(s2) local
?x? in ?s1? end kT(s1) proc ?x? ?y1?
?yn? ?s1? end k if ?x? then ?s1? else
?s2? end k max(T(s1), T(s2)) ?x? ?y1?
?yn? Tx(size(I(y1,,yn)) case ?x?
of ?pattern? then ?s1? else ?s2? end k
max(T(s1), T(s2))

Each instance of k is a different positive real
constant
size(I(y1,,yn)) is the size of the input
arguments, defined as we like
In some special cases, ?x? ?y? and ?x? ?v?
can take more time

121
Example Append

proc Append Xs Ys Zs
case Xs
of nil then Zs Ys XXr then Z Zr in Zs
ZZr Append Xr Ys Zrend
end

Recurrence equation for recursive call
TAppend(size(I(Xs,Ys,Zs))) k1max(k2,
k3TAppend(size(I(Xr,Ys,Zr)))
TAppend(size(Xs)) k1 max(k2,
k3TAppend(size(Xr))
TAppend(n) k1max(k2, k3TAppend(n-1)
TAppend(n) k4TAppend(n-1)
Recurrence equation for base case
TAppend(0) k5
Solution
TAppend(n) k4.n k5 O(n)

122
Recurrence equations

A recurrence equation is of two forms
Define T(n) in terms of T(m1), , T(mk), where
m1, , mk lt n.
Give T(n) directly for certain values of n (e.g.,
T(0) or T(1)).
Recurrence equations of many different kinds pop
up when calculating a programs efficiency, for
example
T(n) k T(n-1) (T(n) is of O(n))
T(n) k1 k2.n T(n-1) (T(n) is of O(n2))
T(n) k T(n/2) (T(n) is of O(log n))
T(n) k 2.T(n/2) (T(n) is of O(n))
T(n) k1 k2.n 2.T(n/2) (T(n) is of O(n.log
n))
Let us investigate the most commonly-encountered
ones
For the others, we refer you to one of the many
books on the topic

123
Example FastPascal

Recursive case
TFP(n) k1 max(k2, k3TFP(n-1)k4.n)
TFP(n) k5k4.nTFP(n-1)
Base case
TFP(1) k6
Solution method
Assume it has form a.n2b.nc
This gives three equations in three unknowns
k4-2.a0
k5a-b0
k6abc
It suffices to see that this is solvable and a?0
Solution TFP(n) is of O(n2)

fun FastPascal N if N1 then 1 else
local L in LFastPascal N-1
AddList ShiftLeft L ShiftRight L end
end end
124
Example Mergesort

Recursive case
TMS(size(Xs)) k1 k2.n T(size(Ys))
T(size(Zs))
TMS(n) k1 k2.n T(?n/2? ) T(?n/2?)
TMS(n) k1 k2.n 2.T(n/2) (assuming n is a
power of 2)
Base cases
TMS(0) k3
TMS(1) k4
Solution
TMS(n) is of O(n.log n)(can show it also holds
if n is not a power of 2)
Do you believe this?
Run the program and measure it!

fun MergeSort Xs case Xs of nil then
nil X then Xselse Ys Zs in Split Xs Ys
Zs Merge MergeSort Ys MergeSort
Zsend end
125
Memory space

Watch out! There are two very different concepts
Instantaneous active memory size (memory words)
How much memory the program needs to continue
executing successfully
Calculated by reachability (memory reachable from
semantic stack)
Instantaneous memory consumption (memory
words/sec)
How much memory the program allocates during its
execution
Measure for how much work the memory management
(e.g., garbage collection) has to do
Calculate with recurrence equations (similar to
execution time complexity)
These two concepts are independent

126
Calculatingmemory consumption
Memory space M(s) of each kernel language
operation ?s?
?s? skip
0 ?x? ?y?
0 ?x? ?v?
memsize(v) ?s1? ?s2? M(s1)
M(s2) local ?x? in ?s1? end 1M(s1)
if ?x? then ?s1? else ?s2? end max(M(s1),
M(s2)) ?x? ?y1? ?yn? Mx(size(I(y1
,,yn)) case ?x? of ?pattern? then ?s1? else
?s2? end max(M(s1), M(s2))

size(I(y1,,yn)) is the size of the input
arguments, defined as we like
In some cases, if, case, and ?x? ?v? take less
space

127
Memory consumption of bind

memsize(v)
integer 0 for small integers, else proportional
to integer size
float 0
list 2
tuple 1n (where nlength(arity(v)))
other records (where n length(arity(v)))
Once for each different arity in the system k.n,
where k depends on the run-time system
For each record instance 1n
procedure kmemsize(s) where ltsgt is the
procedure body and k depends on the compiler
memsize(s)
Roughly proportional to number of statements and
identifiers. Exact value depends on the compiler.

128
Higher-order programming

Higher-order programming the set of programming
techniques that are possible with procedure
values (lexically-scoped closures)
Basic operations
Procedural abstraction creating procedure values
with lexical scoping
Genericity procedure values as arguments
Instantiation procedure values as return values
Embedding procedure values in data structures
Control abstractions
Integer and list loops, accumulator loops,
folding a list (left and right)
Data-driven techniques
List filtering, tree folding
Explicit lazy evaluation, currying
Later chapters higher-order programming is the
foundation of component-based programming and
object-oriented programming

129
Procedural abstraction

Procedural abstraction is the ability to convert
any statement into a procedure value
A procedure value is usually called a closure, or
more precisely, a lexically-scoped closure
A procedure value is a pair it combines the
procedure code with the environment where the
procedure was created (the contextual
environment)
Basic scheme
Consider any statement ltsgt
Convert it into a procedure value P proc
ltsgt end
Executing P has exactly the same effect as
executing ltsgt

130
Procedural abstraction

fun AndThen B1 B2
if B1 then B2 else false
end
end

131
Procedural abstraction

fun AndThen B1 B2
if B1 then B2 else false
end
end

132
A common limitation

Most popular imperative languages (C, C, Java)
do not have procedure values
They have only half of the pair variables can
reference procedure code, but there is no
contextual environment
This means that control abstractions cannot be
programmed in these languages
They provide a predefined set of control
abstractions (for, while loops, if statement)
Generic operations are still possible
They can often get by with just the procedure
code. The contextual environment is often empty.
The limitation is due to the way memory is
managed in these languages
Part of the store is put on the stack and
deallocated when the stack is deallocated
This is supposed to make memory management
simpler for the programmer on systems that have
no garbage collection
It means that contextual environments cannot be
created, since they would be full of dangling
pointers