CPSC 311 Section 502 Analysis of Algorithm

1
CPSC 311 Section 502Analysis of Algorithm

• Fall 2002
• Department of Computer Science
• Texas AM University

2
CPSC 311-502
Instructor O. Burchan Bayazit (pronounced as
bourch han) Office 407 E Harvey R. Bright
Bldg. Email burchanb_at_cs.tamu.edu Office Hours
400pm-530pm Tuesdays Teaching Assistant
Kasthuri Srinivasan Kannan Email
ksk7657_at_cs.tamu.edu Office Hours 400pm-530pm
Thursdays
3
Online Material
Course homepage http//parasol.tamu.edu/burchanb
/Courses/311/
Mailing List (announcements only)
cpsc-311-02c_at_listserv.tamu.edu
Discussion Group (announcementsdiscussions)
news//news.tamu.edu/tamu.classes.cpsc311
4
Course Information
Class Meeting 935-1050am, Tuesdays and
Thursdays ZACH 105B
Textbook Introduction to Algorithms, Cormen,
Leiserson, Rivest, Stein, McGraw-Hill (and MIT
Press), 2nd Edition (latest)
5
Prerequisites

• CPSC 211 (Data Structures) and Math 302 (Discrete
  Mathematics).
• In particular, you should be familiar with
• mathematical solutions to recurrence relations
• mathematical induction
• data structures including linked lists, arrays,
  trees, and graphs
• knowledge of a high level block structured
  language such as Pascal, C or Java

6
Mechanics

• Assignments (25)
• Homework (6-10)
• In-Class
• Programming Assignments
• CS Culture Activities
• Quizzes (15)
• Exams (60, 20 each)
• Two midterms and a final

7
OK, what about algorithms?
Algorithm a sequence of computational steps that
transform given input to the output
8
Course Goals

• At the end of the semester you should
• be familiar with fundamental algorithms and
  algorithmic techniques,
• given a particular application, be able to decide
  which algorithm among a set of possible choices
  is best,
• be able to prove correctness and analyze the
  running time of a given algorithm,
• be able to design efficient algorithms for new
  situations, using as building blocks the
  techniques learned,
• be able to prove a problem is NP-complete using
  reduction.

9
Course Outline

• Introduction and Mathematical Fundamentals - (Ch.
  1, 2, 3, 4 )
• Sorting - (Ch. 6, 7, 8)
• Selection - (Ch. 9)
• EXAM 1
• Hashing - (Ch. 11)
• Basic Graph Algorithms - (Ch. 22 )
• More Graph Algorithms - (Ch. 23, 24, 25)
• Union-Find/Disjoint Set Data Structure - (Ch. 21)
  
• Dynamic Programming - (Ch. 15)
• EXAM 2
• NP-Completeness - (Ch. 34, 35)
• Special Topics TBD
• EXAM 3 (FINAL) (Friday December 13, 1230-230pm)
  

10
Quiz Time !!!!!

• Fill out the personal information in the front
  page
• Answer the questions of the quiz in the back page

11
Answer to Question 1.
Shown binary search for 13
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
12
Answer to Question 1.
Find the middle
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
13
Answer to Question 1.
Select the right side (8lt13)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
14
Answer to Question 1.
Select the right side (8lt13)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
15
Answer to Question 1.
Find the middle
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Assume we have only integers (11 is also correct,
if you take floor of the mid-point)
16
Answer to Question 1.
Select right side (12lt13)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
17
Answer to Question 1.
Find the middle
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
18
Answer to Question 1.
Select the left side (13lt14)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
19
Answer to Question 1.
Find the middle 13
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
20
Answer to Question 2.
What is returned on input 20?
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
21
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x20
Stack whatami(20)
22
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x20
Stack whatami(20)
23
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x20
Stack whatami(20)
24
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x10
Stack whatami(10) whatami(20)
25
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x10
Stack whatami(10) whatami(20)
26
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x10
Stack whatami(10) whatami(20)
27
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x5
Stack whatami(5) whatami(10) whatami(20)
28
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x5
Stack whatami(5) whatami(10) whatami(20)
29
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x5
Stack whatami(5) whatami(10) whatami(20)
30
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x5
Stack whatami(5) whatami(10) whatami(20)
Return 2
31
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x10
Stack whatami(10) whatami(20)
Return 2
32
Answer to Question 2.
Subroutine whatami(integer x) if (x is
even) then return (whatami(x/2))
else return ((x-1)/2)
x20
Stack whatami(20)
Return 2
33
Answer to Question 3.
Show the link list after execution
d
e
a
b
c
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
tmp
34
Answer to Question 3.
d
e
a
b
c
tmp
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
35
Answer to Question 3.
d
e
a
b
c
tmp
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
36
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
37
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
38
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
39
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
40
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
41
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
42
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
43
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
44
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
45
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
46
Answer to Question 3.
d
e
a
b
c
tmp
tmp1
tmp2
tmp1next(next(tmp)) tmp2next(tmp1) next(tmp1)ne
xt(tmp2) next(tmp2)next(tmp) next(tmp)tmp2
47
Answer to Question 4.
Use induction to prove 12nn(n1)/2
48
Answer to Question 4.
Step 1 Verify that eq. holds for n1
Left hand side is 1 and right hand site
1(11)/2, so the equation holds for n1
49
Answer to Question 4.
Step 2 assume that eq. holds for nk
12k k(k1)/2
50
Answer to Question 4.
Step 3 using assumption at Step2, show that
eq. holds for nk1
From Step 2 12k k(k1)/2 If we add k1 to
each side 12.k (k1) k(k1)/2 (k1)
or 12.k (k1)
(k(k1) 2(k1))/2
51
Answer to Question 4.
Step 3 cont.
12.k (k1) (k(k1) 2(k1))/2
Becomes 12.k (k1) (k1)( k2))/2 Hence
Eq. Holds for nk1
52
Insertion Sort
1
2
3
4
5
6
53
Insertion Sort
sorted
Insert 2 into sorted part
54
Insertion Sort
sorted
2
After Insertion
55
Insertion Sort
sorted
2
Insert 4 into the sorted part
56
Insertion Sort
sorted
2
Insert 6 into the sorted part
57
Insertion Sort
sorted
2
Insert 1 into the sorted part
58
Insertion Sort
sorted
1
Insert 3 into the sorted part
59
Insertion Sort
sorted
6
1
2
3
4
5
60
Insertion Sort
1
2
3
4
5
61
Insertion Sort
How to insert? (assume we inserted previous
elements and we are at the last element)
j6 (elements position)
1
Start from ij-1
62
Insertion Sort
How to insert?
1
i5 If(arrayigtElement) switch Arrayi and
Array i1
63
Insertion Sort
How to insert?
1
i4 If(arrayigtElement) switch Arrayi and
Array i1
64
Insertion Sort
How to insert?
1
i3 If(arrayigtElement) switch Arrayi and
Array i1
65
Insertion Sort
How to insert?
1
i2 If(arrayiltElement) stop Insertion
66
Psuedo Algorithm for Insertion Sort
for j2 to Length(Array) keyArrayj
ij-1 while (igt0 and Aigtkey)
AI1AI ii-1
Ai1key
67
Psuedo Algorithm for Insertion Sort
cost
for j2 to Length(Array) keyArrayj
ij-1 while (igt0 and Aigtkey)
AI1AI ii-1
Ai1key
c1
c2
c3
c4
c5
c6
c7
68
Psuedo Algorithm for Insertion Sort
cost times
for j2 to Length(Array) keyArrayj
ij-1 while (igt0 and Aigtkey)
AI1AI ii-1
Ai1key
c1
n
n-1
c2
c3
n-1
c4
sum(2,n)(tj)
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
69
Cost of Insertion Sort
cost times
n
c1
n-1
c2
T(n)c1.nc2.(n-1)c2.(n-1) c4. sum(2,n)(tj)c5.
sum(2,n)(tj-1) c6. sum(2,n)(tj-1)c7.(n-1)
c3
n-1
c4
sum(2,n)(tj)
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
70
Best-Case cost
In the best case, the array is already sorted,
hence tj 1
cost times
n
c1
n-1
c2
c3
n-1
c4
sum(2,n)(tj)
T(n)c1.nc2.(n-1)c2.(n-1) c3.(n-1)c4.(n-1)c7.
(n-1) (c1c2c3c4c7)n-(c2c3c4c7)
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
T(n) is a linear function of n
71
Worst-Case cost
In the worst case, the array is already sorted in
the reverse order hence tj j
cost times
n
c1
n-1
c2
c3
n-1
c4
sum(2,n)(tj)
sum(2,n)(tj) becomes sum(2,n)(j)
n(n1)/2-1 sum(2,n)(tj-1) becomes sum(2,n)(j-1)
n(n-1)/2
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
T(n) is a linear function of n
72
Worst-Case cost
cost times
T(n)c1.nc2.(n-1)c3.(n-1)c4.(n.(n1)/2-1) c5.n
.(n-1)/2c6.n.(n-1)/2c7.(n-1) (c4c5c6)/2.n2
(c1c2c3(c4-c5-c6)/2c7).n (c2c3c4c7)
n
c1
n-1
c2
c3
n-1
c4
sum(2,n)(tj)
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
T(n) is a quadratic function of n
73
Average-Case cost
In the average case, we need the search half of
the sorted part of the array hence tj j/2
cost times
n
c1
n-1
c2
c3
n-1
c4
sum(2,n)(tj)
c5
sum(2,n)(tj-1)
sum(2,n)(tj-1)
c6
c7
n-1
T(n) is a quadratic function of n