Introduction to Algorithms

1
Introduction to Algorithms

• Jiafen Liu

Sept. 2013
2
About me

• TeacherLiu Jiafen
• Mail jfliu_at_swufe.edu.cn
• Research Interest
• Information security
• Formal Method and Verification of Correctness
• Data Mining and Business Intelligence
• Development of soft under iOS
• Homepage
• http//it.swufe.edu.cn/2011-09/24/2011092413430952
  61.html

3
Now its your turn

• Please introduce yourself
• Name
• Age
• Graduated School
• Specialty and Interest
• Background Investigation
• Have you studied data structure and other
  prerequisite courses of CS?
• Have you programmed in practice before?

4
About the Course

• Site
• http//fife.swufe.edu.cn/BILab/lectures/algo/algo.
  htmlLecture
• Notes can be found here.
• Related Resources can also be found on
  http//ocw.mit.edu/courses, you can also watch
  lectures online
• No Textbook

5
About the Course

• 16 weeks (the last 2 is reserved for exam)
• Site Tongbo Building 206
• Time Can we choose another proper time?
• Tuesday afternoon(5-7 or 6-8)?
• Tuesday night(10-12)?
• Thursday night(10-12)
• Or we have to split into 2 period(first 2 in
  classroom, and last 1 on computers)?

6
Grading

• 20 Attendance
• 50 Performance
• 1 presentation at least (10)
• assignments (30)
• 30 Final paper

7
Course Objectives

• What is an algorithm?
• Whats the difference between algorithm and
  program?
• Why we need this course?
• How to evaluate a algorithm?
• How can we design a good algorithm?
• How to make an algorithm implemented?

8
What is an algorithm?

• Algorithm is a problem-solving method or process.
• It helps us to understand scalability.
• Algorithm satisfy the following properties
• Input
• Output
• Unambiguous
• Finite
• Feasible

9
Algorithm and Program

• Program is an implementation of algorithm using a
  programming language.
• Algorithm design takes the most important place
  in Software Engineering.
• Programmer VS Coder?
• How to become an excellent programmer?

10
What is important for a program?
11
Why we need performance?

• Performance is the currency of computing.
• Example C family VS Java
• This course makes theoretical study of computer
  program performance and resource usage.
• How to evaluate an algorithm?
• How can we design a good algorithm?
• How to make an algorithm implemented?

12
The problem of sorting

• Input sequence lta1, a2, , angt of numbers.
• Output permutation lta1', a2', , an'gt such that
  a1' a2' an'.
• Example
• Input 8 2 4 9 3 6
• Output 2 3 4 6 8 9
• How can we do that?
• Recall the process of playing cards.

13
Insertion sort

• 8 2 4 9 3 6

14
Insertion sort

• 8 2 4 9 3 6
• 2 8 4 9 3 6

15
Insertion sort

• 8 2 4 9 3 6
• 2 8 4 9 3 6
• 2 4 8 9 3 6

16
Insertion sort

• 8 2 4 9 3 6
• 2 8 4 9 3 6
• 2 4 8 9 3 6
• 2 4 8 9 3 6

17
Insertion sort

• 8 2 4 9 3 6
• 2 8 4 9 3 6
• 2 4 8 9 3 6
• 2 4 8 9 3 6
• 2 3 4 8 9 6

18
Insertion sort

• 8 2 4 9 3 6
• 2 8 4 9 3 6
• 2 4 8 9 3 6
• 2 4 8 9 3 6
• 2 3 4 8 9 6
• 2 3 4 6 8 9

Done!
19
Insertion sort
20
Running time

• The running time depends on the input an already
  sorted sequence is easier to sort.
• Since short sequences are easier to sort than
  long ones, we will parameterize the running time
  by the size of the input.

21
Kinds of analyses

• Worst-case (usually)
• T(n) maximum time of algorithm on any input of
  size n.
• Average-case (sometimes)
• T(n) expected time of algorithm over all inputs
  of size n.
• Need assumption of statistical distribution of
  inputs.
• Best-case (bogus)
• Cheat with a slow algorithm that works fast on
  some input.

22
Machine-independent time

• Generally, we seek upper bounds on the running
  time, Why?
• Because everybody likes a guarantee.
• What is insertion sorts worst-case time?
• It depends on the speed of our computer
• relative speed (on the same machine)
• absolute speed (on different machines)

23
Asymptotic Analysis
Big Idea

• Ignore machine-dependent constants.
• Look at growth of T(n) as n?8.

24
T-notation

• Math
• T(g(n)) f(n) there exist positive constants
  c1, c2, and n0 such that 0 c1g(n) f (n)
  c2g(n) for all n n0
• Engineering
• Drop low-order terms
• Ignore leading constants.
• Example 3n3 90n25n 6046
• T(n3)

25
T-notation

• T-notation is fabulous because it satisfies our
  issue of being able to compare both relative and
  absolute speed.

26
Asymptotic performance

• When n gets large enough, a T (n2) algorithm
  always beats a T(n3) algorithm.
• We shouldnt ignore asymptotically slower
  algorithms, however.
• Real-world design situations often call for a
  careful balancing of engineering objectives.
• Asymptotic analysis is a useful tool to help to
  structure our thinking.

27
Insertion sort analysis

• Assume that every elemental operation is going to
  take some constant amount of time.
• Worst case Input reverse sorted.
• 
  
• 
  arithmetic series
• Average case All permutations equally likely.

28
About T-notation

• It is more of a descriptive notation than it is a
  manipulative notation.
• Is insertion sort a fast sorting algorithm?
• Moderately so, for small n.
• Not at all, for large n.

29
Another method Merge sort

• Merging two sorted arrays
• 20 12
• 13 11
• 7 9
• 2 1
• Time T(n) to merge a total of n
• elements (linear time).

20
13
12
11
9
7
2
1
30
Merge Sort
T(n) T(1) 2T(n/2)
T(n) T(1) Abuse, but we can ignore it. 2T(n/2)
Should be , but it
turns out not to matter asymptotically.
31
Cost for merge sort

• We shall usually omit stating the base case when
  T(n) T(1) because it has no effect on the
  asymptotic solution to the recurrence.
• Lecture 2 will provide several ways to find a
  good upper bound on T(n), for example, Recursion
  Tree.

32
Recursion Tree

• We can write as T(n) 2T(n/2) cn, where c gt 0
  and it is constant.
• T(n)

33
Recursion Tree

• We can write as T(n) 2T(n/2) cn, where c gt 0
  and it is constant.
• cn
• T(n/2)
  T(n/2)

34
Recursion Tree

• We can write as T(n) 2T(n/2) cn, where c gt 0
  and it is constant.
• cn
• cn/2
  cn/2
• T(n/4) T(n/4) T(n/4)
  T(n/4)

35
Recursion Tree

• We can write as T(n) 2T(n/2) cn, where c gt 0
  and it is constant.
• cn
• cn/2
  cn/2
• T(n/4) T(n/4) T(n/4)
  T(n/4)
• T(1)

Height ?
logn1
n
(leaves) ?
Total cnlognT(n)
T(n logn)
36
Conclusion

• T(nlgn) grows more slowly than T(n2) .
• Therefore, merge sort asymptotically beats
  insertion sort in the worst case.
• In practice, merge sort beats insertion sort for
  n gt 30 or so.
• Go test it out for yourself!

37
Homework

• Read Chapter1 and Chapter 2.
• Recall another sort method Bubble sort and
  express with pseudocode, then compute its cost
  using T-notation (Hand in paper edition the next
  class).

38
Have FUN !