Basic Concepts

1
Basic Concepts

• 2008, Fall
• Pusan National University
• Ki-Joune Li

2
Data Structures ?

• Refrigerator Problem
• If we have only one item in my refrigerator, no
  problem.
• If we have several items in a refrigerator,
• How to place milk, eggs, kimchi, beers, etc. in a
  refrigerator ?
• How to measure the efficiency ?
• Placement and cooking
• Placement something about materials
• Cooking something about activities or processes
• Some Organization or Structures
• Data Structures
• How to place data in memory

3
Measuring the Efficiency of Data Structures

• What is a good placement in refrigerator ?
• Easy to cook (and place)
• What is a good data structure ?
• Easy to process
• Data Structures (Placement of materials)
  Algorithms (Cooking)
• Program (Meal)
• Good Data Structures support good algorithms

4
Abstract Data Type

• Object-Oriented Programming
• Object ?
• Abstraction (or Encapsulation)
• Hiding the internal details
• Implementation
• Internal mechanism and process
• Only provide Interfaces
• Abstract Data Type
• Hiding the internal structures once it has been
  implemented
• Provide only the interface to the users

5
Example of Abstract Data Types

• Example
• ADT

ADT NaturalNumber is functions for all x, y
in NaturalNumber true, false in Boolean, and
where ,-,lt,, and are integer operations
Zero() NaturalNumber 0 IsZero(x)
Boolean if(x0) isZerotrue else
isZerofalse Add(x,y) NaturalNumber
if(xyltMAXINT) Addxy else ADDMAXINT End
NaturalNumber
6
Algorithms

• Algorithm
• A sequence of instructions with specifications of
  
• Input
• Output at least one output
• Definiteness Clear instructions
• Finiteness
• Effectiveness
• Abstract description of a program
• Can be easily converted to a program
• Correctness of algorithm

7
Performance Analysis

• What is a good algorithm ?
• Correctness
• Good documentation and readable code
• Proper structure
• Effective
• How to measure the effectiveness of an algorithm
  ?
• Space complexity
• Amount of memory it needs to run
• Time complexity
• Amount of time (mostly CPU time) it needs to run

8
Space Complexity

• Notation
• Space complexity, f (n) function of input size
  n
• How should the constants be determined ?
• Is it meaningful to count the number of bytes ?

f (n) c1c1 constant
f (n) c2an ?
f (n) c3 nwhy ?
9
Time Complexity

• Notation
• Time complexity, f (n) function of input size n

f (n) 2 ?
f (n) 3 n 1 ?
f (n) (2 ? ) n
Is it really meaningful to determine these
constants ?
10
Asymptotic Notation

• Exact time (or step count) to run
• Depends on machines and implementation (program)
• NOT good measure
• More general (but inexact) notation Asymptotic
  Notation
• Big-O O(n) Upper Bound
• Omega-O ?(n) Lower Bound
• Theta-O ?(n) More precise than Big-O and
  Omega-O
• Big-O notation is the most popular one.

11
Big-O notation

• Definition ( f of n is big-O of g of n)
• f (n) ? O(g (n)) ? there exist c and n0 (c, n0
  gt0) such thatf (n) ? cg(n), for all n (? n0)
• Example
• 3n 2 O(n), 3n 2 O(n2)
• Time complexity of the following algorithm f
  (n) O(n)

int sumArray(int n, int a) if(nlt0) return
0return an-1sumArrary(n-1,a)
12
Big-O notation Some Properties

• Classification
• O(1) constant, O(n) Linear, O(n2) Quadratic,
  O(2n) exponential
• Polynomial function
• If f (n) amnm am-1nm-1 a1n a0, then f
  (n) ? O(nm)
• Big-O is determined by the highest order
• Only the term of the highest order is of our
  concern
• Big-O useful for determining the upper bound of
  time complexity
• When only an upper bound is known, Big-O notation
  is useful
• In most cases, not easy to find the exact f (n)
• Big-O notation is the most used.

13
Omega-O notation

• Definition ( f of n is Omega-O of g of n)
• f (n) ? ?(g (n)) ? there exist c and n0 (c, n0
  gt0) such thatf (n) ? cg(n), for all n (? n0)
• Example
• 3n 2 ?(n), 3n 2 ?(1), 3n2 2 ?(n),
• Time complexity of the following algorithm f (n)
  ?(n)
• If f (n) amnm am-1nm-1 a1na0, then f (n)
  ? ?(nm)
• Omega-O is determined by the highest order
• Omega-O notation useful to describe the lower
  bound

int sumArray(int n, int a) if(nlt0) return
0return an-1sumArrary(n-1,a)
14
Theta-O notation

• Definition ( f of n is theta-O of g of n)
• f (n) ? ?(g (n)) ? there exist c1, c2 and n0 (c1,
  c2, and n0 gt0) such that
• c1g(n) ? f (n) ? c2g(n), for all n (? n0)
• Example
• 3n 2 ?(n), 3n 2 ? ?(1), 3n2 2 ? ?(n),
• Time complexity of the following algorithm f (n)
  ?(n)
• If f (n) amnm am-1nm-1 a1na0, then f (n)
  ? ?(nm)
• Theta-O is determined by the highest order
• Theta-O
• Possible Only if f (n)?(g(n)), and f(n)?(g(n))
• Lower bound and Upper bound is the same

int sumArray(int n, int a) if(nlt0) return
0return an-1sumArrary(n-1,a)
15
Complexity Analysis

• Worst-Case Analysis
• Time complexity for the worst case
• Example
• Linear Search f (n) n
• Average Analysis
• Average time complexity
• f (n) p1f1(n) p2f2(n) pkfk(n),
• where pi is the probability for the i -th case.
• Not easy to find pi
• In most cases, only worst-case analysis
• Why not Best-Case Analysis ?

16
Example Worst-Case Time complexity of Binary
Search

• Binary Search
• Big-O O(n2), O(log n)
• Omega O ?(1), ?(log n)
• Theta O ?(log n)

int BinarySearch(int v, int a, int lower, int
upper) // search m among a sorted array a1ower,
alower1, aupper if(lltu) int
m(lowerupper)/2 if(vgtam), return
BinarySearch(v,a,m1,upper) else
if(vltam) return BinarySearch(v,a,lower,m-1)
else / vam / return m return
-1 // not found
17
Comparison O(1), O(log n), O(n), O(n2), O(2n)

• Graph
• In general
• Algorithm of O(1) is almost impossible
• Algorithms of O(log n) is excellent
• Algorithms of O(n) is very good
• Algorithms of O(n2) is not bad
• Algorithms of O(n3) is acceptable
• Algorithms of O(2n) is not useful