Algorithms and Programs

1
Algorithms and Programs

• CS101B1 Intro to CS
• Monday, 03/19/2001
• Course homepage http//cs-people.bu.edu/prgrsso
  r/teaching/cs101s01.html Course mailing list
  cs101b1_at_cs.bu.edu
• Instructors e-mail prgrssor_at_cs.bu.edu

2
Overview

• Describe the process of conceiving, designing,
  writing, debugging, and maintaining a computer
  program
• Distinguish between the various programming
  languages
• Appreciate the problems faced by software
  engineers in trying to produce large, reliable
  systems

3
Problems, Algorithms and Programs

• An Algorithm is the method one has to use to
  solve a Problem.
• A Program is the precise expression of an
  algorithm in a language that could be followed by
  a computer.

4
Software Development Process

• Understand the Problem
• Come up with an Algorithm to solve it
• Flesh out the Algorithm in a program

5
Software in Perspective Systems Analysis and
Software Lifecycle

• Investigation why is there a problem?
• Analysis what is the problem?
• Design how can the problem be solved?
• Development programmers develop programs
• Implementation the system is put to work
• Maintenance ongoing upgrades/bugfixes
• Retirement phasing out the current system

6
What is an Algorithm?

• An algorithm is a methodical approach to solving
  a problem
• Examples
• An algorithm to multiply X and Y is to add X to
  itself Y times
• An algorithm to compute the average of N numbers
  is to add them up and then divide by N

7
The Makeup of an Algorithm

• An algorithm is made up of a set of basic steps
  that are followed sequentially
• Each algorithm must have a single entry point
  from which the algorithm is started
• Each algorithm must have one or more exit
  points that indicate when the algorithm is
  finished

8
What Constitutes a Basic Algorithmic Step?

• Input a value into a memory variable
• Output a value from a memory variable
• Assigning and reassigning a value to a variable
• Performing a math or some other operation on a
  number of memory variables
• Deciding the next step based on the outcome of a
  yes/no question
• Decision/choice

9
Visualizing Memory

• Until now, talked about memory as mailboxes
  with addresses
• Variables are place holders where values could
  be memorized
• Each variable has a Name and a Value
• A variable could have different values at
  different times.
• X 19
• Y 23
• Z 16
• nameStan Rost
• emailprgrssor_at_cs.bu.edu

10
Memory Assignment Operation

• A memory assignment allows the calculation of a
  value and the storage of that value in a
  variable.
• X Y Z
• X 19
• Y 23
• Z 16
• name Stan Rost
• email prgrssor_at_cs.bu.edu

11
A Case Study Find the Maximum in a List of s

• Problem
• You are given a list of 100 graded homeworks and
  you are asked to find the one with the highest
  grade.
• How would you do it?
• Hint Your technique should work even if the list
  is arbitrarily long!

12
A Case Study Find the Maximum in a List of s

• Problem
• You are given a list of 100 graded homeworks and
  you are asked to find the one with the highest
  grade.
• Algorithm
• Go through the papers (one at a time) while
  memorizing the highest grade you have encountered
  so far.

13
How Do We Express an Algorithm?

• Write it in English
• Example add X to itself Y times
• Disadvantages
• English is ambiguous
• Very difficult to check if two algorithms are
  similar
• Why English and not (say) Japanese?
• Hard to visualize---especially when algorithms
  are complex

14
Expressing an Algorithm Using Flowcharts

• Symbols and their meanings

BEGIN
YES
COMPUTE
CONDITION
END
OUTPUT
NO
STORE
CONNECTOR
INPUT
PRINT
15
From Algorithm to a Program

• Flowcharts are aimed at humans they help
  programmers visualize the algorithm to be
  followed to solve a problem.
• Flowcharts are not suitable for computers to
  execute because they are not detailed and
  specific enough.
• To represent an algorithm to be followed by a
  computer, we have to express that algorithm as a
  program.

16
Programming and Programming Languages

• Programs are precise instructions expressed in
  a programming language.
• The only instructions that computer hardware can
  follow (or execute) are very low level machine
  instructions.
• Writing machine-language programs is an arduous
  task that would make programming very difficult
  (and rather impractical).

17
The Languages of Computers

• Low-level languages
• Difficult to learn and use.
• Examples include machine and assembly languages.
• High-level Languages
• English-like vocabulary.
• Examples include C, Java, Pascal, Fortran, etc.

18
From High-Level to Low-Level Languages

• The process of translating a high-level program
  (suitable for humans) into a low-level program
  (suitable for the computer) could be programmed!
  
• We only have to write the translation program
  once Once written, it can be used over and over
  to convert from high-level to low-level programs.

19
Interpreters and Compilers

• To translate a program into machine language
  requires translator software, such as
• Compilers the entire program is translated into
  machine language before execution begins.
• Interpreters each instruction is translated
  individually when needed during execution.
• Much more on programming languages later

20
Algorithm Design Divide and Conquer Concept

• A complex problem like writing a computer game
  needs to be broken into three parts
• For example
• begin the game
• repeat players turn until the player guesses
  right answer or seven turns are completed
• end the game
• Further refinements may further divide and
  conquer main pieces into smaller ones and/or
  add more details to each part.

21
Algorithm Design Case StudySorting a List of
Numbers

• Problem
• You are given a list of 100 numbers (or names)
  and you are asked to SORT this list in an
  descending order...
• How would you do it?
• Hint Your technique should work even if the list
  is arbitrarily long!

22
Algorithm Design Case StudySorting a List of
Numbers(2)

• Solution
• Find the largest number by going through the
  list
• Find the second largest number by going through
  the remaining list of numbers...
• Find the third largest number by going through
  the remaining list of numbers
• Until we are done...

23
Algorithm Design Case StudySorting a List of
Numbers(3)

• Flowchart of the algorithm (draw)
• Dealing with list size
• To conquer find Max of the list

24
Algorithm Design Case StudySorting a List of
Numbers(4)
END
BEGIN
Yes
I 2M List sizeMax List1
Yes
I gt M?
ListI gt Max ?
No
No
Max ListI
I I 1
25
Example Sorting of a List
26
Merging Two Lists

• Problem
• You are given two lists of grades. Both lists are
  sorted. You are asked to merge the two lists into
  a single sorted list.
• How would you solve it?

27
Lets Try It

• Draw a flowchart!

28
Algorithm DesignRecursion

• Recursion allows the solution is a problem by
  composing solutions to smaller-sized instances of
  the same problem.
• Recursion is similar to divide and conquer in
  that the problem is divided into sub-problems
  that are tackled individually. The difference is
  that recursion divides the problem into
  smaller-sized instances of the same problem.

29
Recursion Introduction

• Problem How many different flags can you make
  using 4 vertical stripes each of which is a
  different color out of 4 different color
  horizontal stripes? (draw)
• What would your answer be if the number of
  stripes/colors is N?

30
Recursion Factorial (1)

• Problem Previous problem amounts to computing
  the factorial of N, where
• Fact(N) N Fact(N-1) Fact(1) 1 Fact(4)
  4Fact(3) 43Fact(2) 432Fact(1)
  4321
• (Also, introduce pseudocode)

31
Recursion Factorial (2)

• (Drawn in class)

32
Recursive Algorithm DesignRevisiting Sorting

• Problem
• You are given a list of 64 numbers (or names) and
  you are asked to SORT this list in an descending
  order...
• How would you do it?
• Hint Your technique should work even if the list
  is arbitrarily long!

33
Recursive Sorting (1)

• Problem
• You are given a list of 64 numbers (or names) and
  you are asked to SORT this list in an descending
  order...
• Solution Divide the list into two lists each
  with 32 numbers sort them and then merge them.
• If we can solve the problem for lists that
  include 32 numbers, then we can solve the
  original problem

34
Recursive Sorting (2)

• Problem
• You are given a list of 32 numbers and you are
  asked to SORT this list.
• Solution
• Divide the list into two lists each with 16
  numbers sort them and then merge them.
• If we can solve the problem for lists that
  include 16 numbers, then we can solve the
  original problem

35
Recursive Sorting (3)

• Problem
• You are given a list of 16 numbers and you are
  asked to SORT this list.
• Solution
• Divide the list into two lists each with 8
  numbers sort them and then merge them.
• If we can solve the problem for lists that
  include 8 numbers, then we can solve the original
  problem

36
Recursive Sorting (4)

• Problem
• You are given a list of 8 numbers and you are
  asked to SORT this list.
• Solution
• Divide the list into two lists each with 4
  numbers sort them and then merge them.
• If we can solve the problem for lists that
  include 4 numbers, then we can solve the original
  problem

37
Recursive Sorting (5)

• Problem
• You are given a list of 4 numbers and you are
  asked to SORT this list.
• Solution
• Divide the list into two lists each with 2
  numbers sort them and then merge them.
• If we can solve the problem for lists that
  include 2 numbers, then we can solve the original
  problem

38
Recursive Sorting (6)

• Problem
• You are given a list of 2 numbers and you are
  asked to SORT this list.
• Solution
• Trivial!
• We are DONE

39
Recursive Sorting The Algorithm

• Input list into variable LIST
• List1 Top_half(LIST)
• List2 Bottom_half(LIST)
• Sort(List1)
• Sort(List2)
• RESULT Merge(List1, List2)
• Output RESULT

40
Which Sorting Algorithm Is Better?

• Find-maximum sort or merge sort?

41
Time Complexity of Algorithms

• Define the notion of time complexity of
  algorithms and understand its importance to
  comparative algorithm evaluation.
• Examine examples of algorithms with various time
  complexities.
• Examine typical time complexities and understand
  the considerable differences between classes
  thereof.

42
Definition of Time Complexity

• The Time Complexity of an Algorithm is a measure
  of how the time it takes to execute an algorithm
  grows as a function of the problem size
• (Draw charts)

43
Time ComplexityWhat Do We Use It For?

• Knowing the time complexity of an algorithm
  allows us to predict how well the algorithm will
  perform as the size of the problem increases.
• An algorithm takes 10 seconds to execute on a
  problem of size X. How long does it take on a
  problem of size Y?
• More importantly, it allows us to compare
  different algorithms in terms of their time
  complexity.

44
Algorithm Time ComplexityEvaluation

• Identify the computational steps in the algorithm
  that consume a constant time (i.e. independent
  of the problem size).
• 2. Identify the parameter that define the problem
  size.
• 3. Evaluate the time it takes to execute the
  algorithm for various problem sizes.
• 4. Establish a relationship between execution
  time and problem size.

45
Example Computing an Average of a List

• Unit Computational Step
• Adding or dividing two numbers.
• Problem Size
• Number of elements to be averaged.
• Execution Time
• Averaging 5 elements takes 51 6 units of time.
• Establish Time Complexity Relationship
• Averaging N elements Time Complexity N

46
Time Complexity for Computingthe Average of a
List

• If computing the average of 10 numbers takes 0.31
  seconds, how long would it take to compute the
  average of 100 numbers? What would it be for
  1000,000 numbers?

47
Algorithm for Testing If Integer is Prime

• Demonstration
• Check if 31 is prime
• 31 / 2 15.5
• 31 / 3 10.333
• 31 / 5 6.2
• gt Number is Prime!

48
Time Complexity for TestingIf Integer is Prime

• Unit Computational Step
• Checking if an integer divides another. Problem
  Size
• The integer to be tested for primality.
• Execution Time
• To check if 31 is prime, we try 2, 3, 4, and 5
• Establish Time Complexity Relationship
• Checking the primality of N has an approximate
• Time Complexity Sqrt(N)

49
Time Complexity for Testing If Integer is Prime

• If finding whether 37 is prime or not took 2
  seconds, how long would it take to test if 371 is
  prime? How long would it be for 3,710,003?
• Answers (approximate)
• For 371 it would take 6 seconds
• For 3,710,003 numbers it would take 600 seconds

50
Successive Maxima Sorting

• Problem
• You are given a list of 64 numbers and you are
  asked to SORT this list.
• Solution
• Find the maximum of the 64-element list, remove
  it, find the maximum of the 63-element list,
  remove it, etc...

51
Time Complexity for Successive Maxima Sorting

• Unit Computational Step
• Comparing/swapping two numbers.
• Problem Size
• Number of elements to be sorted.
• Execution Time
• Sorting 5 elements takes 4321 10 units of
  time.
• Establish Time Complexity Relationship
• Sorting N elements using successive maxima takes
  Time Complexity N(N-1)/2

52
Time Complexity for Recursive Sorting

• Unit Computational Step
• Comparing/swapping two numbers.
• Problem Size
• Number of elements to be sorted.
• Execution Time
• Sorting 64 elements takes 64 2 (time to sort
  32).
• Establish Time Complexity Relationship
• Sorting N elements using merge sort takes Time
  Complexity Nlog(N)

53
Time Complexity for Recursive Sorting

• If sorting 8 numbers takes 2 seconds, how long
  would it take to sort 64 numbers? What would it
  be for 1,000,000 numbers?

54
Which Sorting Algorithm Is Better?

• N log N is better than N N (especially for
  large N)

55
Time Complexity Classes

• Constant complexity
• The algorithm always takes constant time to
  execute (graph)

56
Time Complexity Classes (2)

• Linear complexity
• The amount of time for an algorithm grows in
  proportion to the input size (graph)
• Algorithms in this class are efficient
• Sub-linear complexity
• The algorithm grows slower than the input size
  (I.e. say if the input size doubles, the
  complexity for the algorithm does not quite
  double)
• Very efficient. Examples are sqrt(N) and log (N)

57
Time Complexity Classes (3)

• Super-linear complexity
• The algorithm takes an amount of time that grows
  faster than the size of the problem. (e.g. when
  the problem size doubles, the time it takes to
  solve the problem more than doubles.)
• This class of algorithms is still practical.
  Example complexities that are super-linear (but
  not exponential) include Nlog(N), NN, NNN,
  etc..

58
Time Complexity Classes (4)

• Exponential complexity
• The algorithm takes an amount of time that grows
  much much faster than the size of the problem.
  (e.g. when the problem is 10 times the original
  size, the time it takes to solve the problem is
  1000 times more!)
• This class of algorithms is basically impractical
  for large problem sizes. Example complexities
  that are exponential include 2N, 10N, NN,
  etc...

59
Exponential Complexity

• How fast does an exponential function grow?
• How much would you save in one month if every day
  you deposit into your savings account an amount
  of money equal to what is in there, starting with
  1 cent on day one?
• 11248163264 cents

60
Exponential Growth

• How fast does an exponential function grow?
• How much would you save in one month if every day
  you deposit into your savings account an amount
  of money equal to what is in there, starting with
  1 cent on day zero?
• 10,737,418.24
• Computer Scientists call this the exponential
  explosion!
• The Internet is growing exponentially??!!?!

61
Example Problems with Exponential Complexity

• Problems with exponential time complexity often
  require the search for a solution in an
  exponentially large solution space.
• Traveling Salesman Problem
• Chess and other similar games
• Testing if a program would terminate

62
Approximations to Problems with Exponential
Complexity

• For such problems we look for a good enough
  approximation to the solution. This typically
  works, but...
• For some problems, finding a good approximation
  is itself impractical (e.g. takes an exponential
  amount of time).

63
Into the Computer

• A high-level program needs to be entered into the
  computer using a text editor.
• Once typed into a text editor, it can be saved to
  disk.
• To run the program, it must be translated into
  machine language by means of an interpreter or
  compiler.

64
Programming Methodologies

• Structured Programming
• A technique to make the programming process
  easier and more productive by writing many small
  programs
• Object-oriented Programming
• The program is a collection of interactive
  objects that contain both data and instructions
• Visual Programming
• Programmers write programs by drawing pictures
  and pointing to objects on the screen

65
Programming for End-Users

• End users may find it necessary to write simple
  programs to customize their applications!
• Programming using Macros for the automation of a
  repetitive task.
• Programming using a Query Language to efficiently
  retrieve information from a database.
• Plug-and-play programming to perform simple tasks
  using canned software components.

66
The Future of Programming

• Trends
• Natural languages and artificial intelligence
  will provide users with programming tools that
  will understand the language of the user
• The distinction between user and programmer will
  begin to fade. Users wont need to master
  complicated programming languages to construct
  applications
• Programs will program themselves based on simple
  descriptions provided by the user

67
(No Transcript)