Algorithms and Complexity

1
Algorithms and Complexity
2
Time Complexity of Algorithms

• The Time Complexity of an Algorithm is a measure
  of how the amount of time required to execute
  an algorithm grows as a function (in relation to)
  the problem size.

execution time
execution time
problem size
problem size
3
Problem Sort a List

• Problem
• Given a list of 100 numbers, sort this list in
  descending order.
• How would you do it??

4
Why we care aboutTime Complexity

• The time complexity of an algorithm is a
  predictor of how quickly the algorithm will
  perform as the size of the problem increases.
• If an algorithm takes 10 seconds to complete a
  problem of size X, how long does the algorithm
  take to complete a problem of size X1? X2? Y?
• Why do we want to know time complexity of an
  algorithm?
• Imagine we have two algorithms that solve the
  same problem.
• Is any algorithm that solves the problem good
  enough?

5
Time complexity Classes Constant

• Constant Time Complexity
• The algorithm takes a constant amount of time to
  execute
• Its independent of the size of the problem to be
  solved!
• Obviously, this class is the most efficient class
  of algorithms
• Unfortunately, most useful algorithms do not
  fall into this category. (can you guess why?)

execution time
problem size
6
Time complexity Classes Linear

• Linear Time Complexity
• The algorithm takes an amount of time to execute
  that grows proportionately to the size of the
  problem.
• e.g., when the problem size doubles, so does the
  time taken.
• This class of algorithms is efficient.

execution time
problem size
7
Time complexity Classes Sub-Linear

• Sub-Linear Time Complexity
• The algorithm takes an amount of time to execute
  that grows slower than the size of the problem.
• e.g., when the problem size doubles, the time
  taken doesnt quite double.
• This class of algorithms is quite efficient.
• Examples include sqrt(n) and log(n)

execution time
problem size
8
Time complexity Classes Super-Linear

• Super-Linear Time Complexity
• The algorithm takes an amount of time to execute
  that grows faster than the size of the problem.
• e.g., when the problem size doubles, the time
  taken more than doubles.
• This class of algorithms is practical.
• Examples include nlog(n), nn, nnn

execution time
problem size
9
Time complexity Classes Exponential

• Exponential Time Complexity
• The algorithm takes an amount of time to execute
  that grows much, much, faster than the size of
  the problem.
• e.g., when the problem size is 10 times the
  original size, the time taken is 1000 times more!
• This class of algorithms is impractical for large
  problem sizes
• Examples include 2n, 10n, nn, etc.

execution time
problem size
10
Time complexity Classes Exponential

• Just how fast does an exponential function grow?
• How much would you save in one month if every day
  you deposit into your savings account an amount
  equal to what is already there on that day,
  starting with 1 cent on day one?
• 1 1 2 4 8 16 32 64 cents

11
Time complexity Classes Exponential

• Just how fast does an exponential function grow?
• How much would you save in one month if every day
  you deposit into your savings account an amount
  equal to what is already there on that day,
  starting with 1 cent on day one?
• 10,737,418.24
• This is called an exponential explosion

12
Time complexity Classes Exponential

• Algorithms with exponential time complexity often
  require the search for a solution in an
  exponentially large solution space.
• Travelling Salesman Problem
• Chess and other similar strategy games
• For such problems we look for a good enough
  approximation to the solution. Typically, this
  works, but . . .
• For some problems finding a good approximation is
  itself impractical (e.g., may take an
  exponential amount of time!)

13
Operating Systems
14
Improving the computer

• Consider our view of the computer at this point
• Our resources (memory, CPU, I/O) are controlled
  by programs (really just one program)
• Our computer is programmable, but how are we
  getting these programs into our computer? How
  will they run? Run one, reboot, repeat?
• we don't even know how a single program is loaded
  and run...
• what about more than one program?
• the Von Neumann bottleneck only lets us run one
  program at a time
• there would be a lot of issues in running
  multiple programs simultaneously
• sharing devices, if possible, would be dangerous!
• communicating with all the peripheral devices
  will require a lot of program code for each
  device
• thats a lot of devices to write program code for
• And we havent talked about how humans will use
  the computer.
• Enter the Operating System

15
What is an operating system?

• What does an operating system do?
• Control how programs are run on the computer
• Provide an abstraction of the specific details of
  the hardware of the computer
• That abstraction provides more than just the
  hardware itself
• Provide a mechanism for hardware components to be
  shared among different programs
• Including the CPU!
• Provide the user an interface to the programs and
  files on the computer
• The operating system (OS)
• Collection of programs that form a virtual
  hardware
• The OS is the foundation for other programs
• All programs run on-top of the operating system
  rather than the real hardware
• The OS is (essentially) the first program to run
  when you turn on the computer

16
What is an operating system?

• What does an operating system do?
• Programs (even the OS) run from memory
• Remember that RAM is empty when we first turn on
  our computer
• How does the OS get into memory to run in our
  computer?

software
hardware
17
What really runs first?

• Remember ROM?
• Small non-volatile memory that will be there
  when we turn on the computer
• We place a small hand-off program into ROM and
  it will be run first when we turn on the computer
• BIOS (Basic Input Output System)
• Small unchangeable software stored in ROM
• BIOS software is embedded (built into) in the
  motherboard
• BIOS can be changed
• Flash update its just rarely done
• BIOS
• A collection of programs that have the capability
  of communicating with peripheral devices
  (hardware)
• Keyboards, Disk drives, printers,
  display/monitors, and other devices.
• BIOS is what let's us run our operating system

18
Booting the Computer

• Booting (Starting) the computer
• The computer invokes the BIOS initialization
  program found in ROM
• Makes the computer recognize the keyboard, floppy
  and the hard disk drives.
• 1. Diagnostics are run on the hardware devices in
  the computer
• 2. The program in ROM instructs the computer to
  look for an operating system
• depending on your settings, the computer will
  look in different places at different orders
  usually
• 1st try the Floppy disk
• Then try a CD-ROM
• Then a hard disk (sometimes there are multiple
  hard disks)
• 3. The operating system (wherever found) must be
  loaded into RAM so it can run

19
Booting the Computer

• Cold boot
• Starting up the computer by turning the power on
  (power was off previously).
• BIOS is loaded into RAM and BIOS looks for and
  loads the operating system into RAM.
• Warm boot
• Kick back to BIOS and reload the operating system
  into RAM without disrupting the power to the disk
  drives or power supply.
• (aka reboot/restart)
• both reload the operating system
• the operating system is a collection of programs
  . . .