analysis of algorithms

1
analysis of algorithms

• Richard Kelley

2
welcome!

• Youre in SEM 261.
• This is analysis of algorithms
• Please make sure youre in the right place.

3
administrivia

• We have to get through some boring stuff today.
• Class Policies
• course webpages
• who I am
• textbooks
• homework
• quizzes exams
• well also do a little bit of fun stuff.
• No heavy lifting today.

4
course webpage syllabus

• The main course page is
• http//www.cse.unr.edu/rkelley/algorithms.html
• Youll find the following there
• Syllabus
• Schedule
• Slides
• Example Programs
• Possibly other sites, as demand dictates
• Code submission?
• Blog with lecture notes?

5
meet the course staff

• Im Richard Kelley
• call me Richard
• PhD Student dont call me Dr or Professor
• Social Artificial Intelligence Machine Learning
• Robotics
• Multiagent Systems
• I took this class as an undergrad at UW.
• Ive TAed for this class for a few years here.

6
textbooks

• no required text
• two (highly) recommended texts
• Algorithm Design by Kleinberg Tardos
• Introduction to Algorithms by Cormen, et. al.

7
instead of required textbooks

• my slides
• slides from previous semesters
• links to online resources
• videos
• short notes
• sample programs
• all available from the course website.
• when I use a resource, I will point you to it.

8
homework

• regularly assigned
• Im aiming for at least one per week
• some really easy
• as in, make sure you can compile this program.
• some more involved
• programming
• writing
• English exposition
• proofs
• first (easy) one today . . . maybe.
• Linux only ?

9
quizzes and exams

• necessary evil
• regular quizzes to make sure youre paying
  attention
• these should be easy, count as participation
• a midterm
• about half way through the material
• a final
• comprehensive

10
grading

• homework (55 of your grade)
• some theory
• some programming
• some writing
• midterm (15 of your grade)
• final (25 of your grade)
• attendance participation (5 of your grade)
• short quizzes

11
whats this course about?

• we want to write good programs.
• most advanced courses you take cover topics
• operating systems
• networking
• graphics
• robotics
• some courses are about the processes that lead to
  better code
• human-computer interaction
• software engineering
• the goal is that at the end of this course, your
  programs will be better
• regardless of topic
• regardless of process

12
what is a good program?

• I know it when I see it?
• some common qualities
• easy to read
• easy to modify
• lots of tests
• correct!
• fast
• no memory leaks

13
whats wrong with these measures?

• nothing!
• but were going to focus (mostly) on the last
  two.
• we want to write programs that are fast.
• we want to write programs that use little memory.
• were going to take a slightly more general view,
  though.

14
not programs. algorithms.

• it turns out that focusing on programs is a bad
  idea.
• computers tend to get faster over time.
• one way to make your program faster is to buy a
  better machine.
• although faster machines are good, we can do
  better if were more careful in our design.
• this means choosing better algorithms

15
what is an algorithm?

• we can be really picky here
• but we wont be
• for our purposes, an algorithm is
• a step-by-step process for a well-defined problem
• takes zero or more inputs
• produces zero or more outputs
• is finite
• is clearly defined
• well use pseudocode to describe algorithms

16
what is a good algorithm?

• most problems can be solved in many different
  ways, using many different algorithms.
• some algorithms are better than others
• faster or more space efficient
• we want to choose the best algorithms
• lets focus on time and try to work out what this
  means.
• (Im following Kleinberg Tardos in this process)

17
why time?

• even though there are a few quantities we want to
  optimize our program for, were going to spend
  most of our energy on running time.
• this is pretty standard.
• many of the ideas here carry over to other types
  of analysis.

18
basic notation

• we need to have notation for
• the size of an input
• the running time of the algorithm
• well use variables like n and m to talk about
  input sizes.
• we focus on input sizes instead of inputs
• running time often depends more on the size of
  the input than the actual value of the input.
• well use T( ) to talk about running times.
• for example, T(n) is the running time of an
  algorithm on an input of size n.

19
attempt 1 good means fast

• idea implement the algorithm. run it on some
  inputs, record the times
• this is acceptable in some cases
• in the real world, profiling our programs is a
  common optimization.
• this approach has fundamental limitations that we
  want to (and can easily) avoid.
• at least one homework assignment will involve
  this kind of analysis though.

20
why does this fail?

• problems
• sensitive to the computer we run on
• sensitive to the quality of our implementation
• sensitive to the inputs we choose
• doesnt tell us how running time scales with
  increasing problem sizes.
• scale
• if I run two different algorithms on inputs of
  size one, and then I run them on inputs of size
  two, how much longer do things take?
• rather than look at implementations, we need to
  think mathematically.

21
attempt 2 better than brute force

• lets start by making an agreement
• we will focus on worst-case performance.
• we want to find a bound on the largest possible
  running time an algorithm could have, over all
  inputs of size n.
• this has a nice property
• not sensitive to inputs we choose.
• next, lets define good as better than a
  brute-force search through all possible
  solutions.

22
brute-force search

• most problems have a natural solution space
• sorting
• path finding in graphs
• most optimization problems
• with discrete (finite) problems, we can often
  just look at every possible solution to see if
  its what we want.
• this is usually not tractable.
• solution spaces tend to grow quickly
• in sorting, brute force requires looking at n!
  possible arrangements.

23
whats wrong with this definition?

• good as better than brute force is certainly
  better than our first attempt, but its not
  perfect.
• main problem
• its too vague
• we solve vagueness by making our definition more
  mathematical.

24
attempt 3 polynomial time

• in brute-force search, when we increase the
  problem size by one, we increase the number of
  possibilities multiplicatively.
• in sorting, we go from n! to (n1)!
• a good algorithm should scale better.
• if we double the input size, wed like the
  algorithm to slow down by only a constant factor.
• we get this behavior by agreeing that good
  algorithms are efficient, and
• an algorithm is efficient if its running time is
  bounded above by some fixed polynomial.

25
terminology

• if an algorithms running time is bounded above
  by a fixed polynomial, we may say
• the algorithm has a polynomial running time.
• the algorithm runs in polynomial time.
• the algorithm is a polynomial-time algorithm.
• if a problem has an algorithmic solution that
  runs in polynomial time, we might say
• the problem is in the complexity class P.
• the problem is in P.
• For instance, you probably know that sorting is
  in P.
• bubble sorts running time is bounded above by
  n10.

26
our goal revised

• based on this discussion, we can be more precise
  about the main goals of this course
• we want to find polynomial-time algorithms for
  important problems.
• we want to be able to prove that our algorithms
  run in polynomial time.
• we also want to answer the questions
• are there interesting problems (that are
  solvable) that are not in P?
• for such problems, what do we do?

27
extras

• this is a theory course
• Ill provide opportunities to delve into more
  theoretical items as the course progresses.
• optional for undergrads, less optional for grads.
• this is a practical course
• you should be able to use this stuff every day.
• most of what were learning is at the core of the
  C standard library the boost library.
• as the course progresses, were going to learn
  how to use the standard library to do a lot for
  us.
• were also going to cover C11 (the standard
  formerly known as C0x)
• algorithmic portions only no variadic template
  craziness.

28
the schedule main topics

• basics
• mathematical foundations
• basic analysis
• easy problems
• graph algorithms
• (discrete) optimization dynamic programming
  greedy
• hard problems
• intractability
• randomness and approximation
• concurrency

29
other stuff

• well cover a few topics that are not directly
  related to algorithms, but are useful to know.
• unit testing to make grading easier.
• profiling to figure out what parts of your
  program are taking so long.
• static dynamic analysis tools to find bugs
  and general badness in your code.

30
extra 1 stdvector

• assignment 0
• full instructions online
• download assignment0.cpp from the course webpage.
• make sure you can compile it.
• the program illustrates the use of stdvector,
  which you can think of as a dynamically-resizable
  array.
• you should get in the habit of using stdvector
  whenever you need a contiguous block of storage.
• basic usage is similar to arrays, and is
  illustrated in the program.
• from now on in this class, no more raw arrays.

31
next time math review

• hopefully Ive convinced you that we need to do
  some math.
• the next few lectures will review the most
  important ideas and techniques that we have to
  apply to prove that algorithms run in polynomial
  time.
• also, assignment 0 is due. Assuming we can get
  g to work in the ECC.
• Questions?