Complexity

1
Complexity

• Our fried egg of formal languages classified them
  by structural complexity regular languages are
  simpler than context-free languages because
  finite automata have a simpler structure than
  push-down automata, etc.
• Our fried egg of partial recursive functions also
  recognises varying levels of complexity
• of definition is minimalisation used?
• of calculation work needed intuitively adding
  is simpler to calculate than multiplying, n?m in
  turn is simpler to calculate than nm, which is
  simpler than
• (with m powers of n).
• of rate of growth the values of constant
  functions like c(2)113 grow more slowly than do
  those of add, which in turn grow more slowly than
  those of mult, which grow more slowly than those
  of exp.

2
Decidable problems
The solvable decision problems are total
computable functions. Decision problems can be
represented as languages for a TM to accept. If
accepted by a TM that always halts, the language
is recursive and the problem decidable. Decision
problems can also be represented as functions to
be computed (say, with output 1 for yes and 0
for no). Some are harder than others to solve.
Typically, a hard problem takes longer and uses
more memory. Can we measure how hard a problem is
by measuring how much calculation work is needed?
By taking the rate of growth of the total
computable function that does the
calculating? For example, to multiply two n?n
matrices (arrays) together, we must calculate n2
entries each requiring 2n reads, n
multiplications, n-1 additions, and 1 write. So
in all it takes f(n) 2n3n3n2(n-1)n2 st
eps, where n measures the size of the instance.
3
How much work does the TM do?
Decision problem Is u ? Palindrome?
q0
B/B R
q1
a/B R
b/B R
B/B R
B/B R
q9
q5
q2
a/a R b/b R
a/a R b/b R
a/a R b/b R
a/a R b/b R
q6
q3
B/B L
B/B L
q7
q4
a/B L
b/B L
q8
B/B R
a/a L b/b L
4
Time complexity functions
The time complexity of M is tcM N ? N where
tcM(n) the maximum number of transitions in a
computation for input strings of length n. In
real implementations, some transitions (primitive
operations) take longer than others (this is
machine dependent) and there are overheads
(address calculation, bound checking for arrays,
type checking, overflow handling), and there are
set-up costs (which may be constant and not
depend on the size of the instance or the
algorithm). So any constants that appear in the
theoretical time complexity tcM are unreliable,
and we use the rate of growth of tcM (big O
notation). Definition f O(g) if there are c,
n0 such that for all n gt n0 f(n)
c?g(n). Functions f and g have the same rate of
growth if f O(g) and g O(f).
5
Examples

• Let f(n) 3n2 4n - 3 and g(n) n2.
• f O(g) since if c1 10 and n1 1, then
  f(n) 3n24n-3 3n2 4n2 3n2
  10n2 c1?g(n).
• g O(f) since if c2 1 and n2 1,
  then g(n) n2 3n2 4n - 3
  c2?f(n).
• Let f(n) n2 and g(n) n3.
• Then f O(g) but g ? O(f).
• f O(g) since f(n) n2 n3 1?g(n).
• But if g O(f), there would be some n0, c
  such that g(n) c?f(n) for all n n0.
• Choose some n gt n0 and n gt c.
• Then g(n) n3 n?n2 gt c?f(n). Oops!

6
Polynomial complexity

• In the class of total computable functions, a
  polynomial is a function of the form
• f(n) c0n0 c1n1 c2n2 cknk
• where c0, c1, , ck may be any natural numbers.
• The rate of growth of a polynomial function is
  easy to find and always less than that of an
  exponential or factorial function.
• If f(n) cknk c1n c0, then
• f O(nk) and nk O(f)
• for every m gt k, f O(nm)
• for every m lt k, f ? O(nm)
• if b gt 0, then f O(bn) but bn ? O(f)
• if b gt 0, bn O(n!) but n! ? O(bn).
• In practice, if a problem can be solved by an
  algorithm (Turing machine) with tc O(nk) then
  we can usually find an algorithm with tc O(n2)
  or O(n3). (Its interesting to wonder why )

7
The class P
Consider the class R of all recursive languages
(equivalently, of all decidable problems, or all
total computable functions). Suppose L ? R and M
is a deterministic TM accepting L. (So M halts on
every input.) Then tcM the maximum number of
transitions in a computation for input strings of
length n. So tcM(n) is found by feeding M all
input strings of length n, seeing how many steps
each computation takes, and setting tcM(n) the
largest number of steps. Define P ? R by L ? P
iff there is a DTM that accepts L and has tcM
O(nk) for some fixed k ? N. So P consists of the
decision problems that are deterministically
solvable in polynomial time. Example membership
of a string u to Palindrome is in P because we
saw a DTM solving the problem with tc O(n2)
last time.
8
Is P R ?
There are decision problems in R that are known
not to be in P because it has been proved that
the best algorithm solving them has to be at
least exponential in its time complexity. (Such
proofs are very difficult.) An example is the
generalisation of the game of chess to an n?n
board. Given an arbitrary position of the black
and white pieces on an n?n board, the problem is
to determine which of the two players has a
winning strategy.
R
P
? Palindrome
? n?n chess