Computation

1
Computation
2
Binary Numbers

• Decimal numbers
• Binary numbers

http//schoolscience.rice.edu/duker/robots/binaryn
umber.html
3
Representing Text

• Decide how many characters we need to represent.
• Determine the required number of bits.
• Ascii 7 bits. Can encode 27 128 different
  symbols.

4
Ascii
http//www.neurophys.wisc.edu/comp/docs/ascii.html

5
Representing Text
Four 01000110 01101111 01110101 01110010
6
When We Need More Characters
What about things like
???
7
When We Need More Characters
What about things like
???
Answer Unicode 32 bits. Over 4 million
characters. http//www.unicode.org/charts/
8
Digital Images
9
RGB
The red channel
10
RGB
The green channel
11
RGB
Red Green Blue
12
Representing Pictures
13
Representing Sounds
14
Representing Programs
public static TreeMapltString, Integergt create()
throws IOException public static
TreeMapltString, Integergt create() throws
IOException Integer freq String word
TreeMapltString, Integergt result new
TreeMapltString, Integergt() JFileChooser c
new JFileChooser() int retval
c.showOpenDialog(null) if (retval
JFileChooser.APPROVE_OPTION) Scanner
s new Scanner( c.getSelectedFile()) while(
s.hasNext() ) word s.next().toLowerCase()
freq result.get(word)
result.put(word, (freq null ? 1 freq
1)) return result
15
The Roots of Modern Technology
5thc B.C. Aristotelian logic invented 1642
Pascal built an adding machine 1694 Leibnitz
reckoning machine
16
The Roots, continued
1834 Charles Babbages
Analytical Engine
Ada writes of the engine, The Analytical Engine
has no pretensions whatever to originate
anything. It can do whatever we know how to order
it to perform.
The picture is of a model built in the late 1800s
by Babbages son from Babbages drawings.
17
The Roots Logic
1848 George Boole The Calculus of Logic
chocolate and ? nuts and mint
chocolate nuts mint
18
Boolean Logic
P Q ?P P ? Q P ? Q P ? Q P ? Q
True True False True True True True
True False False True False False False
False True True True False True False
False False True False False True True
19
Using Boolean Logic
P Q ?P P ? Q P ? Q (P ? Q)? Q ?P ? ((P ? Q)? Q)
True True False True True True False
True False False True False False False
False True True True True True True
False False True False True True True
20
How Big Are the Truth Tables?
P Q R
True True True
True True False
True False True
True False False
False True True
False True False
False False True
False False False
21
2n
22
Increased Power of Quantified Logic

• Bear(Smoky).
• ?x (Bear(x) ? Animal(x)).
• ?x (Animal(x) ? Bear(x)).
• ?x (Animal(x) ? ?y (Mother-of(y, x))).
• ?x ((Animal(x) ? ?Dead(x)) ? Alive(x)).

23
Using Quantified Logic (FOL)
We can represent what we know as a set of logical
formulas and then manipulate them.
24
Facts in English
(1) Marcus was a man. (2) Marcus was a
Pompeian. (3) All Pompeians were Romans. (4)
Caesar was a ruler. (5) All Romans were either
loyal to Caesar or hated him. (6) Everyone is
loyal to someone. (7) People only try to
assassinate rulers they are not loyal to. (8)
Marcus tried to assassinate Caesar.
25
Facts in First Order Logic (FOL)
(1) Marcus was a man. man(Marcus) (2) Marcus was
a Pompeian. Pompeian(Marcus) (3) All Pompeians
were Romans. ?x Pompeian(x) ? Roman(x) (4)
Caesar was a ruler. ruler(Caesar) (5) All Romans
were either loyal to Caesar or hated him. ?x
Roman(x) ? loyalto(x, Caesar) ? hate(x,
Caesar) (6) Everyone is loyal to someone. ?x ?y
loyalto(x, y) (7) People only try to assassinate
rulers they are not loyal to. ?x ?y person(x) ?
ruler(y) ? tryassassinate(x, y) ? ?loyalto(x,
y) (8) Marcus tried to assassinate
Caesar. tryassassinate(Marcus, Caesar)
26
Search
27
Breadth-First Search
Is this a good idea?
28
Depth-First Search
29
Scalability
Solving hard problems requires search in a large
space.
To play master-level chess requires searching
about 8 ply deep. So about 358 or 2?1012 nodes
must be examined.
30
Growth Rates of Functions
31
The Advent of the Computer
1945 ENIAC The first electronic digital
computer
32
1949 EDVAC
The first stored program computer
33
Moores Law
http//www.intel.com/technology/mooreslaw/
34
How Much Computer Power Might It Take?
http//www.frc.ri.cmu.edu/hpm/book97/ch3/index.ht
ml
35
How Much Compute Power is There?
From Hans Moravec, Robot Mere Machine to
Transcendent Mind 1998.
36
Kurweils Vision
http//www.pocket-lint.co.uk/news/news.phtml/12920
/13944/Computers-match-humans-by-2030.phtml
37
Limits to What We Can Compute

• Are there fundamentally uncomputable things?
• Does God exist?
• Whats the best way to run a country?
• Does this puzzle have a solution?

38
Mathematics in the Early 20th Century
1900 Hilberts program and the effort to
formalize mathematics 1931 Kurt Gödels paper,
On Formally Undecidable Propositions 1936 Alan
Turings paper, On Computable Numbers with an
application to the Entscheidungs problem
39
What Can Algorithms Do?

1. Can we make all true statements theorems?
2. Can we decide whether a statement is a theorem?

40
Gödels Incompleteness Theorem
Kurt Gödel showed, in the proof of his
Incompleteness Theorem Gödel 1931, that the
answer to question 1 is no. In particular, he
showed that there exists no decidable
axiomatization of Peano arithmetic that is both
consistent and complete.
41
The Entscheidungsproblem
Does there exist an algorithm to decide, given an
arbitrary sentence w in first order logic,
whether w is valid? Given a set of axioms A
and a sentence w, does there exist an algorithm
to decide whether w is entailed by A? Given a
set of axioms, A, and a sentence, w, does there
exist an algorithm to decide whether w can be
proved from A?
42
Turing Machines
At each step, the machine must ? choose its
next state, ? write on the current square,
and ? move left or right.
43
An Example
44
The Church-Turing Thesis
All formalisms powerful enough to describe
everything we think of as a computational
algorithm are equivalent.

• Turing machines
• Lambda calculus
• Standard programming languages like Java
• Conways game of life
• DNA computing

45
The Game of Life
Playing the game
The rules ? A dead cell with exactly three
live neighbors becomes a live cell (birth).
? A live cell with two or three live neighbors
stays alive (survival). ? In all other cases,
a cell dies or remains dead (overcrowding or
loneliness). A game halts iff it reaches some
stable configuration.
46
Back to the Entscheidungsproblem
Theorem The Entscheidungsproblem is
unsolvable. Proof (A variant of Turings proof)

• Given a Turing machine M, we can construct a
  logical formula F that is true iff M ever halts.
• If there were a solution to the
  Entscheidungsproblem, then we could determine the
  truth of F and thus be able to decide whether M
  ever halts.
• But there is no procedure for determining (in the
  general case) whether M halts.
• So there is no solution to the Entscheidungsproble
  m.

47
The Halting Problem
Turing machine, M input string, w
Does M halt on w?
Accept Reject
48
An Example of a Similar Question
Does this program halt on all inputs?
times3(x positive integer) While x ? 1
do If x is even then x x/2. Else x
3x 1.
Lets try it.
49
The Halting Problem Is Undecidable
Proof If it were decidable, then some TM MH
would decide it. MH would implement the
specification halts(ltM string, w stringgt)
If ltMgt is a Turing machine description
and M halts on input w then accept.
Else reject.
50
Trouble
Trouble(x string) if halts(x, x) then loop
forever, else halt. What is Trouble(ltTroublegt)?
What is halts(ltTrouble, Troublegt)? ?
If halts reports that Trouble(ltTroublegt) halts,
Trouble ________. ? If halts reports that
Trouble(ltTroublegt) does not halt, Trouble
__________.
51
Another Undecidable Problem
The Post Correspondence Problem
52
A PCP Instance With a Simple Solution
i X Y
1 b aab
2 abb b
3 aba a
4 baaa baba
53
A PCP Instance With a Simple Solution
i X Y
1 b aab
2 abb b
3 aba a
4 baaa baba
Solution 3, 4, 1
54
Another PCP Instance
i X Y
1 11 011
2 01 0
3 001 110
55
A PCP Instance With No Simple Solution
i X Y
1 1101 1
2 0110 11
3 1 110
56
A PCP Instance With No Simple Solution
i X Y
1 1101 1
2 0110 11
3 1 110
Shortest solution has length 252.
57
A Tiling Problem
Given a finite set T of tiles of the form Is
it possible to tile an arbitrary surface in the
plane?
58
A Set of TilesThat Cannot Tile the Plane
59
Is the Tiling Problem Decidable?
Wangs conjecture If a given set of
tiles can be used to tile an arbitrary surface,
then it can always do so periodically. In
other words, there must exist a finite area that
can be tiled and then repeated infinitely often
to cover any desired surface. But Wangs
conjecture is false.
60
The Implications

• The Entscheidungs problem is undecidable.
• Theres no black box reasoning engine for FOL.
• Would quantum computing change the picture?
• Does undecidability doom our attempt to make
  artificial copies of ourselves?

61
Is Decidability Enough?
62
Boolean Logic, Again
P Q ?P P ? Q P ? Q (P ? Q)? Q ?P ? ((P ? Q)? Q)
True True False True True True False
True False False True False False False
False True True True True True True
False False True False True True True
How many steps would it take a deterministic
Turing machine to examine this table?
63
Nondeterminism
64
Nondeterministically Deciding SAT
P Q ?P P ? Q P ? Q (P ? Q)? Q ?P ? ((P ? Q)? Q)
True True False True True True False
True False False True False False False
False True True True True True True
False False True False True True True
How many steps in a single path of the process of
examining this table?
65
P and NP

• P problems that are solvable deterministically
  in polynomial time.
• NP problems that are solvable
  nondeterministically in polynomial time.

66
The Traveling Salesman Problem
15
25
10
28
20
4
8
40
9
7
3
23
Given n cities and the distances between each
pair of them, find the shortest tour that returns
to its starting point and visits each other city
exactly once along the way.
67
The Traveling Salesman Problem
15
25
10
28
20
4
8
40
9
7
3
23
Given n cities Choose a first city n Choose
a second n-1 Choose a third n-2 n!
68
The Traveling Salesman Problem
Can we do better than n! ? First city
doesnt matter. ? Order doesnt matter. So
we get (n-1!)/2.
69
The Growth Rate of n!
2 2 11 479001600
3 6 12 6227020800
4 24 13 87178291200
5 120 14 1307674368000
6 720 15 20922789888000
7 5040 16 355687428096000
8 40320 17 6402373705728000
9 362880 18 121645100408832000
10 3628800 19 2432902008176640000
11 39916800 36 3.6?1041
70
The Traveling Salesman Problem
71
Solving it Nondeterministically
15
25
10
28
20
4
8
40
9
7
3
23
72
SAT and Traveling Salesman
D ND
SAT 2n n
Traveling Salesman n! n
73
P and NP

• P problems that are solvable deterministically
  in polynomial time.
• NP problems that are solvable
  nondeterministically in polynomial time.
• NP-complete NP problems that are at least as
  hard as every other NP-complete problem.

74
Growth Rates of Functions, Again
75
Does Complexity Doom AI?