Data on Trial: Lessons from The Turing Test

1
Data on TrialLessons from The Turing Test

• Minds Machines

2
I believe that in about fifty years time it
will be possible to programme computers, with
a storage capacity of about 109, to make them
play the imitation game so well that an
average interrogator will not have more than 70
per cent chance of making the right
identification after 5 minutes of
questioning -Alan Turing (1950)
3
The Imitation Game, or The Turing Test!
Machine
Interrogator
Human
4
A Definition of Intelligence?

• Some commentators see the Turing Test as a
  definition of intelligence.
• Many people have commented on the shortcomings of
  the Turing Test as a definition of intelligence
• This definition would be philosophical
  behaviorism, which has obvious problems. In
  short, we think that being intelligent causes the
  behavior, but doesnt consist in the behavior.
• This definition would define intelligence based
  on the subjective judgment of whomever happens to
  be the interrogator, however long the
  conversation was, and what the conversation was
  about.

5
Not a Definition

• Turing himself clearly did not intend to propose
  a definition of intelligence. For example, in his
  paper he readily acknowledges that one could have
  intelligent beings not being able to pass the
  test simply by not having a human-like intellect
• May not machines carry out something which ought
  to be described as thinking but which is very
  different from what a man does? This objection is
  a very strong one, but at least we can say that
  if, nevertheless, a machine can be constructed to
  play the imitation game satisfactorily, we need
  not be troubled by this objection

6
A Sufficient Condition for Intelligence?

• Many commentators interpret this statement as
  saying that if a machine passes the Turing Test,
  then it is intelligent, i.e. that passing the
  Turing Test is a sufficient condition for
  intelligence (since intelligence is a necessary
  condition to pass it), but not a necessary one
  (and hence it is not a definition).
• In logic
• P ? I
• But not I ? P

7
Not a Great Criterion Either

• As a sufficient condition for being intelligent,
  the Turing Test suffers from some of the same
  problems as before such a criterion would still
  amount to a subjective judgment based on
  imprecisely defined behavioral criteria.
• In short, this seems to be a rather sloppy
  criterion!
• Now, would Turing really not have anticipated
  this (rather straightforward) objection? Hmm

8
Eliza

• A psychotherapist program developed by Joseph
  Weizenbaum in 1966.
• Many people conversing with Eliza had no idea
  that they werent talking to a human.
• Did Eliza pass the Turing Test?
• Or is it just easy being a psychotherapist?
• Eliza wasnt really tested in the format that
  Turing proposed.
• Still, it is interesting that humans were quick
  to attribute human-level intelligence to such a
  simple program.
• Maybe in a real Turing Test a relatively simple
  computer program can trick the interrogator as
  well?

9
The Loebner Competition

• Multiple judges rank-order multiple humans and
  multiple computer programs from most likely to
  be human to least likely to be human.
• Loebner promises 100,000 for the first computer
  program to be indistinguishable from a human.
• Thus far, Loebner is still a rich man!
• So maybe it is difficult to pass the test.
• Maybe we could say that if something does pass
  the test, then there is at least a good chance
  for it to be intelligent.
• Maybe, but this is not very exciting.

10
Contrary Views

• In his paper Turing goes over a list of Contrary
  Views on the Main Question
• Machines
• can only do what theyre told to do (Lady
  Lovelace)
• cant learn
• cant be creative
• cant make mistakes
• cant (fill in the blank)
• Turing Our mistakes are that
• We generalize from existing (special-purpose)
  machines (Turing-machines are general-purpose)
• We equate level of mechanics with level of
  functioning (emergent behavior emergent
  properties)

11
Another Question

• If Turings point of his article was to propose a
  test or criteria for intelligence, then why are
  none of these objections about the validity of
  this test?
• At best, these seem to be objections to the claim
  that machines can pass this test.
• Hmm, so what is Turings real point of the paper?

12
Turings Argument for AI?

• Some commentators see Turings paper as an
  argument for the possibility of AI
• Machines (computers) can pass the Turing Test
• Anything that passes the Turing Test is
  intelligent
• Therefore, machines (computers) can be
  intelligent

13
Can Machines pass the Turing Test?

• As we saw, Turing thinks that this is true (or at
  least, he expresses confidence that machines will
  do well in the Imitation Game).
• However, he never really makes clear why he
  thinks this is so.
• You would think that Turing would have made some
  effort to argue for this (rather controversial)
  claim if it is used as a premise of the earlier
  argument. Hmm
• Presumably, Turing thinks that passing the test
  requires nothing more than some kind of
  information processing ability, which is exactly
  what computers do.

14
A Puzzle

• But wait, cant we then just argue as follows
• Intelligence requires nothing more than some kind
  of information processing ability
• Computers can have this information processing
  ability
• Therefore, computers can be intelligent
• Indeed, this is exactly how proponents of AI make
  the argument today.
• So why didnt Turing make this very argument? Why
  bring in the game?

15
The Puzzle Again And A Little Question

• Going back to the Contrary Views, AI opponents
  think machines cant do certain things, but
  Turing thinks they can. Now, if the issue is
  whether or not computers can be intelligent,
  isnt that indeed the crucial, make-or-break
  issue?
• The Turing Test doesnt seem to be able to shed
  any more light on this issue, so why bring in
  the game?
• Oh, and why the strange set-up of the Turing-Test
  anyway? Why did Turing pit a machine against a
  human in some kind of contest? Why not have the
  interrogator simply interact with a machine, see
  what it is or is not able to do, and determine
  whether or not the machine is intelligent based
  on those interactions?
• If we are so concerned about what machines can
  and cannot do, why not simply do

16
The Super-Simplified Turing Test!!
Interrogator
Machine
17
Answer Bias

• The mere knowledge that we are dealing with a
  machine will bias our judgment as to whether that
  machine can think or not, as we may bring certain
  preconceptions about machines to the table.
• Moreover, knowing that we are dealing with a
  machine will most likely lead us to raise the bar
  for intelligence it cant write a sonnet? Ha, I
  knew it!
• By shielding the interrogator from the
  interrogated, such a bias and bar-raising is
  eliminated in the Turing-Test.
• OK, but still, why not

18
The Simplified Turing Test
Interrogator
Machine or Human
Note this is exactly how many commentators talk
about the Turing Test
19
Level the Playing Field

• Since we know we might be dealing with a machine,
  we still raise the bar for the entity on the
  other side being intelligent.
• Through his set-up of the test, Turing made sure
  that the bar for being intelligent wouldnt be
  raised any higher for machines than we do for
  fellow humans.

20
Back to The Puzzle

• OK, fine, but we are still left with our basic
  puzzle why bring up any test at all? Why not
  simply consider the questions as to whether
  machines can or cannot have certain abilities
  head-on, and try and define some more precise and
  objective criteria for intelligence, instead of
  having this be determined by such a sloppy Game?
• I believe that the answer to the little question
  provides us with an answer to our puzzle the
  convoluted set-up wasnt merely a practical
  consideration to eliminate bias in some strange
  game, but rather the whole point of his article!
• That is, I believe the point of Turings article
  wasnt to make any argument for the possibility
  of AI, but rather that if we put a label
  intelligent being on other human beings based
  on their behavior then, just to be fair, we
  should do the same for machines, whether we are
  correct in any such attributions or not.

21
Imitation Game vs Turing Test

• I think it is likely that Turing never intended
  to propose any kind of test for machine
  intelligence (let alone propose a definition!).
• At best, Turing would say that passing the test
  means that we should call that entity
  intelligent, correct or not.
• In other words, Turings point was about language
  use!
• Talking about language use, I think we really
  should no longer refer to the Turing Test as the
  Turing Test.
• Interesting fact In his original article Turing
  uses the word pass or passing 0 times, test
  4 times, and game 37 times.

22
In Turings Words
The original question, Can machines think?, I
believe to be too meaningless to deserve
discussion. Nevertheless I believe that at the
end of the century the use of words and general
educated opinion will have altered so much that
one will be able to speak of machines thinking
without expecting to be contradicted. -Alan
Turing (1950)
23
Oh, and another thing

• I believe that seeing Turings contribution as
  laying out a test, and our obsession to try and
  pass that test (or at least thinking about AI
  that way) has been (and still is) detrimental to
  the field.
• E.g. In Essentials of Artificial Intelligence,
  Ginsberg defines AI as the enterprise of
  constructing a physical symbol system that can
  reliably pass the Turing Test
• But trying to pass the test encourages building
  cheap tricks to convince the interrogator that
  he/she is dealing with a human, which is exactly
  what we have seen with Eliza, Parry, and the
  modern-day Alice and Jabberwacky.
• This kind of work has advanced the field of AI
  exactly zilch!
• Can we please stop talking about a Test?!?
• Thank you!