Steven Pinker Linguist Psychologist

1
Steven Pinker (Linguist / Psychologist)
The main lesson of thirty-five years of AI
research is that the hard problems are easy and
the easy problems are hard. The mental abilities
of a four-year-old that we take for granted
recognizing a face, lifting a pencil, walking
across a room, answering a question in fact
solve some of the hardest engineering problems
ever conceived....
2
Steven Pinker (Linguist / Psychologist)
As the new generation of intelligent devices
appears, it will be the stock analysts and
petrochemical engineers and parole board members
who are in danger of being replaced by machines.
The gardeners, receptionists, and cooks are
secure in their jobs for decades to come.
3

• Pinker says were successful on hard problems,
  but not the easy
• We can say more More and more progress on the
  hard problems seems to be taking us no closer
  to solving the easy ones
• Were able to tackle specific specialist
  problems,i.e. Engineer a solution to a
  specialist problem
• But the more we go into them, the further we get
  from the original goal of AI
• (original goal AI as good as a human)
• Like language moving more shallow than deep
• We move more to specific techniques,
• but gain no insight into general intelligence
• What about general purpose AI?

4
Summing up 50 years progress in AI(From Part I
of Course)

• Pinker says were successful on hard problems,
  but not the easy
• We can say more More and more progress on the
  hard problems seems to be taking us no closer
  to solving the easy ones
• Were able to tackle specific specialist
  problems,i.e. Engineer a solution to a
  specialist problem
• But the more we go into them, the further we get
  from the original goal of AI
• (original goal AI as good as a human)
• Like language moving more shallow than deep
• We move more to specific techniques,
• but gain no insight into general intelligence
• What about general purpose AI?

5
Course Overview

• What is AI?
• What are the Major Challenges?
• What are the Main Techniques?
• Where are we failing, and why?
• Step back and look at the Science
• Step back and look at the History of AI
• What are the Major Schools of Thought?
• What of the Future?

?Done
Part IIGive you an appreciation for the big
picture ? Why it is a grand challenge
6
Course Overview

• What is AI?
• What are the Major Challenges?
• What are the Main Techniques?
• Where are we failing, and why?
• Step back and look at the Science
• Step back and look at the History of AI
• What are the Major Schools of Thought?
• What of the Future?

?Done
Part IIGive you an appreciation for the big
picture ? Why it is a grand challenge
7
Matt Ginsberg, 1995reported in SIGART bulletin
Vol 6, No.2 April 1995
AI is an Engineering discipline built on an
unfinished Science.
8
The Science and Engineering of AI

• AI has an Engineering aspect and a Science aspect
• Engineering
• Build stuff that works, serves a practical
  function
• Physical
• Bridge, Aeroplane
• Information Processing
• Translation system, Autonomous Vehicle
• Science
• Discovery of knowledge general truths and laws
• Physical
• Mechanical forces, stresses, tension, material
  strength, aeronautics
• Information Processing We have some science
• Speed of certain routines (Computer Science)
• Limits and abilities of certain learning
  algorithms
• but we would really like a Science of
  Intelligence

9
The Science and Engineering of AI

• Good Engineering should rest on a solid
  scientific foundation
• AIs foundation looks a bit shaky
• Consider something like bridge building
• Science exists, can make it strong enough to hold
  a certain load, know how many pillars/cables etc.
  needed
• Similar for Aeroplanes. Science also exists, how
  many engines, power, aerodynamic shape etc.
• What about AI problems?
• For Machine Translation only have science for
  some subtasks parsing, n-gram language model
• For Natural Language Understanding not even sure
  how to describe the problem!
• Yet more ambitious What we really want to build
  is something intelligent
• What about the Science of Intelligence?
• AI seems obsessed with better and better
  engineering

10
John McCarthy
Chess is the Drosophila of artificial
intelligence.
11
Drosophila

• Drosophila Fruit Fly
• Drosophila Melanogaster heavily used in research
  in genetics
• Small, easy to grow in laboratory
• Short generation time (two weeks)
• Only four pairs of chromosomes easy to study
• Genome sequenced in 2000
• Some say Chess is Drosophila of AI
• Easy to study
• Studied a lot

12
John McCarthy
Chess is the Drosophila of artificial
intelligence. However, computer chess has
developed much as genetics might have if the
geneticists had concentrated their efforts
starting in 1910 on breeding racing Drosophila.
We would have some science, but mainly we would
have very fast fruit flies.
13
John McCarthy
Chess is the Drosophila of artificial
intelligence. However, computer chess has
developed much as genetics might have if the
geneticists had concentrated their efforts
starting in 1910 on breeding racing Drosophila.
We would have some science, but mainly we would
have very fast fruit flies.
AI seems obsessed with better and better
engineering Where is the Science of Intelligence?
14
What is Intelligence?

• There is no widely agreed-upon scientific
  definition of intelligence
• Try some dictionary definitions
• Understand world, reason about it
• Able to use knowledge to manipulate it ( to
  achieve any desired end)
• Profit from experience (i.e. not static,
  improving all the time, learning)
• There seems to be an internal aspect
• Understand, reason
• Difficult to come up with a precise definition
  for what this is
• What constitutes adequate understanding?
• Tied up with human meaning of things in world
• There seems to be an external aspect
• Manipulate the world
• Difficult to come up with a precise definition
  for what this is
• Manipulate what exactly? And manipulate it in
  what way?
• Tied up with external objects/forces/relationships
  in the world
• We would like some clear abstract theory of
  processing information
• Not tied up with human meanings of internal
  processes
• Not tied up with external world objects

15
What is Artificial Intelligence?

• (See have the AI guys done any better for a
  definition)
• Definitions tied up with internal processes
• To automate activities that we associate with
  human thinking, activities such as decision
  making, problem solving, learning...(Bellman,1978
  )
• The exciting new effort to make computers think
  machines with minds (Haugeland, 1985)
• The study of mental faculties through the use of
  computational models. (Charniak and McDermott,
  1985)
• The study of computations that make it possible
  to perceive, reason, and act. (Winston, 1992)
• Definitions tied up with external world objects
• The art of creating machines that perform
  functions that require intelligence when
  performed by people. (Kurzweil, 1990)
• The study of how to make computers do things at
  which, at the moment, people are better. (Rich
  and Knight, 1991)
• AI . . . Is concerned with intelligent behavior
  in artifacts. (Nilsson, 1998)
• AI definitions still tied up with poorly defined
  external or internal stuff
• AI definitions bring in a new aspect
• Explicit mention of humans
• Not very helpful!

16
What is Artificial Intelligence?

• (See have the AI guys done any better for a
  definition)
• Definitions tied up with internal processes
• To automate activities that we associate with
  human thinking, activities such as decision
  making, problem solving, learning...(Bellman,1978
  )
• The exciting new effort to make computers think
  machines with minds (Haugeland, 1985)
• The study of mental faculties through the use of
  computational models. (Charniak and McDermott,
  1985)
• The study of computations that make it possible
  to perceive, reason, and act. (Winston, 1992)
• Definitions tied up with external world objects
• The art of creating machines that perform
  functions that require intelligence when
  performed by people. (Kurzweil, 1990)
• The study of how to make computers do things at
  which, at the moment, people are better. (Rich
  and Knight, 1991)
• AI . . . Is concerned with intelligent behavior
  in artifacts. (Nilsson, 1998)
• AI definitions still tied up with poorly defined
  external or internal stuff
• AI definitions bring in a new aspect
• Explicit mention of humans
• Not very helpful!

17
Towards a Scientific Definition of Intelligence

• What would a precise definition of intelligence
  look like?
• Can expect it to be similar to the definition of
  communication
• Also a human activity, very complicated with lots
  of human meaning
• But can be studied purely abstractly as a
  mathematical problem
• Possibly a good example for AI because
• Both are about processing information
• Unlike Physics/Chemistry/Biology where theories
  are about physical objects/forces/processes

18
Claude Shannon,A mathematical theory of
communication, 1948
The fundamental problem of communication is that
of reproducing at one point either exactly or
approximately a message selected at another
point. Frequently the messages have meaning
that is they refer to or are correlated according
to some system with certain physical or
conceptual entities. These semantic aspects of
communication are irrelevant to the engineering
problem. The significant aspect is that the
actual message is one selected from a set of
possible messages. The system must be designed
to operate for each possible selection, not just
the one which will actually be chosen since this
is unknown at the time of design.
19
Claude Shannon,A mathematical theory of
communication, 1948
The fundamental problem of communication is that
of reproducing at one point either exactly or
approximately a message selected at another
point. Frequently the messages have meaning
that is they refer to or are correlated according
to some system with certain physical or
conceptual entities. These semantic aspects of
communication are irrelevant to the engineering
problem. The significant aspect is that the
actual message is one selected from a set of
possible messages. The system must be designed
to operate for each possible selection, not just
the one which will actually be chosen since this
is unknown at the time of design.
20
What if there is no theory?

• Maybe there is no clean theory of Intelligence
• Maybe its just some stuff that happens in your
  head
• Gravity, Electromagnetism, Light, Motions of
  planets, etc. all have clean theories
• but theres no reason why intelligence must have
  a clean theory
• Human intelligence evolved over millions of years
• Could well be just a messy load of neuron wiring
  that is intelligence
• David Marr (1945-1980) described Type 1 and
  Type 2 theories

21
Marrs Personal View

• Two types of theory
• Type 1 clean theories
• Clear what and how
• What Clear description of what input needs to
  get transformed to what output
• Different programs (how) could solve the same
  computational problem (what)
• Type 2 messy theories
• Problem is solved by the simultaneous action of a
  considerable number of different processes,
• whose interaction is its own simplest description
• There is no reason why all theories should be
  Type 1
• (Marr acknowledges that it is not a pure
  dichotomy
• a spectrum of possibilities exists in between 12)

22
Marrs Personal View

• Progress in AI can consist in
• Isolate an information processing problem
• Formulate a computational theory for it (what)
• Construct a program that implements it (how)
• Example
• Find shape from shading in an image
• Mathematical description of how input related to
  output
• Working program
• Part 2 tells you what and explains why
• This never needs to be reformulated
• Like a result in mathematics, or hard natural
  sciences
• Part 3 tells you how (often many options)

23
Marrs Personal View

• Progress in AI can consist in
• Isolate an information processing problem
• Formulate a computational theory for it
• tells you what and explains why its important
• Construct a program that implements it
• tells you how
• Marr criticises Mimicry
• Behaviour
• Mimic some aspect of human behaviour
  (chatterbot, IF-THEN rules)
• Structure
• Mimic some aspect of low level structures
  (neurons)
• Problem is they are studying how (3) without
  any clear idea of what and why (2)

24
Marrs Personal View

• Marr criticises Mimicry
• Behaviour
• Mimic some aspect of human behaviour
  (chatterbot, IF-THEN rules)
• Structure
• Mimic some aspect of low level structures
  (neurons)
• Problem is they are studying how (3) without
  any clear idea of what and why (2)
• No need to copy flapping or feathers to fly
• Need to study what flight is
• Not how bird is built

25
Marrs Personal View

• But remember, the breakdown only works for Type 1
  theories
• Isolate an information processing problem
• Formulate a computational theory for it (what)
• Construct a program that implements it (how)
• For Type 2 what and how are tangled
• Some dangers
• Going for Type 2 theories when Type 1 exist
• Can get something that works,
• But sheds no light on the Type 1 theory if there
  is one
• (?) Maybe this is what AI has been doing (Part I
  of this course)
• Looking for Type 1 theory when the problem is
  messier
• Type 1 theory might approximate a real Type 2
  process
• Might be refusing to see the reality because
  there seems to be a nice elegant theory (which is
  wrong)

26
What if there is no Type 1 theory?

• Some science would help the Engineering effort of
  building systems
• but if science is hard to formulate, then
• Why not just keep building stuff that works,
  serves a practical function?
• We have seen from Part I of course
• There seem to be severe limits on what we can do
  by building specific systems
• Natural Language Understanding
• Recognising objects in vision
• Adapting old knowledge to new problems
• Having commonsense
• Doesnt look like we are getting closer to
  general solutions
• Even if we cant find a clean Type 1 theory for
  all of intelligence
• It might still be worthwhile to take a more
  scientific approach
• Rather than Engineering all the time

27
Where to Find Inspiration?

• It looks like we should step back from specific
  problems
• Diagnosing diseases, recognising vowels, playing
  chess, recognising faces
• We should also step back from specific techniques
• Search, logic, neural network, genetic algorithm
• We need to look at the big picture of what
  intelligence is
• Where can we get some hints about this?
• Cognitive Science
• But always bear in mind that we want to be clear
  about what we are doing and why.
• We dont want to mimic behaviour or structure for
  its own sake

28
Cognitive Science

• Definition
• the scientific study either of mind or of
  intelligence

• Essential Questions
• What is intelligence?
• How is it possible to model it computationally?
• Takes ideas from
• Psychology
• Philosophy
• Linguistics
• Neuroscience
• Artificial Intelligence / Computer Science
• Maybe also minor contributions from
• Anthropology, Sociology, Emotion studies, Animal
  Cognition, Evolution