CES 510 Intelligent System Design

1
CES 510 Intelligent System Design
B. Ravikumar Department of Engg Science 116 I
Darwin Hall 664 3335 ravi93_at_gmail.com
2
Textbook

• Chris Manning and Hinrich Shutze, Foundations of
  Statistical Natural Language Processing, MIT
  Press, 1999.
• Various supplementary readings.
• Other Useful Books
• Jurafsky Martin, SPEECH and LANGUAGE
  PROCESSING An Introduction to Natural Language
  Processing, Computational Linguistics, and Speech
  Recognition.

3

• Overview of Artificial Intelligence
• major applications
• image processing and vision
• robotics
• game playing
• speech recognition
• natural language understanding
• etc.

4
What is Artificial Intelligence(John McCarthy ,
Basic Questions)

• What is artificial intelligence?
• It is the science and engineering of making
  intelligent machines, especially intelligent
  computer programs. It is related to the similar
  task of using computers to understand human
  intelligence, but AI does not have to confine
  itself to methods that are biologically
  observable.
• Yes, but what is intelligence?
• Intelligence is the computational part of the
  ability to achieve goals in the world. Varying
  kinds and degrees of intelligence occur in
  people, many animals and some machines.
• Isn't there a solid definition of intelligence
  that doesn't depend on relating it to human
  intelligence?
• Not yet. The problem is that we cannot yet
  characterize in general what kinds of
  computational procedures we want to call
  intelligent. We understand some of the mechanisms
  of intelligence and not others.
• More in http//www-formal.stanford.edu/jmc/whatis
  ai/node1.html

5
What is Artificial Intelligence?

• Human-like (How to simulate humans intellect and
  behavior on by a machine.)
• Mathematical problems (puzzles, games, theorems)
• Common-sense reasoning (if there is
  parking-space, probably illegal to park)
• Expert knowledge lawyers, medicine, diagnosis
• Social behavior
• Rational-like
• achieve goals, have performance measure

6
What is Artificial Intelligence

• Thought processes
• The exciting new effort to make computers think
  .. Machines with minds, in the full and literal
  sense (Haugeland, 1985)
• Behavior
• The study of how to make computers do things at
  which, at the moment, people are better. (Rich,
  and Knight, 1991)

The automation of activities that we associate
with human thinking, activities such as
decision-making, problem solving, learning
(Bellman)
7
The Turing Test(Can Machine think? A. M. Turing,
1950)

• Requires
• Natural language
• Knowledge representation
• Automated reasoning
• Machine learning
• (vision, robotics) for full test

8
What is AI?

• Turing test (1950)
• Requires
• Natural language
• Knowledge representation
• automated reasoning
• machine learning
• (vision, robotics.) for full test
• Thinking humanly
• Introspection, the general problem solver (Newell
  and Simon 1961)
• Cognitive sciences
• Thinking rationally
• Logic
• Problems how to represent and reason in a domain
• Acting rationally
• Agents Perceive and act

9
History of AI

• McCulloch and Pitts (1943)
• Neural networks that learn
• Minsky (1951)
• Built a neural net computer
• Darmouth conference (1956)
• McCarthy, Minsky, Newell, Simon met,
• Logic theorist (LT)- proves a theorem in
  Principia Mathematica-Russel.
• The name Artficial Intelligence was coined.
• 1952-1969
• GPS- Newell and Simon
• Geometry theorem prover - Gelernter (1959)
• Samuel Checkers that learns (1952)
• McCarthy - Lisp (1958), Advice Taker, Robinsons
  resolution
• Microworlds Integration, block-worlds.
• 1962- the perceptron convergence (Rosenblatt)

10
The Birthplace of Artificial Intelligence, 1956

• Darmouth workshop, 1956 historical meeting of
  the perceived founders of AI met John McCarthy,
  Marvin Minsky, Alan Newell, and Herbert Simon.
• A Proposal for the Dartmouth Summer Research
  Project on Artificial Intelligence. J. McCarthy,
  M. L. Minsky, N. Rochester, and C.E. Shannon.
  August 31, 1955. "We propose that a 2 month, 10
  man study of artificial intelligence be carried
  out during the summer of 1956 at Dartmouth
  College in Hanover, New Hampshire. The study is
  to proceed on the basis of the conjecture that
  every aspect of learning or any other feature of
  intelligence can in principle be so precisely
  described that a machine can be made to simulate
  it." And this marks the debut of the term
  "artificial intelligence.

11
History, continued

• 1966-1974 a dose of reality
• Problems with computation
• 1969-1979 Knowledge-based system
• Expert systems
• DendralInferring molecular structures
• Mycin diagnosing blood infections
• Prospector recomending exploratory drilling
  (Duda).
• 1986-present return to neural networks
• Machine learning theory
• Genetic algorithms, genetic programming
• Statistical approaches and data mining

12
State of the art

• Deep Blue defeated the reigning world chess
  champion Garry Kasparov in 1997
• Proved a mathematical conjecture (Robbins
  conjecture) unsolved for decades
• No hands across America (driving autonomously 98
  of the time from Pittsburgh to San Diego)
• During the 1991 Gulf War, US forces deployed an
  AI logistics planning and scheduling program that
  involved up to 50,000 vehicles, cargo, and people
  
• NASA's on-board autonomous planning program
  controlled the scheduling of operations for a
  spacecraft
• Proverb solves crossword puzzles better than most
  humans
• DARPA grand challenge 2003-2005, Robocup

13
Whats involved in Intelligence?Intelligent
agents

• Ability to interact with the real world
• to perceive, understand, and act
• e.g., speech recognition and understanding and
  synthesis
• e.g., image understanding
• e.g., ability to take actions, have an effect
• Knowledge Representation, Reasoning and Planning
• modeling the external world, given input
• solving new problems, planning and making
  decisions
• ability to deal with unexpected problems,
  uncertainties
• Learning and Adaptation
• we are continuously learning and adapting
• our internal models are always being updated
• e.g. a baby learning to categorize and recognize
  animals

14
Course overview

• Intelligent systems are autonomous systems
  (hardware / software or a combination) that
  behaves as if it exhibits some form of
  intelligence.
• Concept goes back to Alan Turing who thought
  about machine intelligence and devised Turing
  test to distinguish a machine from a human
  through interaction.
• Some major areas
• Symbolic information processing deductive
  systems
• Game playing chess, backgammon etc.
• natural language understanding answering
  queries, translation, text classification etc.
• Machine learning - adaptive behavior through
  stimulus
• Neural networks
• Statistical modeling
• Fuzzy logic, genetic programming etc.

15
Course overview

• In this course we will introduce statistical
  techniques for inferring structure from text. The
  aim of the course is to introduce existing
  techniques in statistical NLP and to stimulate
  thought into bettering these.
• Applications of NLP
• Information Retrieval
• Information Extraction
• Natural language interface to database
• Statistical Machine Translation

16
Tools

• Probability Theory
• Information Theory
• Algorithms
• Data Structures
• Probabilistic AI
• Grammars and automata

17
The Steps in NLP
18
The steps in NLP (Cont.)

• Morphology Concerns the way words are built up
  from smaller meaning bearing units.
  (come(s),co(mes))
• Syntax concerns how words are put together to
  form correct sentences and what structural role
  each word has.
• Semantics concerns what words mean and how these
  meanings combine in sentences to form sentence
  meanings.
• Pragmatics concerns how sentences are used in
  different situations and how use affects the
  interpretation of the sentence.
• Discourse concerns how the immediately preceding
  sentences affect the interpretation of the next
  sentence.

19
Parsing (Syntactic Analysis)

• Assigning a syntactic and logical form to an
  input sentence
• uses knowledge about word and word meanings
  (lexicon)
• uses a set of rules defining legal structures
  (grammar)
• (S (NP (NAME Sam))
• (VP (V ate)
• (NP (ART the)
• (N apple))))
• I made her duck.

20
Word Sense Resolution

• Many words have many meanings or senses.
• We need to resolve which of the senses of an
  ambiguous word is invoked in a particular use of
  the word.
• I made her duck. (made her a bird for lunch or
  made her move her head quickly downwards?)

21
Reference Resolution

• Domain Knowledge (banking transaction)
• Discourse Knowledge
• World Knowledge
• U I would like to open a fixed deposit account.
• S For what amount?
• U Make it for 800 dollars.
• S For what duration?
• U What is the interest rate for 3 months?
• S Six percent.
• U Oh good then make it for that duration.

22
Why NLP is difficult?

• Different ways of Parsing a sentence
• Word category ambiguity
• Word sense ambiguity
• Words can mean more than their sum of parts (The
  Times of India)
• Imparting world knowledge is difficult ("the blue
  pen ate the ice-cream")
• Fictitious worlds ("people on mars can fly")
• Defining scope ("people like ice-cream," does
  this mean all people like ice cream?)
• Language is changing and evolving
• Complex ways of interaction between the kinds of
  knowledge
• exponential complexity at each point in using the
  knowledge

23
Inferring Knowledge from text

• Words
• word frequencies
• collocations
• word sense
• n-grams (words appear in certain order)
• Grammar
• word categories
• syntactic structure
• Discourse
• Sentence meanings
• Applications
• Information Retrieval
• Information Extraction
• Natural language interface
• Statistical Machine Translation

24
Simple Applications

• Word counters (wc in UNIX)
• Spell Checkers, grammar checkers
• Predictive Text on mobile handsets

25
More significant Applications

• Intelligent computer systems
• NLU interfaces to databases
• Computer aided instruction, automatic graders
• Information retrieval
• Intelligent Web searching
• Data mining
• Machine translation
• Speech recognition
• Natural language generation
• Question answering

26
Spoken Dialogue System
Us e r
Discourse
Semantic
Speech
Interpretation
Interpretation
Recognition
Response
Dialogue
Speech
Generation
Management
Synthesis
27
Parts of the Spoken Dialogue System

• Signal Processing
• Convert the audio wave into a sequence of feature
  vectors.
• Speech Recognition
• Decode the sequence of feature vectors into a
  sequence of words.
• Semantic Interpretation
• Determine the meaning of the words.
• Discourse Interpretation
• Understand what the user intends by interpreting
  utterances in context.
• Dialogue Management
• Determine system goals in response to user
  utterances based on user intention.
• Speech Synthesis
• Generate synthetic speech as a response.

28
Levels of Sophistication in a Dialogue System

• Touch-tone replacement
• System Prompt "For checking information, press
  or say one."
  
• Caller Response "One."
• Directed dialogue
• System Prompt "Would you like checking account
  information or rate information?"
  
  Caller Response "Checking", or
  "checking account," or "rates."
• Natural language
• System Prompt "What transaction would you like
  to perform?"
  
• Caller Response "Transfer Rs. 500 from
  checking to savings."