Artificial Intelligence CIS 342

1
Artificial IntelligenceCIS 342

• The College of Saint Rose
• David Goldschmidt, Ph.D.

April 24, 2007
2
Remaining Assignments!

• Remaining assignments
• 4/24 Draft of Research Presentation
• 5/1 Research Paper
• 5/1 Research Presentation
• 5/8 Final Exam
• 5/9 Extra Credit Project 4 (Genetic Algorithms)

3
Computational Complexity (i)

• How long does it take for an algorithm to produce
  a solution?
• Depends on the size of the input and
  the complexity of
  the algorithm
• The size of the input is n
• The complexity of the algorithm is classified
  based on its expected runtime

4
Computational Complexity (ii)

• Consider a telephone directory with n entries
• Name, address, telephone number, etc.
• How do we look someone up...
• ...if names are not in alphabetical order?
• ...if names are alphabetized?
• How do we sort into alphabetical order?

5
Computational Complexity (iii)

• Big-O notation measures the expected runtime of
  an algorithm (i.e. its computational complexity)
• Constant time O(1)
• Logarithmic time O(log n)
• Linear time O(n)
• Linearithmic time O(n log n)
• Quadratic time O(n2)
• Exponential time O(c n)
• Factorial time O(n!)

P
NP
6
Genetic Algorithms

• Genetic algorithms are often well-suited to
  producing reasonable solutions to intractable
  problems
• Intractable problems are problems with excessive
  computational complexity
• Polynomial (P) vs. Non-Polynomial (NP)
• A reasonable solution is a partial or inexact
  solution that adequately solves the problem in
  polynomial time

7
Genetic Algorithms Example (i)

• Consider the Traveling Salesman Problem (TSP) in
  which a salesman aims to visit n cities exactly
  once covering the least distance/cost/etc.
• http//mathworld.wolfram.com/TravelingSalesmanPro
  blem.html
• http//www.tsp.gatech.edu/games/index.html
• Starting at any given node, choose from n1
  remaining nodes, then choose from n2 remaining
  nodes, etc.
• Testing every possible route takes (n1)! steps
• see http//bio.math.berkeley.edu/classes/195/2000/
  lec14/index.html

8
Genetic Algorithms Example (ii)

• Use a genetic algorithm to evolve a near-optimal
  solution to the TSP
• Label cities A, B, C, D, E, F, etc.
• Example circuits ABCDEF, BDAFCE, FBECAD
• How do we perform crossover or mutation
  operations?
• Similar to the cryptogram project, basic
  crossovers might result in invalid members of the
  population
• Combining ABCDEF and BDAFCE may result in ABAFEF

9
Genetic Algorithms Example (iii)

• Key challenge of developing a genetic algorithm
  is often the representation of the problem
• For TSP, consider a standard ordering ABCDEF,
  assigning code 123456
• All other sequences encoded
  based on the removal
  of letters
• Basic crossover works

10
Genetic Algorithms Example (iv)

• All other sequences encoded based on
  the removal of letters from standard
  ordering
• Sequence BDAFCE has code 231311
• B is 2 in ABCDEF
• D is 3 in ACDEF
• A is 1 in ACEF
• F is 3 in CEF
• C is 1 in CE
• E is 1 in E

11
Genetic Algorithms Example (v)

• Combining ACEDB with ABCED...
• Crossover Operation

12
Genetic Algorithms Example (vi)
another approach http//www.dna-evolutions.com/d
naappletsample.html

• Combining ACEDB with ABCED...
• ...yields ACBED
• from A.K. Dewdneys The (New) Turing Omnibus,
  Computer Science Press, New York, 1993

13
Definitions of Intelligence

• Essential English Dictionary, Collins, London,
  1990
• Ability to understand and learn things
• Ability to think and understand instead of doing
  things by instinct or automatically
• Random House Unabridged Dictionary, 2006
• Capacity for learning, reasoning, understanding
• Aptitude in grasping truths, relationships,
  facts, etc.

14
Machine Learning

• Machine learning involves adaptive mechanisms
  that enable computers to
• Learn from experience
• Learn by example
• Learn by analogy
• Learning capabilities improve the performance of
  intelligent systems over time

15
The Turing Test (i)

• Alan Turing, British mathematician (1912-1954)
• Computing machinery and intelligence paper in
  1950
• Can machines think?
• The Turing Test
• A computer passes the Turing test if human
  interrogators cannot distinguish the machine from
  a human based on answers to their questions

16
The Turing Test (ii)

• Turing Test
• Objective standard view on intelligence
• Test is independent of the details of the
  experiment (i.e. numerous variations)
• Provides basis for verification and validation of
  intelligent systems
• Program thought intelligent in some narrow area
  of expertise is evaluated by comparing its
  performance to human performance

17
History of AI (i)

• Warren McCulloch Walter Pitts (1943)
• Research on the human central nervous system led
  to a model of neurons of the brain
• Birth of Artificial Neural Networks (ANN)
• John von Neumann
• ENIAC, EDVAC, etc.
• Von Neumann Architecture

18
Artificial Neural Networks (i)
Q
X x1w1 x2w2 ... xnwn
19
Artificial Neural Networks (ii)
X x1w1 x2w2 ... xnwn
Y Ysign
20
Multilayer Neural Networks
O u t p u t S i g n a l s
I n p u t S i g n a l s
21
Using Neural Networks

• Advantages of neural networks
• Given a solid training dataset, neural networks
  learn
• Powerful classification and pattern matching
  applications
• Drawbacks of neural networks
• Solution is a black box
• Computationally intensive

22
History of AI (ii)

• Claude Shannon, MIT, Bell Labs (1950)
• Computers playing chess
• Chess game involved about 10120 possible moves!
• Even examining one move per microsecond would
  require 3 x 10106 years to make its first move
• Need to incorporate intelligence via heuristics

23
History of AI (iii)

• John McCarthy, Dartmouth, MIT (1950s)
• LISP defined
• Only two years after FORTRAN
• Formal logic
• Programs with Common Sense paper (1958)
• Marvin Minsky, Princeton, MIT
• Anti-logical approach to knowledge representation
  and reasoning called frames (1975)

24
Refocusing AI Efforts

• Great expectations during 1950s and 1960s
• But very limited success
• Researchers focused too much on all-purpose
  intelligent machines with goals to learn and
  reason with human-scale knowledge (and beyond)
• Refocus on specific problem domains (1970s)
• Domain-specific expert systems with facts, rules,
  etc.
• Analyze chemicals, medical diagnoses, etc.

25
Knowledge

• Knowledge is a theoretical or practical
  understanding of a subject or a domain
• Those who possess knowledge are called experts or
  domain experts
• Experts need deep knowledge of both facts and
  rules
• In general, an expert is a skillful person who
  can do things other people cannot

26
Facts and Rules

• A fact may be thought of as a unit of knowledge
• A rule enables an artificially intelligent system
  derive new facts from existing facts
• Rules typically take the form of IF-THEN
  statements
• IF is the antecedent, premise, or condition
• THEN is the consequent, conclusion, or action
• For example
• http//familydoctor.org/symptom.xml

27
Expert Systems (i)

• Expert system performs at a human expert level in
  a narrow and specialized domain
• Tradeoff between accuracy and speed (e.g. 9-1-1)
• Expert systems use symbolic reasoning to solve
  problems
• Symbols represent facts and rules (i.e.
  knowledge)
• Symbols are usually human-readable as opposed to
  cryptic variable names, etc.

28
Expert Systems (ii)

• Expert systems apply heuristics to guide the
  reasoning process
• Reduces the search space and time to produce a
  solution
• Remember the goal is not always
  to solve the problem
  exactly,
  but often to identify one or more
  good solutions

29
Expert Systems (iii)

• Expert systems provide explanation facilities to
  display reasoning (i.e. rules) to users
• How did you come to that conclusion or diagnosis?
• Expert systems make mistakes
• So do human experts!
• Users have to be aware
  of this
  possibility

30
Inference Engine (i)

• New facts are discovered (i.e. inferred) by an
  expert systems inference engine
• Facts serve as data within the knowledge base
• Rules (IF-THEN) describe how new data is inferred
• Inference engine compares each rule with facts it
  knows about, matching the antecedent (IF
  condition)
• When the antecedent matches one or more known
  facts, the rule fires and its consequent (THEN)
  is executed

31
Inference Engine (ii)
1
4
2
3
32
Inference Chain (i)

• An inference chain indicates how an expert system
  applies rules to reach a conclusion

33
Inference Chain (ii)

• An inference chain indicates how an expert system
  applies rules to reach a conclusion

34
Inference Chain (iii)

• An inference chain indicates how an expert system
  applies rules to reach a conclusion

35
Forward Chaining
36
Backward Chaining (i)
37
Backward Chaining (ii)
38
Rule-Based Expert System
39
The Bayesian Rule (i)

• The Bayesian rule (named after Thomas Bayes, an
  18th-century British mathematician)

40
The Bayesian Rule (ii)

• Experts provide p(H), p(H), p(EH), and p(EH)
• Users describe observed evidence E
• Using the Bayesian rule, expert system calculates
  p(HE), the posterior probability of hypothesis H
  upon observing evidence E
• What about multiple hypotheses and evidences?

41
The Bayesian Rule (iii)

• Expand the Bayesian rule to work with multiple
  hypotheses (H1...Hm) and evidences (E1...En)
• Assuming conditional independence among evidences
  E1...En

42
History of AI (iv)

• Rebirth of neural networks (1980s-today)
• Adaptive resonance theory (Grossberg, 1980)
  incorporated self-organization principles
• Hopfield networks (Hopfield, 1982)
  introduced neural networks with
  feedback loops
• Back-propagation learning algorithm
  (Bryson
  and Ho, 1969) for training
  multilayer perceptrons

43
History of AI (v)

• Knowledge engineering (1980s-today)
• Fuzzy set theory (Zadeh, 1965) associates words
  with degrees of truth or value
• Rule-based knowledge systems
• Combining information from multiple experts
• Semantic Web

44
Fuzzy Logic (i)

• Need the ability to represent expert knowledge
  using vague and inexact terms
• Fuzzy Logic describes fuzziness and degrees using
  English vocabulary
• e.g. degrees of height, speed, distance,
  temperature, beauty, intelligence, etc.

45
Fuzzy Logic (ii)

• 1965 paper Fuzzy Sets (Lotfi Zadeh)
• Apply natural language terms to a formal
  system of mathematical logic
• http//www.cs.berkeley.edu/zadeh
• Fuzzy Logic is a set of mathematical principles
  for knowledge representation based on
  degrees of membership

46
Fuzzy Logic (iii)

• Unlike two-valued Boolean logic, fuzzy logic is
  multi-valued
• Fuzzy logic deals with degrees of membership and
  degrees of truth

47
Fuzzy Sets (i)

• Fuzzy set depicting height
• Compare to Crisp (Boolean) set

48
Fuzzy Sets (ii)

• A fuzzy set provides a natural fit
• High applicability to real-world
  knowledge/concepts

49
Representing Fuzzy Sets (i)

• Representing height using crisp sets

50
Representing Fuzzy Sets (ii)

• Representing height using fuzzy sets

51
Representing Fuzzy Sets (iii)
52
Fuzzy Expert Systems

• IF height is tall
• THEN weight is heavy

53
Applications of Fuzzy Logic (i)

• Why use fuzzy expert systems or fuzzy control
  systems?
• Apply fuzziness (and therefore accuracy) to
  linguistically defined terms and rules
• Lack of crisp or concrete mathematical models
  exist
• When do you avoid fuzzy expert systems?
• Traditional approaches produce acceptable results
• Crisp or concrete mathematical models exist and
  are easily implemented

54
Applications of Fuzzy Logic (ii)

• Real-world applications include
• Control of robots, engines, automobiles,
  elevators, etc.
• Cruise-control in automobiles
• Temperature control
• Reduce vibrations in camcorders
• http//www.esru.strath.ac.uk/Reference/concepts/fu
  zzy/fuzzy_appl.de20.htm
• Handwriting recognition, OCR
• Predictive and diagnostic systems (e.g. cancer)

55
History of AI (vi)

• Evolutionary computation (1970s-today)
• Natural intelligence is a product of evolution
• Can we solve problems by simulating biological
  evolution?
• Survival of the fittest

56
Evolutionary Computing

• Evolutionary computation uses an evolutionary
  approach to finding high-quality partial
  solutions to a problem
• Optimization algorithms
• Think of natural
  biological systems
  
  as intelligent
• Adaptive!

57
Genetic Algorithms (i)

• Crossover Operation

58
Genetic Algorithms (ii)

• Mutation
  
  Operation

59
Genetic Algorithms (iii)

• Generation

60
Using Genetic Algorithms

• Advantages of genetic algorithms
• Often outperform brute force approaches by
  randomly jumping around the search space
• Ideal for problem domains in which near-optimal
  (as opposed to exact) solutions are adequate
• Disadvantages of genetic algorithms
• Might not find any satisfactory partial solutions
• Tuning can be a challenge

61
Thank You
See you in the future!