Automata theory and formal languages

1
Automata theoryand formal languages
The Chinese University of Hong KongFall 2009

• Andrej Bogdanov
• http//www.cse.cuhk.edu.hk/andrejb/csc3130

2
What are computers good at?
In 1997, IBM Deep Blue defeated world chess
champion Gary Kasparov in a six match tournament.
3 ½
2 ½
3
What are computers good at?
The search engine Google indexes 2,000,000,000
web pages. It lets you find pretty much anything
you want.
4
What else?
Fly airplanes
Recommend books
Is there anything a computer cannot do?
5
Impossibilities
Why do we care about the impossible?
6
Perpetual motion
In the middle ages, people wanted a machine that
does not use any energy
Later, discoveries in physics showed that energy
cannot be created out of thin air
Perpetual motion is a futile endeavor
Understanding the impossible helps us channel our
energies towards the more useful.
7
The laws of computation
Just like the laws of physics tell uswhat is
(im)possible for nature to do...
...the laws of computation tell us what is
(im)possible for computers.
8
Automata theory
Automata theory studies the laws of computation.
In reality, the laws of computation are not quite
understood, but automata theory is a good start.
9
A simple computer
SWITCH
BATTERY
input switch output light bulb actions flip
switch states on, off
10
A simple computer
SWITCH
f
BATTERY
on
start
off
f
input switch output light bulb actions f for
flip switch states on, off
bulb is on if and only if there was an odd number
of flips
11
Another computer
1
off
1
off
start
1
2
2
BATTERY
2
2
1
2
on
off
1
inputs switches 1 and 2 actions 1 for flip
switch 1 actions 2 for flip switch 2 states
on, off
bulb is on if and only if both switches were
flipped an odd number of times
12
A design problem
4

Design a circuit where the light is on only when
all switches are flipped the same number of times
13
A design problem

• Such devices are difficult to reason about,
  because they can be designed in an infinite
  number of ways
• By representing them as abstract computational
  devices, or automata, we will learn how to answer
  such questions

f
on
off
f
14
These devices can model many things

• They can describe the operation of any small
  computer, like the control component of an alarm
  clock or a microwave
• They are also used in lexical analyzers to
  recognize well formed expressions in programming
  languages

ab1 is a legal name of a variable in C 5u is not
15
Different kinds of automata

• This was only one example of a computational
  device, and there are others
• We will look at different devices, and look at
  these kinds of questions
• What kinds of problems can a given type of device
  solve?
• What things are impossible for this kind of
  device?
• Is one type of device more powerful than another?

16
Some devices we will see
17
Some highlights of the course

• Finite automata
• We will understand what kinds of things a device
  with finite memory can do, and what it cannot do
• Introduce simulation the ability of one device
  to imitate another device
• Introduce nondeterminism the ability of a device
  to make arbitrary choices
• Push-down automata
• These devices are related to grammars, which
  describe the structure of programming (and
  natural) languages

18
Some highlights of the course

• Turing Machines
• This is a general model of a computer, capturing
  anything we could ever hope to compute
• But there are many things that computers cannot
  do

Given the code of a computer program, can
you tell if it prints banana?
include ltstdio.hgt main(t,_,a)char
areturn!0ltt?tlt3?main(-79,-13,amain(-87,1-_, ma
in(-86,0,a1)a))1,tlt_?main(t1,_,a)3,main(-94,-
27t,a)t2?_lt13? main(2,_1,"s d
d\n")916tlt0?tlt-72?main(_,t, "_at_n','/w/wc
dnr/,r/de,/,/w,/wqn,/l,,/nn,/
n,/\ qn,/k,/'r 'd'3,wK
w'K'e'dq'l \ q'd'K!/kq'reKKw'reKK
nl'/qn'))w'))nl'/n'drw' i
\ )nl!/nn' rw'r ncnl'/l,'K rw'
iKnl'/wqn'wk nw' \ iwkKKnl!/w'lw'
i nl'/q'ldr'nlwb!/de'c
\ nl'-rw'/,'nc,',nw'/kd'e'rdq
w! nr'/ ') rl'n' ') \ '(!!/") tlt-50?
_a?putchar(31a)main(-65,_,a1)main((a'/'
)t,_,a1) 0ltt?main(2,2,"s")a'/'main(0,m
ain(-61,a, "!ekdc i_at_bK'(q)-wnr3l,\nuwlo
ca-Om .vpbks,fxntdCeghiry"),a1)
banana
?
19
Some highlights of the course

• Time-bounded Turing Machines
• Many problems are possible to solve on a computer
  in principle, but take too much time in practice
• Traveling salesman Given a list of cities, find
  the shortest way to visit them and come back home
• Easy in principle Try the cities in every
  possible order
• Hard in practice For 100 cities, this would take
  100 years even on the fastest computer!

Beijing
Xian
Chengdu
Shanghai
Guangzhou
Hong Kong
20
Preliminaries of automata theory

• How do we formalize the question
• First, we need a way of describing the problems
  that we are interested in solving

Can device A solve problem B?
21
Problems

• Examples of problems we will consider
• Given a word s, does it contain the subword
  fool?
• Given a number n, is it divisible by 7?
• Given a pair of words s and t, are they the same?
• Given an expression with brackets, e.g. (()()),
  does every left bracket match with a subsequent
  right bracket?
• All of these have yes/no answers.
• There are other types of problems, like Find
  this or How many of that but we wont look at
  them.

22
Alphabets and strings

• A common way to talk about words, numbers, pairs
  of words, etc. is by representing them as strings
• To define strings, we start with an alphabet
• Examples

An alphabet is a finite set of symbols.
S1 a, b, c, d, , z the set of letters in
English S2 0, 1, , 9 the set of (base 10)
digits S3 a, b, , z, the set of letters
plus the special symbol S4 (, ) the
set of open and closed brackets
23
Strings
A string over alphabet S is a finite sequenceof
symbols in S.

• The empty string will be denoted by e
• Examples

abfbz is a string over S1 a, b, c, d, ,
z 9021 is a string over S2 0, 1, , 9 abbc
is a string over S3 a, b, , z, ))()(() is
a string over S4 (, )
24
Languages
A language is a set of strings over an alphabet.

• Languages can be used to describe problems with
  yes/no answers, for example

L1 The set of all strings over S1 that
contain the substring fool L2 The set of
all strings over S2 that are divisible by 7
7, 14, 21, L3 The set of all strings of
the form ss where s is any string over a, b,
, z L4 The set of all strings over S4 where
every ( can be matched with a subsequent )
25
Finite Automata
26
Example of a finite automaton
f
on
off
f

• There are states off and on, the automaton starts
  in off and tries to reach the accept state on
• What sequences of fs lead to the accept state?
• Answer f, fff, fffff, f n n is odd
• This is a finite automaton over alphabet f

27
Deterministic finite automata

• A deterministic finite automaton (DFA) is a
  5-tuple (Q, S, d, q0, F) where
• Q is a finite set of states
• S is an alphabet
• d Q S ? Q is a transition function
• q0 Î Q is the initial state
• F Í Q is a set of accepting states (or final
  states).
• In diagrams, the accepting states will be denoted
  by double loops

28
Example
0
0,1
1
1
0
q0
q1
q2
table of transition function d
alphabet S 0, 1 states Q q0, q1,
q2 initial state q0 accepting states F q0, q1
inputs
0
1
q0
q0
q1
q1
q2
q1
states
q2
q2
q2
29
Language of a DFA
The language of a DFA (Q, S, d, q0, F) is the set
of all strings over S that, starting from q0 and
following the transitions as the string is read
leftto right, will reach some accepting state.
f
on
M
off
f

• Language of M is f, fff, fffff, f n n is
  odd

30
Examples
S 0, 1
S a, b
q0
b
a
0
1
q1
q3
b
a
1
q0
q1
b
a
a
b
0
q2
q4
a
b
0
1
0, 1
1
0
q2
q0
q1
What are the languages of these automata?
31
Examples

• Construct a DFA over alphabet 0, 1 that accepts
  all strings with at most three 1s

32
Examples

• Construct a DFA over alphabet 0, 1 that accepts
  all strings with at most three 1s
• Answer

0
0
0, 1
0
0
1
1
1
1
q4
q2
q0
q1
q3
33
Examples

• Construct a DFA that accepts the language

L 010, 1
( S 0, 1 )
34
Examples

• Construct a DFA that accepts the language
• Answer

L 010, 1
( S 0, 1 )
1
0
q0
q01
q010
0
0
1
qe
0, 1
1
0, 1
q1
qdie
0, 1
35
Examples

• Construct a DFA over alphabet 0, 1 that accepts
  all strings that end in 101

36
Examples

• Construct a DFA over alphabet 0, 1 that accepts
  all strings that end in 101
• Hint The DFA must remember the last 3 bits of
  the string it is reading

37
Examples

• Construct a DFA over alphabet 0, 1 that accepts
  all strings that end in 101
• Sketch of answer

0
0
q000
1
0
q00
1
q001
q0
1

0
q01

qe
1
q101
q10
0

1
q1

1
q11
1
q111
1