Computational Linguistics

1
Computational Linguistics

• An Introduction

2
Introduction

• Computational Linguistics within Artificial
  Intelligence
• Linguistics what it is, past influences
• Language underlying set of rules
• Linguistics todays issues and how they impact
  computational linguistics

3
Computational Linguistics

• sub-field of Artificial Intelligence
• Natural Language Processing (NLP)
• used in interfaces, search engines etc.
• applies techniques developed in Linguistics

4
Artificial Intelligence

• Connectionism
• based roughly on our brain neural nets
• nodes in network
• network learns - connections strengthen

• Classical AI
• symbols plus rules for manipulating symbols
• NLP
• words symbols
• grammar rules rules to manipulate words.

5
Classical AI

• successful in limited domain
• (eg Big Blue chess - used brute force)
• not so good when problems imbedded in world
• NLP
• When she hit the nail with the hammer the head
  flew off.
• requires world knowledge to disambiguate

6
Connectionism

• node (computing unit)
• connections between nodes vary in strength
• learning rule error signal ideal - actual
• input output error signal

7
Connectionism..some success

• connectionist net was trained to learn the past
  tense of verbs. (Rummelhart McClelland 1986)
• followed same U shaped learning pattern of
  children
• went
• learn rule over generalized goed
• master both regular and irregular past tense

8
Connectionismnot solution

• when children hear a sentence
• The dog chased the cat.
• spontaneously treat cat as if belonging to the
  same category as dog
• neural nets fail to generalize in this way...

9
Unique Challenges

• not all problems similar to other areas of AI
• for example early search engines
• what is a word?
• computer equals computers but not computation
• need a deeper analysis of language linguistics

10
Linguistics

• history
• evolutionary history of sets of related languages
  
• structures
• the structures that make up language
• (SVO-English) (SOV-Japanese) (VSO-Arabic)
• use
• what do words and sentences mean? Mental
  representations? Function of use?

11
Sub-fields of Linguistics

• phonetics (sounds in language)
• where sounds are made
• phonology (sound patterns in language)
• what sound combinations are allowed
• morphology (words and word structures)
• rules for combining morphemes
• syntax (sentence and sentence structures)
• rules for coming words in a sentence
• semantics (meaning)

12
Grammar

• descriptive
• how languages are actually used
• prescriptive
• the correct way to speak, based on current
  standards
• theoretical
• theories of language structures
• formalize rules we unconsciously apply when
  speaking

13
Examples of Rules We Apply(From English)

• Morphology
• encounter new verb in present tense flig
• would use it in the past tense fligged
• Phonology
• ptil, sorlint, bkat
• which one could be an English word?

14
Unconscious Rules

• only sorlint is a candidate for a new English
  word because
• Phonological Rule in English
• words cannot begin with two stop consonants
• p, b, t, d, k, g

15
Linguistics Early 20th Century

• structuralism classifying/taxanomic
• Focus on language form (S -gt NP VP) as opposed to
  meaning (S -gt complete thought)
• Edward Sapir
• analyzed Amerindian Languages
• Bloomfield
• influenced by behaviourism
• WWII
• need to analyze and learn many new languages

16
Language Form Underlying System of Rules

• Phrase Structure Rules
• Representing sentences tree structures
• Parts of Speech
• Constituents and Categories
• Ambiguity more than one possible way to
  parse sentence

17
Phrase Structure Rules

• Rules Lexicon
• S NP VP Det the
• NP Det N N child
• VP V V sleeps

18
Representation

• tree structures used to represent sentences
• sentences broken down into constituents
• S
• NP VP
• Det N V
• The child sleeps

19
Parts of Speech

• NOUNS
• person, animal, place, quality, idea
• Mary, university, thought, childhood
• PRONOUN
• refer to, or stand in for, a noun
• everyone, she, they
• VERB
• action, state of being
• walk, to be (is, are), know

20
Parts of Speech

• ADJECTIVE
• describes or modifies a noun
• happy, red, many (Many people like chocolate)
• ADVERB
• modifies a verb, adjective, adverb, phrase,
  clause, sentence
• slowly, very slowly, immediately (Immediately,
  they...)
• PREPOSITION
• links nouns, pronouns, and phrases to other parts
  of the sentence - express relationships in time
  or space between things and events
• to, before, beside, on, at, of, under, since,
  after.

21
Parts of Speech

• CONJUNCTION
• joins two or more words, phrases, or clauses
• and, but, or
• INTERJECTION
• used to protest, exclaim, command
• oh, wow, hey
• DETERMINER
• articles the, a/an (only two in English)
• demonstratives (modifies noun, indicates
  position of something in relation to speaker)
  that, those, this, these

22
Constituents and Categories

• Categories parts of speech
• (nouns, verbs, etc.)
• Constituents words that hang together
• can move together
• can be deleted together
• makes sense to talk of the meaning of a
  constituent structure (over other combinations)

23
Constituents and Categories

• S
• NP VP
• det N V NP PP
• det N P NP
• det N
• the dog chased the cat into the
  garden

24
Ambiguity in Phrase Structure

• ambiguity can result when there is more than one
  possible phrase structure for a sentence
• Example
• The girl saw the boy with the telescope.
• Who has the telescope?
• The girl? Or the boy?

25
Phrase Structure Rules

• S NP VP det the
• NP det N N girl, boy, telescope
• NP det N PP P with
• PP P NP V saw
• VP V
• VP V NP
• VP V NP PP

26
Phrase Structure Rules

• The girl has the telescope.
• S
• NP VP
• det N V NP PP
• the girl saw
• det N P NP
• the boy
  with
• 
  det N
• the telescope

27
Phrase Structure Rules

• The boy has the telescope.
• S
• NP VP
• det N V NP
• the girl saw
• D N PP
• the boy
• P NP
• with
• D N
• 
  the telescope

28
Phrase Structure Rules Complete Grammar?

• no
• many constructions impossible to describe
  adequately

29
Wh movement

• Jane said Paul believed Fido barked.
• Who said Paul believed Fido barked? (Jane)
• Who did Jane say believed Fido barked? (Paul)
• Who did Jane say Paul believed barked? (Fido)

30
Transformational Rules

• Wh - movement any amount of structure between
  original position of moved word and where it ends
  up.
• Phrase structure rules cannot account for it.

31
Transformational-Generative Grammar

• Noam Chomsky
• Syntactic Structures (1957)
• 1 - 100 most influential works in Cognitive
  Science in the 20th Century
• A Review of Skinners Verbal Behaviour (1959)
• Together launched Cognitive Revolution
• mind rules representation
• descriptive theories to explanatory theories

32
Universal Grammar

• language is innate
• each language is an example of UG
• biological basis of language

33
Universal Grammar Requires

• descriptive adequacy
• detailed enough to describe any possible
  construction in any human language
• explanatory adequacy
• simple enough to reflect the small set of inborn
  principles that allow humans to acquire and use
  language

34
Tension between two requirements of UG

• A theory has explanatory adequacy to the extent
  that it is able to show how, from the data
  available to a child, a child can get a
  descriptively adequate grammar.
• to be more descriptively adequate, need more
  descriptive devices
• more devices, harder it is for a child to gain
  descriptively adequate grammar from data

35
How does this help computational linguistics?

• Phrase structure rules inadequate to describe
  language.
• Transformational rules difficult to parse -
  seldom used in NLP.
• TG caused explosion in rules.
• Chomsky has since moved on to minimalist
  program.

36
Minimalist Program

• two levels of representation
• phonetic form
• logical form
• implication of language being innate - language
  has evolved (not mathematically elegant)
• does not follow rules perfectlyin some respects
  unusable

37
Is natural language processing possible?

• One reason for studying language - and for me
  personally the most compelling reason - is that
  it is tempting to regard language, in the
  traditional phrase, as a mirror of mind.
• Chomsky, 1975