NLP history, trends, and resources

1
NLP history, trends, and resources

2
NLP tensions

• Human-centered vs. engineering
• Neat vs. scruffy
• Symbolic vs. stochastic
• Theory-dependent vs. theory-independent
• Academic vs. industrial
• Top-down vs. bottom-up
• Black-box vs. glass-box
• Empiricist vs. rationalist

3
Human-centered vs. engineering

• Human-centered
• Model/assist an idealized (or perhaps all),
  average (or perhaps skilled) human being in the
  performance of some language task
• Stay as close to human performance as possible
• Engineering
• Forget/overcome human considerations
• Use any techniques possible, no matter how
  unhuman
• Airplanes dont flap their wings...

4
Neat vs. scruffy

• AI rich in controversy, debate
• The 1970s "logical/neat" AI vs.
  "procedural/scruffy" AI
• 1970s reasoning with explicit knowledge
  representations
• 1980s and 1990s, "connectionist" and "reactive"
  approaches to AI
• Had immediate implications for NLP

5
Rationalist approach

• Genetically predetermined capability (1960-1985)
• AI hand-code initial knowledge reasoning
  ability
• Linguistics innate language faculty, universals
  I-language generative grammar

6
Empiricist approach

• General cognitive operations (reasoning, analogy,
  pattern matching) used to leverage rich sensory
  input (1920-1960, now renewed focus)
• AI parameter instantiation via statistical
  techniques, pattern recognition, machine learning
• Linguistics must specify language model, induce
  parameters E-language corpora

7
Machine learning inroads

• Knowledge engineering is costly
• Corpora are more easily available
• A wide array of techniques is available
• Most NLP conference papers must have some
  component of learning

8
The NLP pendulum

• Knowledge engineering
• AI system dependent on human experts entering
  large amounts of knowledge
• Data-driven learning
• Developing some sort of abstract (often
  statistical) model of a problem and training the
  model on large amounts of data, either as it
  naturally occurs or with human annotation.
• Last 15 years knowledge engineering ? data-driven

9
Useful resources

• aclweb.org
• ACL Anthology (papers e.g. Coling, ACL)
• SIGs
• ACL Universe (resources)
• journals (e.g. Computational Linguistics)
• arXiv.org (Computation and Language)
• LinguistList.org
• Citeseer, Google Books, etc.
• HBLL resources ebrary, etc.

10
Main conferences/workshops

• (NA)ACL
• HLT
• Coling
• AAAI
• ICML
• NLDB
• LREC

11
Corpora

• Linguistic Data Consortium
• European Language Resources Association
• NIST
• ACLweb
• Various WWW sites