Automatic Bibliographic Extraction System ABES

1
Automatic Bibliographic Extraction System (ABES)

• NLP Term Project
• 8 December 2005
• Nick Gorski

2
What is it?
ABES is a software project that takes as input a
document consisting of sentences and returns a
database containing various information about all
of the people mentioned in that document. In
effect, it creates a concise and accurate
bibliography for every person mentioned.
3
ABES algorithm

• First pass names consisting of NNPs
• Name extraction
• Name resolution
• Information extraction
• Entry
• Second pass names represented by pronouns
• Update possible antecedents
• Pronoun extraction
• Pronoun/antecedent resolution
• Information extraction and entry

4
Name extraction and resolution

Name extraction ABES extracts groups of NNPs
contained within NPs. Name resolution After
names are extracted, titles (such as Mr. and
Senator) are removed and stored away. The name
is then either resolved with a name already in
the data- base (Gorski resolves with Nick
Gorski) or are entered under a new name if no
resolution can be made.
5
Information extraction
Types of information that can be extracted
include
Titles Gender Facts Quotes Actions Sentences
Titles include both proper titles (Mr.) as
well as common nouns used as titles
(Senator) Gender information is extracted from
titles and used later for pronoun/antecedent
resolution Extracted from appositions following a
name Extracted from direct quotes as well as
clauses attached to the verb to say Verb
phrases in which a person acts as the subject All
sentences mentioning a person are listed for
completeness
6
Pronoun/Antecedent resolution

• Some pronouns are resolved in a second pass
  through the
• sentences.
• Steps taken during resolution
• Two lists of possible antecedents are maintained
  the first is
• names appearing in the previous sentence
  containing names,
• while the second lists names appearing in the
  current sentence.
• 2. Pronouns that fit one of two simple cases are
  resolved with
• their name the name then appears in double
  parentheses in
• the sentence.
• 3. Information is extracted as in the previous
  pass.

7
Pronoun/Antecedent resolution
NO
Does the sentence contain only one pronoun?
Dont resolve
YES
Does the current sentence contain a name?
Does the name grammatically appear before the
pronoun in the sentence?
YES
NO
NO
YES
Resolve the pronoun with the best fitting
possible antecedent
Resolve with the name in the current sentence
The best fitting name is one that was
mentioned in the previous sentence containing a
name, and out of the names that did not cause a
gender conflict was most important in the
sentence (for example, subjects are preferred
over objects, etc.)
8
Simple, contrived examples
Justice John G. Roberts discussed Senator Gerald
B. H. Solomon, a conservative Republican from New
York.
Nick Gorski beat Vishal Soni in foosball
yesterday. He is still at the top of the ATL
foosball ladder. Will anyone ever be able to beat
him? Trayton Otto, currently in second place on
the ladder, thinks that he will.
Name Vishal Soni Sentences Nick
Gorski beat Vishal Soni in foosball yesterday .
Name Nick Gorski Actions beat
Vishal Soni in foosball yesterday is still
at the top of the ATL foosball ladder
Sentences Nick Gorski beat Vishal Soni in
foosball yesterday . He ((Nick Gorski)) is
still at the top of the ATL foosball
ladder . Will anyone ever be able to
beat him ((Nick Gorski)) ? Name Trayton
Otto Facts second place on the ladder
Sentences Trayton Otto , second place
on the ladder , thinks that he ((Trayton
Otto)) will .
Name Gerald B. H. Solomon Titles 'Senator'
Facts a conservative Republican
Sentences Justice John G. Roberts
discussed Senator Gerald B. H. Solomon , a
conservative Republican from New York .
Name John G. Roberts Titles 'Justice'
Actions discussed Senator Gerald B. H.
Solomon , a conservative Republican from
New York Sentences Justice John G.
Roberts discussed Senator Gerald B. H.
Solomon , a conservative Republican from
New York .
9
New York Times article
In an abrupt political shift after a drubbing at
the polls three weeks ago, Gov. Arnold
Schwarzenegger named a former top Democratic
official on Wednesday as his chief of staff. The
move touched off howls from conservative
Republicans, who called the appointment a
betrayal of the party that helped him win the
governorship two years ago. The governor's new
top aide is Susan P. Kennedy, 45, who was deputy
chief of staff and cabinet secretary to Gov. Gray
Davis, the Democrat Mr. Schwarzenegger deposed in
a recall election. Ms. Kennedy, an advocate of
abortion rights and gay rights, is a former
executive director of the California Democratic
Party and communications director for Senator
Dianne Feinstein, a Democrat. As chief of
staff, she will direct the governor's staff and
be responsible for carrying out his agenda. Mr.
Schwarzenegger said Ms. Kennedy knew how the
levers of power worked in the capital and was
more interested in getting things done than in
partisan mudslinging. "She's willing to put her
Democratic philosophy aside and do the job to
fulfill my vision," he said at a news briefing
Wednesday in Sacramento. Ms. Kennedy succeeds
Pat Clarey, a Republican who served in the
administration of Gov. Pete Wilson. Since the
electoral defeat, it was widely believed that Mr.
Schwarzenegger would shake up his inner circle,
beginning with Ms. Clarey. She will stay on
through the transition in January, the governor
said.
Name Gray Davis Titles 'Gov.' Facts
the Democrat Mr. Schwarzenegger deposed in a
recall election Sentences The governor
's new top aide is Susan P. Kennedy , 45 , who
was deputy chief of staff and cabinet secretary
to Gov. Gray Davis , the Democrat Mr.
Schwarzenegger deposed in a recall election .
Name Pete Wilson Titles 'Gov.'
Sentences Ms. Kennedy succeeds Pat Clarey
, a Republican who served in the administration
of Gov. Pete Wilson .
10
Name Arnold Schwarzenegger Titles 'Gov.',
'Mr.' Gender Masculine Quotes Ms.
Kennedy knew how the levers of power worked in
the capital and was more interested in
getting things done than in partisan
mudslinging Actions named a former top
Democratic official on Wednesday as his
chief of staff would shake up his inner
circle , beginning with Ms. Clarey win the
governorship two years ago Sentences
In an abrupt political shift after a drubbing at
the polls three weeks ago , Gov. Arnold
Schwarzenegger named a former top
Democratic official on Wednesday as his chief of
staff . Mr. Schwarzenegger said Ms.
Kennedy knew how the levers of power
worked in the capital and was more interested in
getting things done than in partisan
mudslinging . Since the electoral defeat ,
it was widely believed that Mr.
Schwarzenegger would shake up his inner circle ,
beginning with Ms. Clarey . The
move touched off howls from conservative
Republicans , who called the appointment a
betrayal of the party that helped him
((Arnold Schwarzenegger)) win the governorship
two years ago . Name Dianne Feinstein
Titles 'Senator' Facts a
Democrat Sentences Ms. Kennedy , an
advocate of abortion rights and gay rights , is
a former executive director of the
California Democratic Party and
communications director for Senator Dianne
Feinstein , a Democrat .
Name Susan P. Kennedy Titles 'Ms.'
Gender Feminine Facts 45 an
advocate of abortion rights and gay rights
Actions knew how the levers of power
worked in the capital and was more
interested in getting things done than in
partisan mudslinging succeeds Pat
Clarey , a Republican who served in the
administration of Gov. Pete Wilson will
direct the governor 's staff and be responsible
for carrying out his agenda Sentences
The governor 's new top aide is Susan P.
Kennedy , 45 , who was deputy chief of
staff and cabinet secretary to Gov. Gray
Davis , the Democrat Mr. Schwarzenegger deposed
in a recall election . Ms. Kennedy
, an advocate of abortion rights and gay rights ,
is a former executive director of the
California Democratic Party and
communications director for Senator Dianne
Feinstein , a Democrat . Mr.
Schwarzenegger said Ms. Kennedy knew how the
levers of power worked in the capital and
was more interested in getting things done
than in partisan mudslinging . Ms. Kennedy
succeeds Pat Clarey , a Republican who served
in the administration of Gov. Pete Wilson .
As chief of staff , she ((Susan P. Kennedy))
will direct the governor 's staff and be
responsible for carrying out his agenda. Name
Pat Clarey Titles 'Ms.' Gender Feminine
Facts a Republican who served in the
administration of Gov. Pete Wilson
Actions will stay on through the
transition in January Sentences Ms.
Kennedy succeeds Pat Clarey , a Republican who
served in the administration of Gov. Pete
Wilson . And so on...
11
Is ABES perfect?

• Of course not!
• Tagging ABES relies on the Charniak parser for
  tagging. When the parser makes a mistake it
  causes ABES to make (sometimes humorous) mistakes
  as well.
• Pronoun/Antecedent resolution While newspapers
  mostly rely on simple, predictable sentence
  constructions, other domains dont and
  sometimes newspapers can trick ABES, too.
• Missed facts and actions ABES sometimes misses
  useful facts (e.g. appositions that arent NPs)
  and actions (e.g. VPs following appositions).