Polar Bears and Their Fellow Travellors

1
Polar Bears and Their Fellow Travellors

• Dr. Stuart W. Shulman

2
http//erulemaking.ucsur.pitt.edu/
3
Jamie Callan, Carnegie Mellon University -
Computer Science Eduard Hovy, USC/Information
Sciences Institute - Computer Science David
Scholsberg, Northern Arizona University -
Political Theory Stuart Shulman, University of
Pittsburgh - Political Science Mack C. Shelley,
Iowa State University - Political Science
Statistics Stephen Zavestoski, University of San
Francisco - Sociology
4

• This research has been supported by grants from
  the National Science Foundation and supplemented
  through inter-agency agreements with the U.S. EPA
  the U.S. Fish Wildlife Service
• EIA 0089892, 30,000 (2001-2002)
• SGER Citizen Agenda-Setting in the Regulatory
  Process Electronic Collection and Synthesis of
  Public Commentary
• EIA 0327979, 0328175, 0328914 0328618, 100,000
  (2003-2004)
• SGER Collaborative A Testbed for eRulemaking
  Data
• SES 0322662, 260,000 (2003-2005)
• Democracy and E-Rulemaking Comparing
  Traditional vs. Electronic Comment from a
  Discursive Democratic Framework
• IIS 0429293, 0429102, 0429360 0429243, 1.375M
  (2004-2007)
• Collaborative Research Language Processing
  Technology for Electronic Rulemaking
• IIS-0705566, 900,000 (2007-2009)
• Collaborative Research III-COR From a Pile of
  Documents to a Collection of Information A
  Framework for Multi-Dimensional Text Analysis
• Any opinions, findings and conclusions or
  recommendations expressed in this material are
  those of the authors and do not necessarily
  reflect those of the National Science Foundation

5
(No Transcript)
6
(No Transcript)
7
Stuart W. Shulman, "An Experiment in Digital
Government at the United States National Organic
Program," Agriculture and Human Values 20, 3
(Fall 2003), 253-265.
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
Source October 14, 2004 Press release from the
University of Pittsburghhttp//erulemaking.ucsur.
pitt.edu/doc/ReportPressRelease.pdf
13
One Core Problem Duplicate Detection

• Many public comments are form letters or edited
  form letters
• Real grassroots or astroturf created by
  interest groups

14
What if it were all paper?

• Past research focused on the mercury dataset
• 536,000 emails, all plain text
• Many duplicative (similar identical) comments
• If all 1.8 gigabytes were on paper (which it is)
• it would weigh 5,350 pounds (about 2.7 tons)
• it would make a stack 214 feet high

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
OAR-2003-0214
OAR-2001-0017
OAR-2003-0214
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
Our application sorts the e-mail into major
message groups
and sub-groups
49
The Biggest NRDC Sub-Group
50
The 2nd Biggest NRDC Sub-Group
51
The 3rd Biggest NRDC Sub-Groupwith Deleted Text
Shown
52
The 5th Biggest NRDC Sub-Group
53
5th Largest NRDC Sub-Group Disaggregated
54
Most Unique NRDC Comments at the End of the
Sub-group List
55
Most Unique NRDC Comments at the End of the
Sub-group List
56
(No Transcript)
57
As a supporter of Defenders of Wildlife and
someone concerned about global climate change, I
am writing today to voice my strong support for
the proposal to list polar bears as "Threatened"
under the Endangered Species Act. The U.S.
Geological Survey USGS has just published a
series of reports predicting that melting Arctic
sea ice due to global warming could result in
polar bear population declines of two-thirds or
more in the next 50 years. In light of these dire
- and in the USGS's own words, conservative -
predictions, I strongly urge you to grant
Endangered Species Act protections to polar bears
immediately. In Alaska, we already see the
effects of climate change, with the birth and
survival rates of our polar bears plummeting as
ice packs shrink and the bears' hunting
opportunities become more limited. The USGS's
research underscores what we already know -
global warming is real and is taking a
devastating toll on polar bears. In order to save
these magnificent animals, listing polar bears as
threatened must be coupled with concrete steps to
reduce greenhouse gas emissions in order to
address the root cause of global warming. In
addition, critical habitat for polar bears must
be identified and protected. Essential
protections against harmful oil and gas drilling
in the Arctic National Wildlife Refuge, Bristol
Bay and other crucial habitat areas must be
provided to ensure the survival of polar bears.
Thank you for considering my comments.
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
Total number of emails per day over four public
comment periods
64
A new breed of mass public commenters The
Plebers
Shulmans Definition commenters contributing to
the plebiscitary notion of electronic rulemaking
by sending two or more emails.
65
Largest email campaigns in the polar bear listing
66
Working Hypotheses

• H1 Modified form letters add few new issues or
  useful information for agencies to consider.
• H2 Longer modifications are more likely to
  introduce new issues and useful information than
  shorter modifications.

67
Data Collection Method

• Use DURIAN to identify the largest email campaign
  in the mercury rulemaking
• MoveOn.org web-based campaign was responsible for
  223,300 of the 536,967 emails
• Set aside the 122,474 unmodified exact dupes
• Identify the longest modifications in the 100,828
  near dupes
• Extract the 1,000 longest modified MoveOn.org
  form letters and prepare them for coding in
  ATLAS.ti

68
Coding the Mercury Top 1000 Near Dupes
Raw Counts, Kappa and F-Measure Reliability
Scores from Six Comparisons Based on Coding Team
2 Members Overlap on 252 of the Documents
69
Whats Next?

• Continued efforts to build more and better Tools
  for Rules and to shape federal eRulemaking
• A new grant starts this fall
• Funded partnership with US Fish Wildlife
  Service
• Citation discovery tools (3/100 so far)
• Continued analysis and writing about what is in
  the modified form letters
• Note Probably nothing too surprising to report
• More dialogue with the groups and the officials
  who get their emails
• Is this really the best way to do e-democratic
  engagement?

70
Thank-you!

• Dr. Stuart W. ShulmanUniversity of Pittsburgh
• Shulman_at_pitt.edu (e-mail)
• http//shulman.ucsur.pitt.edu (home page)
• 412.624.3776 (voice)