Democracy Theory and Practice

1
Democracy Theory and Practice

• A cross-disciplinary
• literature classification project

Donna Schenck-Hamlin, Bill Schenck-Hamlin,
Elizabeth Westmoreland, Kevin Keatley
Institute for Civic Discourse and Democracy
2
Begun as information support to Institute for
Civic Discourse and Democracy
3
GOAL Aggregate useful scholarship to promote
interdisciplinary democracy research
4
What type of democracy? W of S title phrases
1945 - 2007

• Social democracy 969
• American democracy 451
• Liberal democracy 274
• Industrial democracy 197
• Deliberative democracy 162
• Direct democracy 161
• Athenian democracy 137
• Economic democracy 129
• Local democracy 110
• Modern democracy 84
• Participatory democracy 74
• Socialist democracy 61
• Chinese democracy 60
• Political democracy 39
• German democracy 29
• Cosmopolitan democracy 27
• Popular democracy 24
• African democracy 23
• Western democracy 22

• Contemporary democracy 13
• European democracy 11
• Real democracy 11
• Canadian democracy 9
• Environmental democracy 7
• Technological democracy 7
• Militant democracy 7
• Nuclear democracy 4
• Proletarian democracy 3
• Military democracy 3
• Green democracy 2
• Basic democracy 2
• Concordant democracy 1
• Utilitarian democracy 1

5
Developed a database of over 21,000 references
for ICDDs website from various sources other
databases, websites, individual contributions.
6
QUESTION Can we develop a set of useful
descriptors covering most democracy research that
would optimize searching?
7
Objectives of the Project

1. Improve database searching through value-added
   subject descriptors
2. Test the reliability and validity of the
   descriptors coded by 4 persons
3. Map the emphases of democracy scholarship through
   a quarter-century, 1980-2005

8
Introduced 3 Levels of Descriptors

• CATEGORIES areas of democracy scholarship
  sought by ICDD researchers
• TOPICS public issues associated with democracy
  in a given document
• DOMAINS geographic level of interest addressed
  in a given document

9
CATEGORY Politics and Government about
current affairs in specific states
10
TOPIC Social Equity employing collective
resources to promote societal welfare
11
DOMAIN National focus on specific nations
12
Initial Set of ICDD-DevelopedCategories, Topics,
and Domains

• 1 Deliberation Facilitation Methods Models
• 2 Politics and Government
• 3 Online/Internet Communication E-democracy
• 4 Citizen Participation Civic Engagement
  Citizenship Enfranchisement
• 5 Democracy Education
• 6 Evaluation
• 7 Media and Democracy
• 8 Political/Philosophical Theory
• 9 Public Opinion
• 11 Social Influences
• 12 Economic Influences
• 13 Organizational Democracy
• 14 History
• 10 Undetermined 1 Does not apply to democracy
• 15 Undetermined 2 Need more info. to determine
• 16 Undetermined 3 Fits none of our descriptors
• 17 Arts and Letters
• 100 Environment

• 140 Foreign Relations
• 150 Poverty
• 160 Conflict Resolution Mediation
• 170 Pluralism Identity Multiculturalism Gender
  Race Relations
• 180 Development
• 190 Policy
• 200 Planning
• 210 Agriculture
• 220 Security Safety
• 230 Religion
• 240 Social Equity
• 1100 Community domain
• 1110 Regional domain
• 1120 State domain
• 1130 National domain
• 1140 International domain
• 1150 Global domain
• 1160 Rural domain

13
Methodology

• Downloaded records from Web of Science into
  Refworks title Democracy 1980-2005 18,834
  but w/o redundant book review titles
  7,276
• Content Variables 40 coder-applied descriptors
  17 democracy categories 15 topics 8 domains
• Sampling Procedure stratified sample,
  stratification variable year
• Sampling Fraction 30 or 2,170 titles
• Over 10,000 international scholarly journals
  indexed humanities, social sciences, sciences

14
Sampling Fraction compared the distribution of
selected descriptors across coders between two
halves of two randomly selected years. If the
two pairs had similar distributions the sample
size was sufficiently large. 30 was determined
to be an adequate sample fraction. Sampling
Procedure Randomly selected records
representing 30 of a given years citations w/
Democracy in the title
15
Checking the reliability of our classification
system

• Answers the question How likely is it that a
  team of catalogers will use the same descriptor
  (e.g. history) for a given document (e.g.
  Athenian Democracy)?
• If inter-coder reliability scores are low, that
  suggests that
• coders dont have a shared understanding of the
  meaning of the descriptor history or
• dont agree on its application to some titles

16
2 reliability scores are used, but each have a
problem

• RCA or Agreement (agreements/total titles
  reviewed)
• Too liberal If most of the time the descriptor
  history was rarely applied by either coder in
  the title set (they agreed that it got a 0 for
  nearly every title they reviewed), results are
  artificially high. What do you really know about
  their shared understanding of the term history
  if it was rarely applied by them?

17

• Kappa score (agreement is weighted to account for
  the quantity of non-selected variables)
• Too conservative If there was only a small
  number of titles out of 100 selected by the two
  coders as history, say 8, and they only agreed
  on 4 titles out of the 8, you get a very low
  score, despite the coders agreement that 90
  titles had nothing to do with history.

18
Improving Reliability

• Taking a conservative approach,.4 kappa is
  adequate, but gt.59 kappa is excellent.
• Catalogers consult with one another on
  disagreements, visit the documents again, then
  fine-tune the descriptor definition or the
  descriptor itself.
• Over time, scores can improve as both the coders
  and the schema itself (the terms and their
  definitions) improve.

19
INITIAL INTER-CODER RELIABILITY SCORES FOR
CATEGORIES
Category (a primary category of democracy literature) RCA Kappa
Politics and Government .79 .52
Political/Philosophical Theory .87 .57
History .90 .59
Social Influences needs work .87 .18
Economic Influences .92 .46
Citizen Participation, Civic Engagement, Citizenship, Enfranchisement needs work .91 .36
Organizational Democracy .96 .62
Democracy and Education .98 .79
Arts and Letters .98 .64
Deliberation, Facilitation, Methods, Models .98 .52
Public Opinion .98 .52
Media and Democracy .99 .76
Online/Internet Communication, E-democracy .99 .81
20

INITIAL INTER-CODER RELIABILITY SCORES FOR TOPICS
Topic (a public issue addressed in relation to democracy) RCA Kappa
Pluralism, Identity, Multiculturalism, Gender, Race Relations .94 .55
Social Equity needs work .93 .39
Foreign Relations .94 .45
Development .94 .42
Security Safety .95 .43
Policy needs work .95 .32
Religion .99 .79
Science .99 .69
Environment .98 .40
Health .99 .58
Conflict Resolution Mediation .98 .43
Technology .98 .40
Poverty needs work .98 .24
Planning needs work .98 .35
Agriculture .99 0
21

INITIAL INTER-CODER RELIABILITY SCORES FOR DOMAINS
Domain (a geographical level of focus for the article) RCA Kappa
Community .97 .51
Regional needs work .99 .20
State needs work .98 .33
National .83 .61
International .90 .55
Global .99 .48
Rural .99 0
Urban needs work .99 .31

22
Results for Reliability and Validity

• Coding Reliability The majority of descriptors
  achieved adequate to excellent inter-coder
  reliability
• Content Validity 2 of articles could not be
  classified based on our category system
  (Undetermined 1,2,3)
• To move on make adjustments to descriptors
  that need more work, or drop them entirely

23
Given a classification system that was adequate
in describing the 30 sample of democracy
articles, what trends are shownover 25 years?
24
Number of titles in each category of democracy
research
Politics and Government 845 39
Political/Philosophical Theory 513 24
History 313 14
Social Influences 261 12
Economic Influences 238 11
Citizen Participation, Civic Engagement, Citizenship, Enfranchisement 228 10
Organizational Democracy 151 7
Democracy and Education 102 5
Arts and Letters 75 3
Deliberation, Facilitation, Methods, Models 58 3
Public Opinion 51 2
Media and Democracy 46 2
Online/Internet Communication, E-democracy 41 2
25
Politics and Government

• On the Prospects of Democracy in Russia
• Direct Democracy - Politicoeconomic Lessons from
  Swiss Experience
• The Spanish Transition to Democracy seen through
  the Spanish Database Isoc
• Presidential Democracy in America - Toward the
  Homogenized Regime

26
(No Transcript)
27
Political/Philosophical Theory

• Does democracy reveal the voice of the people?
  Four takes on Rousseau
• The crisis of modern democracy in theoretical
  reflection.
• Can democracy promote the general welfare?
• Democracy, merit, and presumptive virtue

28
(No Transcript)
29
Citizen Participation, etc.

• Democracy in our backyards - A study of community
  involvement in administrative decision making
• Participation, decentralization, and civil
  society - Indigenous rights and democracy in
  environmental planning
• Viewpoint Implications of participatory
  democracy for public land planning

30
(No Transcript)
31
Number of titles in each public issue topic of
democracy research
Pluralism, Identity, Multiculturalism, Gender, Race Relations 153 7
Social Equity 147 7
Foreign Relations 147 7
Development 97 4
Security Safety 80 4
Policy 74 3
Religion 66 3
Science 41 2
Environment 38 2
Health 36 2
Conflict Resolution Mediation 36 2
Technology 32 1
Poverty 32 1
Planning 30 1
Agriculture 1 lt 1
32
Pluralism, Multiculturalism, etc.

• Group dominance and the myth of racial democracy
  Antiracism attitudes in Brazil
• Citation Classic - the Politics of Accommodation
  - Pluralism and Democracy in the Netherlands
• The Modernization of Castes and the Indianization
  of Democracy - the Case of the Lingayat

33
(No Transcript)
34
Social Equity

• Democracy, inequality, and inflation
• On social democracy and the Kibbutz
• Does liberte equals egalite? A survey of the
  empirical links between democracy and inequality
  with some evidence on the transition economies
• Mergers, concentration, and the erosion of
  democracy

35
(No Transcript)
36
Number of titles in each domain of democracy
research
National 762 35
International gt 1 nation-state e.g. NAFTA countries 310 14
Community 78 3
Global encompasses more than geopolitical entities e.g. biosphere 28 1
State subdivision of a nation 27 1
Urban 22 1
Regional gt1 state within a nation e.g. Prairie Provinces 14 lt1
Rural 2 lt1
37
Qualifications about trends in this data

• Actual frequencies for some descriptors were
  small
• Increases in W of S indexed journals could affect
  trends in democracy titles
• Much democracy research is moving from journal to
  Internet publication, so how representative is W
  of S?

38
Emphases and trends in democracy titles over 25
years

• Categories Politics and Government make up
  less than half of the literature many other
  cross-disciplinary categories make up the rest.
• Topics Pluralism Identity Multiculturalism
  Gender Race Relations leads other topics in
  total quantity, even more than Foreign
  Relations, which is tied with Social Equity in
  total quantity.
• Domain National has been the primary
  geographic focus.

39
Where do we go from here?

• Adjust our category scheme with advice from
  subject field experts
• Run end-user tests for utility of category
  browsing
• Apply category scheme to remainder of the online
  Refworks database

40
(No Transcript)