Perspectival Diversity and Consensus Analysis - PowerPoint PPT Presentation

1 / 35
About This Presentation
Title:

Perspectival Diversity and Consensus Analysis

Description:

Perspectival Diversity and Consensus Analysis John Gatewood . . . . . . Lehigh University John Lowe . . . . . . . Cultural Analysis Group AAA Meetings, Philadelphia ... – PowerPoint PPT presentation

Number of Views:133
Avg rating:3.0/5.0
Slides: 36
Provided by: JohnG278
Learn more at: https://www.lehigh.edu
Category:

less

Transcript and Presenter's Notes

Title: Perspectival Diversity and Consensus Analysis


1
Perspectival Diversity and Consensus Analysis
  • John Gatewood . . . . . . Lehigh University
  • John Lowe . . . . . . . Cultural Analysis Group
  • AAA Meetings, Philadelphia, Dec 5, 2009

2
Preview
  • INTRODUCTORY REMARKS
  • Problem of culture-sharing (and non-sharing)
  • Basic patterns of inter-informant agreement
    addingperspectival diversity to the list
  • OUR CURRENT STUDY
  • Assessing effects of different distributional
    patterns on consensus analysiss key indicators
  • Some initial findings
  • CONCLUSIONS
  • Methodological lessons for researchers
  • Future directions

3
INTRODUCTORY REMARKS
4
Problem of Culture-Sharing
  • By definition, culture is socially transmitted
    knowledge hence, it must be shared but
    sharing is always a matter of degree
  • Hence, two related issues for any given cultural
    domain
  • How much knowledge is shared? (the AVERAGE
    cultural competence)
  • How is the knowledge socially distributed? (the
    DISTRIBUTIONAL PATTERN)
  • KEY INSIGHT assess degree of culture-sharing by
    examining patterning of inter-informant agreement

5
Basic Patterns of Agreement
  • Boster (1980, 1985) four basic patterns of
    agreement (paraphrasing expanding)
  • UNIFORM agreement traditional view of culture
  • RANDOM agreement free variation ? no culture
  • EXPERTISE gradient experts tend to agree with
    one another whereas non-experts deviate
    randomly
  • SUBCULTURAL variation more than one school
    of thought
  • Competing answer sets different groups,
    different truths
  • Complementary knowledge different groups
    systematically know different things
  • Romney, Weller Batchelder (1986) cultural
    consensus theory consensus analysis

6
Perspectival Diversity a 5th Pattern
  • Pilot study of credit union employees (Gatewood
    Lowe 2006)
  • No consensus in sample
  • No identifiable subcultural groups
  • gt Fish-scale overlappings of partial
    knowledge perspectival diversity
  • i.e., social interaction and knowledge among the
    employees was rather departmentalized their
    understandings of credit unions reflected what
    they needed to know to perform their own jobs,
    not necessarily what might be relevant to other
    people
  • _______________________
  • Pilot studys conclusion about no
    consensus turned out to be an artifact of our
    failure to counter-balance items in the
    questionnaire form (see Gatewood Lowe 2008)
    but thats another story

7
  • To generalize, perspectival diversity occurs
    when
  • All individuals have limited knowledge with
    respect to a given domain and to approximately
    the same degree
  • Each individuals range of knowledge only
    partially overlaps with the ranges known by
    others
  • And, consistent with this definition, different
    geometries of perspectival diversity are
    possible e.g., circular pattern, linear
    pattern, taxonomic-hierarchical, overlapping
    polygons on a surface, etc.

8
OUR CURRENT STUDY
9
  • RESEARCH QUESTION
  • Ceteris paribus, do the different distributional
    patterns affect the key indicators of consensus
    analysis?
  • As the average knowledge in a sample varies, do
    different distributional patterns show
    consensus more readily than other patterns?
  • Do some distributional patterns mask cultural
    consensus when other patterns reveal it?
  • If NO ? nothing to worry about yippee!
  • If YES ? distributional patterning has an
    independent effect that needs to be taken into
    account when interpreting results of consensus
    analyses

10
  • ITEM FORMAT
  • Counter-balanced Likert-style questions, i.e.,
    6-point strongly agree to strongly disagree
    response scale because these are so common in
    survey research
  • ANALYSES
  • Such data can be analyzed two ways
  • INFORMAL MODEL of consensus analysis i.e.,
    input to factor analysis is a Resp x Resp
    correlation matrix (data treated as
    interval-scale)
  • FORMAL MODEL of consensus analysis i.e., input
    to factor analysis is a chance-corrected
    agreement matrix (data treated as nominal-scale,
    e.g., dichotomized responses)

11
Research Design with average knowledge and
distributional pattern as manipulated variables
Distributional Patterns Key Indicators Key Indicators Key Indicators variety of other measures
Distributional Patterns Ratio of eigenvalues Mean 1st factor loading Number of negative loadings variety of other measures
Uniform-to-Random ? ? ? ?
Expertise ? ? ? ?
Subcultures ? ? ? ?
Perspectival ? ? ? ?
12
Theoretical Predictions
Distributional Pattern Prediction
Uniform-to-Random ? None serves as benchmark for other patterns
Expertise Gradient ? INCREASE consensus indicators
Subcultures ? DECREASE consensus indicators
Perspectival ? ???
13
Implementation
  • How to experimentally manipulate key parameters
    for different distributional models while holding
    others constant ?? computer simulation to the
    rescue !
  • See Excel data-generating file Excel
    findings file

14
SOME INITIAL FINDINGS
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Key Findings
  • Distributional pattern has an independent effect
    with respect to consensus indicators
  • w/r/to RATIO OF EIGENVALUES
  • ( compared to the Uniform-to-Random model )
  • Expertise patterns INCREASE the ratio
  • Subcultural patterns DECREASE the ratio
  • Perspectival patterns DECREASE the ratio
  • w/r/to MEAN 1st FACTOR LOADING
  • Distributional patterns have little effect on
    this indicator, AND consensus analysis estimates
    actual competence very well with one exception
  • Expertise (triangular) pattern INFLATES mean
    competence as well as the ratio of eigenvalues
    (because it violates the homogeneity of items
    assumption?)

27
  • Expertise (rectangular) pattern
  • The range of expertise about the same average
    competence also makes a difference greater
    range ? larger ratio of eigenvalues
  • Subcultural patterns
  • As expected, systematic differences in sub-group
    knowledge undermine consensus
  • By question sub-groups may still show consensus
    overall, with the groups showing up on the 2nd
    factor
  • Different answer keys just destroy consensus
  • Formal consensus model (on dichotomized data)
    and informal consensus model yield very similar
    results

28
CONCLUSIONS
29
Lessons for Researchers
  • Since the ratio of eigenvalues is particularly
    sensitive to the distributional pattern of
    knowledge, REPORT MORE than just the ratio
  • Minimally, include
  • Ratio of 1st to 2nd eigenvalues
  • Mean 1st factor loading (and st.dev. of those
    loadings)
  • Number of negative loadings
  • And, comparable guidelines should be
    established for evaluating these additional
    measures e.g., 0.500 for mean loading fewer
    than 5 negative loadings in sample
  • These output statistics are necessary for more
    meaningful interpretations of ones data

30
  • IF your data show a hefty mean 1st factor loading
    but a low ratio of eigenvalues DO NOT leap to
    the conclusion that either (a) subcultures exist
    or (b) there is free variation in the domain
  • You may be dealing with a case of PERSPECTIVAL
    DIVERSITY which would warrant further
    investigation, such as examining the inter-person
    correlation matrix and the response-profiles of
    individuals one at a time to see if you can
    detect a subtle social patterning to
    who-knows-what

31
  • Try to formulate questions that are EQUALLY
    DIFFICULT ( and ask lots of questions )
  • Violations of Assumption 3 will inflate both the
    obtained ratio of eigenvalues the mean 1st
    factor loading
  • e.g., Expertise (triangular) pattern INFLATES
    both indicators
  • So ex post factoif you notice that some
    questions were much easier than others, then
    either (a) use higher threshold criteria before
    claiming the data conform to the cultural
    consensus model, and/or (b) remove the very
    easy questions and re-analyze

32
Future Directions
  • Developing additional geometries of
    perspectival overlapping
  • Analyzing relations between a variety of measures
    describing the initial Resp x Resp correlation
    matrix and the key indicators from consensus
    analysis
  • Exploring different instantiations of guessing
    (binomial, truncated-normal, beta distributions)
  • Exploring other possible measures from the factor
    analysis as predictors of culture-sharing, e.g.,
    1st eigenvalue divided by sample size

33
Thank you
  • and we would be happy to continuetalking with
    interested folksafter the session

34
Scalar data analyzed via Informal Method
(vertical axis)VSDichotomized data analyzed via
Formal Method (horizontal axis)
Mean 1st factor loadings( r .969 )
Ratios of eigenvalues( r .933 )
35
Real Survey Item
Real Survey Item
Simulated Item
POTENTIAL PROBLEM Frequency distributions of
itemsfrom real surveys (top panels)are more
graded than oursimulated data (lower
right) something were trying to resolve,but
not there yet
Write a Comment
User Comments (0)
About PowerShow.com