Documents, Secondary Analysis and Official Statistics

1
Documents, Secondary Analysis and Official
Statistics

• Chapter 7

2
Extant Documents. . .

• already exist in the social world
• have not been produced for the purpose of social
  research
• have been preserved and are available for
  analysis
• are personal or official documents (Scott, 1990)
• are unobtrusive and non-reactive method

3
Criteria to Evaluate Documents (Scott, 1990)

• Authenticity
• is it genuine?
• Credibility
• is it free from error and distortion?
• Representativeness
• is the evidence typical of its kind?
• Meaning
• is it clear and comprehensible?

4
Unsolicited Personal Documents

• diaries, letters, and autobiographies
• e.g., letters by and about a child with epilepsy
  in the early 1800s (Dickinson, 1993)
• e.g., suicide notes (Jacobs, 1967)
• prevalence of data is related to socio-historical
  context
• blurred distinction between autobiographies and
  biographies
• e.g., Disney case

5

• Authenticity
• is the purported author the real author?
• Credibility
• how are the writers feelings represented?
• Representativeness
• selective survival of documents
• Meaning
• what is left unsaid?

6
Visual Objects

• Photographs
• as illustrations, data, or prompts
• Scott (1990) 3 types of home photograph
• idealization
• natural portrayal
• demystification

7

• What do image-makers seek to represent?
• e.g., selected/discarded photographs (Sutton,
  1992)
• Different interpretations of images

8
Official/Government Documents

• From the state
• budgets, white papers, reports of public
  inquiries, scientists testimonies, consultation
  papers, Hansard, etc.
• From private sources
• company annual reports, press releases, web
  sites, meeting minutes, memos, etc.
• Evaluate how documents reflect interests of
  authors

9
Mass Media Outputs

• Newspapers, magazines, radio, TV, film,
  advertising, music
• e.g., newspaper representations of squeegee kids
  in Toronto (Parnaby,2003)
• Contradictions and different styles of reporting
  reflects values of editors/ journalists

10
Virtual Outputs

• Official documents published on Internet
• Personal websites
• Chat rooms, blogs, listservs, and newsgroups
• Private email communication
• Text messaging

11

• Authenticity
• problem of identity deception
• Meaning
• need insider knowledge to understand texts

12
The World as Text

• Reading cultural texts
• written documents, visual images, artifacts,
  landscapes, performances, etc.
• Readers/audiences
• active or passive?
• audiences may interpret different meanings to
  those intended by author
• social researchers analysis adds another layer
  of interpretation

13
What is Secondary Analysis?

• Analysis of data that were collected by others
  for different purposes
• other researchers
• institutions of the state/business organizations
• Primary data
• collected by oneself
• Blurred boundary between primary and secondary
  data

14
Advantages of Secondary Analysis

• Practical use for students
• saves cost and time because the data is already
  collected
• High quality data
• rigorous sampling
• large sample size
• experienced researchers

15

• Opportunity for longitudinal analysis
• previous waves of a survey
• track social change over time
• Subgroup analysis
• study specialized subcategories of sample
• e.g., Myles and Hou (2004) study of racial
  minority immigrant settlement patterns

16

• Opportunity for cross-cultural analysis
• fewer limits of time, cost, language
• More time for data analysis
• Reanalysis may offer new insights
• focus on one variable or subgroup
• new theories can be applied

17

• Wider obligations of social researcher
• participants have given a lot to researcher
• ethical duty to maximize the use of this data
• minimizes intrusion into peoples lives
• reduces the chance of data being destroyed or
  forgotten about

18
Disadvantages of Secondary Data

• Lack of familiarity with data
• how was it collected?
• how was it coded and managed?
• Complexity of data
• volume of data
• hierarchical data sets

19

• No control over data quality
• validity and reliability
• Absence of key variables
• there may be no data on a variable of interest to
  you
• inability to apply new theories

20
Official Statistics

• Collected by agencies of the state in the course
  of their business
• police data on the crime rate
• Canadian census
• Statistics Canada economic data

21
Advantages of Official Statistics

• Data often based on populations, not samples
• Reduced time and cost
• Reduced problems of reactivity
• Cross-sectional and longitudinal analysis
• Cross-cultural analysis

22
Disadvantages of Official Statistics

• Only reveal tip of the iceberg
• unrecorded events, such as crime rates and
  suicide rates
• Reveal more about collection procedures than
  about underlying phenomenon
• dubious measurement validity

23
The Social Construction of Crime Statistics

• Crime rate offences recorded by police
• Contingent on social processes of decision-making
• Stages of (de)selection
• offences not witnessed, reported, taken
  seriously, and recorded
• police discretion, priorities, and
  decision-making

24
Reliability and Validity of Official Statistics

• Reliability
• definitions, categories, and allocated resources
  change over time
• reflects priorities of agencies/organizations
• e.g., police crackdowns
• Validity
• fiddling the crime figures
• ecological fallacy

25
Condemning Official Statistics

• Influential critiques in 1960s
• disputed claims to objectivity
• social processes involved in constructing
  statistics
• became topic of research in itself
• neglect of official statistics as source of data

26
Resurrecting Official Statistics (Bulmer, 1980)

• Critiques had focused on problems unique to crime
  statistics
• No need to generalize these to all official
  statistics
• Definitions employed by official agencies and
  social researchers are not very different
• All social measurement is prone to error

27
Official Statistics as an Unobtrusive Method

• Webb et al. (1966)
• problem of reactive effects
• search for unobtrusive measures
• e.g., archive material
• Now prefer the term unobtrusive method (Lee,
  2000)
• Researcher avoids direct involvement in the
  events being studied