Research and policymaking: utilization of research evidence

1
Research and policy-makingutilization of
research evidence

• Marleen Brans
• (marleen.brans_at_soc.kuleuven.ac.be)
• Joint Seminar 18-20 March 2008
• Leuven

2
contents

• 1. Policy evidence from knowledge, information,
  research
• 2. Utilization of knowledge
• 3. Case studies
• 4. Challenges for better knowledge utilization

3
1. Policy evidence from knowledge, information,
research

• Research produces a type of evidence in a policy
  inquiry research competes with other types of
  information such as data and analysis
• Distinction between content of information and
  form of information
• Dissemination is crucial

4
Complexity of Policy KnowledgeSource Webber,
1992390
5
Distribution of Policy KnowledgeSource Webber,
1992407
6
Policy Inquiry (Lindquist 1990)
Information-generation activies
Publication acitivies
memos briefings reports articles papers mono
grafies books
Data Analysis

research Convocation
activities
workshops seminars symposia conferences brief
ings speeches
7
2. Utilization of knowledge

• Assumptions
• Dynamic perspective KU is a process
• Use or non-use in different phases of
  policy-cycle
• Use is not the prerogative of policy-makers
• Use by third actors
• Questions
• what is research evidence used for? Functions?
• who by and what for? Users and interactions
• how can we measure utilization?
• how can we explain utilization?

8
Social and policy relevant functions of knowledge
(Liberatore 2000)
Empowerment
Delaying decisions
Early warning
Clarification of conflicts and interests
Legitimation
Goal setting
Problem definition
Policy Evaluation
Formulating policy options
Implementation assessment
9
Theoretical models of KU

• Bipolar models
• Goals of two communities
• Dominance of communities
• Direct or indirect transfer
• Weiss
• Instrumental
• Legitimating
• Conceptual (knowledge creep)
• Hoppe (Wittrock)
• Use is explained by matches and mismatches
• language, time horizon, faulty dissemination
• Absence of research brokers, matchmakers

10
Different models boundary work policy and science
(Hoppe, Wittrock)
Knowledge creep
Divergent logics
Loyalty is our profession
Advocay coalition we provide ammunition.
Bureau- cracy
Enlightenment
Discourse coalitions we share a vision
Advocacy models
Political primacy
Science primacy
Pure learning we do it evidence-based
We muddle through with whatever useful info we
have
Policy learning
Buy our scienfic product
Politics spoils good policies
(Social) Engineering
technocracy
Convergent logics
11
Beyond two communities third community

• Three communities
• Community of researchers
• Community of policy-makers
• Third community

12
Measuring UtilizationClimbing the Ladder of
Utilization (1)Adapted from Knott Wildavsky,
1980, Hoppe 2006

• Step 1. Transmission.
• inputs transmitted to policymakers and relevant
  social stakeholders/citizens.
• Step 2. Cognition.
• inputs read, seen, and understood.
• Step 3. Reference.
• inputs cited or referred to in public debate and
  policy deliberations.

13
Measuring UtilizationClimbing the Ladder of
Utilization (2)Adapted from Knott Wildavsky,
1980, Hoppe 2006

• Step 4. Effort.
• Policymakers have made an effort to adopt
  recommendations.
• Step 5. Adoption.
• results have somehow influenced authoritative
  policy choice and decision.
• Step 6. Application.
• results gave rise to implementation and/or
  elaboration in policy practice.

14
Explaining Utilization (Hoppe 2006)

• Science Push (or Technological) Model.
• Advancement of knowledge, product types, sources
  of funding, publication assets.
• Demand Pull (or Economic) Model.
• Focus on users needs, receptivity of users to
  research.
• Institutional or Dissemination Model.
• Adaptation of products, amount of dissemination
  efforts.
• Social Interaction Model.
• Sustained and intense interaction of producers
  and users, linkage mechanisms.
• Transaction costs or barriers to entry in
  attempts to climb the ladder of utilization.
• Public policy barriers, absolute cost advantages,
  economies of scale, product customization.

15
Some Findings on Climbing the Ladder of
UtilizationSource Landry, Amara Lamri, 1999
Hoppe 2006

• Variables in technological model are not
  supported at all.
• Focus on users needs only fails to explain a
  climb in the ladder of utilization.
• Institutional dissemination efforts (adaptation
  of products, dissemination strategy) successfully
  predicts only transmission but not climbing to
  higher echelons.
• Linkage mechanisms successfully predict
  transmission and cognition but fail to predict
  climbing higher.
• Users context (users consider research
  pertinent, research coincides with users needs,
  users attitude gives credibility to research,
  results reach users in time) is clearly the best
  predictor for climbing to higher echelons.
• Overall, theory successfully explains the step to
  transmission, but rather indeterminate on
  climbing the higher, more relevant echelons.

16
Simplified model interactions science,
policy-making, society (Liberatore 2000)
Conditions for matching supply and
demand windows of attention, perceptions of
use dialogue trust and credibility
Supply of information universities,
administrations, consultants, think tanks, Data
analysis and research Publications and
convocations
Demand for information From government, ngos,
parliament, private sector for problem
defintion- and dolution, legitimation,
empowerment
Arguments and advocates Defended and ignored
arguments in societal and policy debate
Advocates Coaliti ons and conflicts between
advocates
17
3. Case studies

• UNESCO DWTC
• Research in 22 countries
• ULg, ULB, KUL (Tinne Van der Straeten, Dirk
  Jacobs, Marleen Brans, Marc Swyngedouw)
• Two case studies
• Research immigrants - education
• Research labour market participation of
  immigrants

18
Case study 1 Research and policyimmigrant
children and education

• Policy formulation
• Onderwijsvoorrangsbeleid and target group
  setting
• Big impact of research early 90 adoption and
  application
• Policy reversal against research findings 2000
• Policy evaluation
• Many evaluation studies adoption and
  application
• Often behind closed doors with many uses
• Implementation support
• Lots of instrumental use
• Didactical instruments and formation of teachers
  (bicultural education)
• Researc pillars as special interfaces
• Facilitates application
• Facilitates interaction
• Non linear

19
Case 1 Conclusions

• Lots of research
• Fruitful links between research and policy
• Signalling problems
• Policy support
• Praktical advice
• Evaluation
• Interaction mainly between two communities
• Instrumental use
• Legitimation
• Special interfaces
• Research broker, hybrid researcher policy-maker
• Research pillar
• Facilitates trust, perception of usefulness
• Survival change of legislature is not guaranteed
• Closed policy inquiry
• Third community not involved, privileged access
  to evaluation limited to policy-makers

20
Case 2 Research and policyImmigrants and labour
market participation

• ILO research
• Launch with research brokerage
• Third party Centre for Equality of Opportunities
  and Opposition to Racism
• Advocacy coalition
• Dissemination
• Convocation activities third party all
  policy-makers implicated, incl. employers
• Press attention
• Use of research
• Problem definition, framing problem demand side
  versus supply side
• Positive acition
• Booster for other research with same evidence

21
Case 2 Research and policy conclusions

• Nature of research
• Methodological quality
• Concise presentation of results
• Clear directions for action
• Nature of dissemination
• Convocation activities third actor
• Press
• Use of research
• Agendasetting, direct impact on identification,
  size and cause of problem
• Indirect impact on policy formation by providing
• Booster for creating access for previous research
• Role of third actor
• Advocate, broker, disseminator
• Importance of policy advocate when focusssing
  events absent

22
4. Challenges for better knowledge utilization

• Research and expertise are no supermarket
• Policy decisions are not singular but the outcome
  of a process
• Dialogue research and policy-making requires
  investments in longlasting interactions

23
Challenges for policy-makers dealing with
scientific evidence

• Openness to multiple perspectives
• transparency
• independence
• Conscious dealing with uncertainty (risk averse
  policy-makers!!)
• Structurally invest in good relationship and
  support for academic research

24
Challenges for researchers

• A scientific/expert institute keen on significant
  utilization, will have to invest heavily in
  skills, expertise, and know-how, and bear
  substantial costs for customization tailored to
  one or a few users.
• Linkage mechanisms for knowledge mandating/
  mobilization important for utilization capacity
  to take advantage of window of opportunity.
• Produce validated knowledge of proven quality
• Invest in dissemination of different published
  products and multiple convocation activities
• Involve third community actors
• Policy-analysis is not for the innocent!!