Cultural Competence and Evaluation: An Overview

1
Cultural Competence and Evaluation An Overview

• Ranjana Damle, Ph.D.
• Albuquerque Public Schools
• New Mexico Evaluators
• 2009 Summer Conference

2
Outline of the Presentation

• Purposes of evaluation
• Brief discussion of evaluation theory
• Types of evaluation new discourse, trends
  efforts to live up to evaluations promise
• Context of evaluation
• Social construction of reality common
  assumptions and blind spots
• Issues of power and social justice
• Culture, cultural competence, culturally
  competent evaluation
• A case study that ties in many of these points

3
Evaluation Purposes

• Accountability and program improvement
• Policy development and informed decisions
• Systematic inquiry to generate knowledge

4
Three Main Streams in Evaluation Theory

• Methods creating knowledge using rigorous
  research techniques
• Valuing evaluators discern and place a value on
  their findings favors decision makers
• Use evaluations intended to inform decision
  making various stakeholders perspective
  incorporated to increase evaluation use

Evaluation Roots Tracing Theorists Views and
Influences. Marvin C. Alkin (ed.). Sage
Publications, 2004.
5
Evolution of Evaluation

• Participatory a continuum, minimal to full
  participation of the stakeholders a buy-in in
  the organization for the evaluation - Bradley
  Cousins
• Collaborative to increase the chance of
  evaluation use
• Utilization-Focused Michael Patton
• Responsive stakeholder perspective needs to be
  represented In the evaluation range of
  perspectives extending from the decision-makers
  to a variety of stakeholders. However, evaluator
  is the final authority - Robert Stake

6
Use for the Intended Users

• Empowerment stakeholder controlled
  self-determination as a goal of the approach -
  David Fetterman
• Advocacy democratic Jennifer Greene
  (Evaluators inevitably take sides.)
• Social Justice Evaluation is never value
  neutral and it should tilt in the direction of
  the poor and the powerless - Ernie House
• Cultural Competence in Evaluation Karen
  Kirkhart, Rodney Hopson, Saumitra SenGupta, Melva
  Thompson-Robinson, many others
• In Search of Cultural Competence in Evaluation.
  Melva Thompson_Robinson, Rodney Hopson, Saumitra
  Sengupta (ed.s). New Directions for Evaluation,
  102, Summer 2004.

7
Context of Evaluation

• Political, economic, social, cultural
• Diversity - age, sexual orientation, differential
  abilities, race and ethnicity, gender, immigrant
  status, language, etc.
• Less discussed differences - access to power,
  education, healthcare socioeconomic status,
  class living situation

8
Social Theory

• Social construction of reality
• There is a social dimension to human behavior
• We create, attach meaning to symbols
• Reality is not absolute but as individuals or
  groups define it
• Cultural relativity examples, adult son living
  at home, parental participation in schools

9
Some Common Observations, Experiences across
Societies

• We relate better to, or feel comfortable with,
  the people that share our experiences
• In-groups and out-groups embracing the norms
  and beliefs of the group we belong to (in-group),
  while being critical of out-groups
• Middle class bias in education in other social
  institutions

10
Characteristics Largely Invisible, Hidden yet
Impactful

• Privilege taken for granted by those who have
  it
• Blindness towards the underprivileged e.g.
  invisibility of the poor and powerless caste
  societies
• Dominant norms - treatment of else or others
  as something deficit

11
Evaluation and Issues of Power, Equity, Social
Justice

• Unequal power relations between different
  stakeholders
• Whose interests are represented in the
  evaluation?
• Objectivity and neutrality versus inherent
  biases, e.g. favoring the management

12
Defining Culture

• Food, music, clothes, artifacts
• Customs, norms, beliefs, myths, social
  institutions, language, other symbols, traits
• Values - While there are some universal values,
  there is a variation in values across cultural
  groups

13
Cultural Competence

• Being effective in cross-cultural situations and
  able to relate to many cultures
• As an evaluator, being able to recognize the
  culture-specific strengths and uniqueness
• A distinction to be noted
• Evaluating cultural competence
• Cultural competence in evaluation

14
Cultural Competence in Programs Evaluation

• Cultural competence in program development
• Respect, appreciation, flexibility, recognition
  of values and needs although different from ours
  programs likely to be more effective if
  culturally relevant, e.g. population control
  programs
• Cultural competence in evaluation
• Culturally responsive evaluations
• Ethics, equity, awareness of the power imbalance,
  identifying program elements that seem to make
  programs work for non-dominant cultures

15
Towards Successful Evaluations

• Cultural competence
• Is essential in study design, validity of
  measurements, and data collection
• Leads to pertinent questions what strategies
  and components in a successful program served the
  needs of the (minority) clients and why so
• Leads to a greater buy-in, hence evaluation
  findings are more effectively applied towards
  program improvement

16
A Case of Lofty Goals, Unilateral Program
Design, Wasted Effort

• The Myth of Population Control by Mahmood Mamdani
  (1973)
• Program was designed to educate the villagers
  about family planning but without taking into
  account their experience, perceptions, values,
  and realities of their lifes challenges
• Cultural competence Discerning rationality
  from the villagers point of view
• Validity of measurement closely tied to cultural
  competence what was measured and how
• Outcome and lessons

17
Contact Information

• Ranjana Damle
• Research and Evaluation
• 872-6801
• damle_at_aps.edu