Issues in the Creation

1
Issues in the Creation Conduct of
Community-Based Indicators
Jim Frankish Senior Scholar, Michael Smith
Foundation Associate Director, IHPR Associate
Professor, HCEP Grad Studies
2
Current Research Projects

• Research Training Program in Community
  Partnership Research
• BC Homelessness Health Research Network Project
• Development Evaluation of Homeless Information
  Systems
• Service Access Utilization for Homeless Persons
  with Mental Illness
• Social Construction of Homelessness
• Measurement of Health Literacy
• Health Promotion in primary healthcare Project

3
Current Research Projects

• Report Card on Impacts of 2010 Games on Health
  Quality of Life
• Measuring Community Capacity
• Measures of Community Health
• Evaluation of "Cooking Fun for Families"
• Health Regions Non-Medical Determinants of
  Health
• Literacy and Health
• Health Literacy in Schools
• Mid-Life Health Interventions for Healthy Aging

4
Current Student Research Projects

• Primary healthcare reform and disadvantaged
  populations
• Health education in women with Aids in Latin
  America. (SFU)
• Mental health promotion and early psychosis
• Food security in persons with disabilities
• Criteria for health resource allocations to
  "special interest" groups
• Pediatric HIV education
• Adolescent depression
• IV drug use and street kids
• Living well with chronic illness
• Poverty and nutrition in inner-city schools
• Child injury prevention in low-income parents
• Social capital in disadvantaged resource-based
  communities
• Poverty and nutrition - cultural variations

5
Introduction

• Definitions of Key Terms
• Rationale for Measuring Community-Level
  Indicators (CLI)
• Issues related to Measurement at a Community
  Level
• Multi-Step Approach to Conceptualization
  Measurement of CLIs
• Potential Implications

6
Definitions

• Population Health epidemiological social
  condition of a community (defined by geography or
  common interests) that minimizes morbidity
  mortality, ensures equitable opportunities,
  promotes protects health, achieves optimal
  quality of life. (Frankish, Veenstra Moulton,
  1999)or "the health of a population as measured
  by health status indicators and as influenced by
  social, economic, and physical environments,
  personal health practices, individual health
  capacity and coping skills, human biology, early
  childhood development and health services. (FPT
  Advisory Committee on Population Health, 1997).
• Evaluation is the comparison of objects of
  interest against standards of acceptability
  (Green, 1974)

7
Definition of Community

• Spatial communities have geographic boundaries
  that may be of diverse forms (e.g., perceptual,
  political, bureaucratic structural, natural).
  Spatial communities may also overlap, and
  individuals may live in and have an affinity for
  multiple spatial communities at a given time
  (Hancock et al. 1999).
• Non-spatial communities are communities of
  affinity." They may transcend, overlap with
  and/or be contained within spatial boundaries of
  one form or another. Examples of non-spatial
  communities include ethnic or racial groups,
  socioeconomic groups, communities of shared
  interests (e.g., health problems, recreational or
  social concerns). Non-spatial communities may be
  highly localized or increasingly they may be
  spread widely, as in emerging "virtual
  communities" that are technologically-based on
  the Internet.

8
Rationale for Community Research

• Many health professionals and researchers are
  increasingly engaged in research (and training)
  activities with diverse community partners.
• This work present fundamental challenges
  regarding the creation and conduct of CBR. CBR
  often present unique and interesting
  philosophical, ethical, practical, design and
  measurement issues.
• These issues are further complicated in work with
  vulnerable or multicultural communities.
• There is an urgent need for discourse, education
  on issues in CBR and opportunities to share
  experiences, lessons and perspectives on relevant
  issues.

9
Rationale for Measuring Community-Level
Indicators

• Growing recognition of the importance of
  community as a determinant of health.
• Importance of community functions and processes
  in health and quality-of-life
• Value of creating policies and programs in
  support of social cohesion and social capital
• Building on key elements of community health in
  government documents and surveys

10
Issues in Measurement of Community-Level
Initiatives

• Definitional and Conceptual Issues
• Values
• Time Frames
• Responsibility for Decision-Making
• Making the Measurement of CLIs Popular
• Structural Constraints
• Accountability
• Relations Between Health Sector Participants and
  Other Stakeholders
• Resources to Facilitate and Strengthen the
  Measurement of Community Health

11
Conceptualizing Community Health

• Concepts the 'idea' of community health
• Constructs the operational definition or
  characteristics of community health
• Theories/Models Hypothesized relations between
  important constructs
• Measures/Indicators Data on important constructs
• Strategies Means of acting to change important
  relations between constructs in the model or
  theory
• Standards definitions of success for each aspect
  of community health

12
Theories of Change Community Indicators

• Systematic and cumulative study of the links
  between activities, outcomes, and contexts of the
  initiative Connell and Kubisch 1998 . The theory
  of change is an approach, and not an evaluation
  method that stands on its own. The advantage to
  using the theory of change approach is that it
  makes explicit people's ideas underlying the
  initiative, i.e., what the expected outcomes are,
  and what needs to be done to achieve them.

13
Theory of Change Questions

• What long-term outcomes of community health does
  the initiative seek to accomplish?
• What interim outcomes of community health are
  required to produce those longer-term outcome and
  what activities should be initiated to achieve
  the interim outcomes?
• What contextual supports are required to achieve
  the interim outcomes?
• What resources are required to implement the
  activities and maintain the contextual supports
  required for the activities to be effective?

14
THEORIES OF CHANGE FOR COMMUNITY CAPACITY
(RESEARCH TEAM)
CONTEXTUAL SUPPORTS
RESOURCES
ACTIVITIES
INTERIM OUTCOMES
LONG TERM OUTCOMES
Policy support
Champion for community capacity
Increased community control
Get people interested in participating
Grassroots support
Participation of community members
Healthy communities
Human resources
Involve people in identifying perception of own
reality (problems/solutions)
Lateral and vertical perspective of issues
Increased health
Funding
Increased empowerment
Accessible information
Expertise in planning, evaluation, and needs
assessment
Actions that can be seen as immediate successes
Collaboration
Quality of partnerships (longer-term vs. one-time)
Expertise in building relationships, getting
people to work together
Raise awareness and share knowledge
Identify common understanding/values
Empathy (for other peoples life conditions)
Power mapping (identify players and relate to
policy realm)
Number of actions taken around issue(s) of concern
Community asset mapping (McKnight)
Funding provided to communities
ACTIVITIES (cont.)
Develop materials for organizations to use, e.g.,
self-help
Fun and food
Human resource involvement, e.g., volunteer hours
(including professionals)
Link resources across communities
Capitalize on opportunities of the moment
(flexibility in working with non-health
organizations
Longer-term (monetary) support
15
THEORIES OF CHANGE FOR COMMUNITY CAPACITY (HEALTH
CANADA)
CONTEXTUAL SUPPORTS
RESOURCES (that Health Canada provides)
ACTIVITIES
INTERIM OUTCOMES
LONG TERM OUTCOMES
Priority focus (population health approach vs.
disease prevention)
Organizational development
Increase community awareness
Offer longer term funding to projects
Increase skills, e.g., bureaucratic confidence
Staff
Develop good relationships with community organiza
tions
Role of popular media in discourse about
population health
Community mobilization, e.g., community initiates
activities
Decrease health inequities, e.g., morbidity,
mortality
Evaluation
Provide linkages with other players,
e.g., researchers, government
Use of population health approach
Sense of community control
Development of knowledge and evidence on
population health
Partnerships
Financial management
Organizational development
Sustainable communities (development)
Create projects that are strongly grounded in the
needs of the community
Political context (can be asset or liability)
Access to resources
Spin-off effects, e.g., new community projects
Share information
Meaningful participation, e.g., volunteer
involvement in governance
Provide a model for access to decision making
Increase social support
16
Conceptualizing Criteria for Measurement of CLIs

• Criteria are "principles, standards or tests by
  which something (i.e., policy or program) is
  judged" or as a "measure of value."
• Definitions of criteria often confound two
  important elements the 'quality' of a program or
  policy, and its 'outcomes'.
• Our approach is based on Green and Kreuter's
  (1999) definition of evaluation as "the
  comparison of objects of interests against
  standards of acceptability.

17

• Objects of interests are factors that underpin
  the goals, objectives and targets for
  community-level initiatives. They represent
  aspects of a program or policy that should be
  monitored to ascertain whether it has achieved
  its desired and intended outcomes.
• Standards of acceptability represent the
  'definitions of success' associated with specific
  indicators of the process, impact and outcomes of
  a given intervention.

18
A Suggested Three-Stage Solution to Development
of Explicit Standards

• Explicit Identification of Objects of Interest
• Articulation Collection of Measures
  (Indicators)
• Explicit Identification of Standards of
  Acceptability

19
Step 1

• Articulation of
• Objects of Interest

20
A Suggested Taxonomy of Objects of Interest

• Values-based or philosophical foci
• Process-related foci refer to the nature and
  quality of day-to-day operations of
  community-level initiatives and the manner in
  which key stakeholders and participants work
  together
• Structural-foci characteristics are those that
  pertain to the personnel, resources, and
  organizational and administrative structures that
  exist in a given community-level intervention to
  enable and support the planning and
  implementation of policies or programs
• Strategy-related foci, refer to the strategies or
  methods used to effect change and their
  implementation
• Outcome-related foci -- impact (short-term,
  proximal effects) and outcomes (longer-term, more
  distal effects)

21
Examples of Potential Objects of Interest

• Sustainability (energy use, water consumption,
  renewable resource consumption, waste production
  reduction, local production, land use,
  ecosystem health, ecological footprint)
• Viability (air quality, water quality, production
  use of toxics, soil contamination, food chain
  contamination)
• Livability (housing , density land use,
  transportation, automobile dominance)
• Walkability (green space/open space, community
  safety security, smoke-free spaces, noise
  pollution)

22
Examples of Potential Objects of Interest

• Conviviality (family safety security, sense of
  neighbourhood/place, social support networks,
  charitable donations, public services,
  demographics)
• Equity (economic disparity, housing
  affordability, discrimination exclusion, access
  to power control)
• Prosperity (a diverse economy, local control,
  employment/unemployment, quality of employment,
  traditional economic activity indicators)

23
Conceptual Framework
DETERMINANTS Sustainability Energy use Water
consumption Renewable resource
consumption Waste production and
reduction Local production Land use
Ecosystem health Ecological footprint Viability
Air quality Water quality Toxics
production and use Soil contamination Food
chain contamination Livability Housing
quality Density and land use Community safety
and security Transportation/automobile
dominance Walkability Green/open space
Smoke-free space Noise pollution
Conviviality Family safety and security Sense
of neighbourhood/place Social support networks
Charitable donations Public services
Demographics Equity Economic disparity
Housing affordability Discrimination and
exclusion Access to power and
control Prosperity Diverse economy Local
control Employment/unemployment Quality of
employment Traditional economic activity
indicators PROCESSES Education Early
childhood development Education
attainment/school quality Adult literacy
Lifelong learning
Governance Voluntarism/associational life
Citizen action/civicness Human and civil
rights Voter turnout Perception of political
leaders and government services Healthy
public policy HEALTH STATUS Positive Health and
Quality of Life Well-being/self-reported
health Life satisfaction Happiness Mastery/Se
lf-esteem/Coherence Health-promoting
Behaviours Negative Health Stress/anxiety
Other morbidity/disability measures Health
utility index Mortality Overall mortality
rate Infant mortality rate Suicide rate
Life expectancy
Adapted from Hancock T, Labonte R and Edwards E
(1999). Indicators that count! Measuring
population health at the community level.
Canadian Journal of Public Health 90(Suppl
1)S22-S26.
24
Step 2

• Linking of Objects of Interest to
  Indicators/Measures

25
Connecting Objects of Interest and Indicators of
Community Health

• Conceptual projects are those that ask
  stakeholders their views and 'definitions' of a
  'healthy' community
• Needs assessment projects by definition, attempt
  to assess the perceived needs of community
  members
• Tool development, i.e., the creation of better
  means of measuring some aspect of community
  health
• Measurement of specific aspects of community
  health and the evaluation of a given initiative
• Intervention" and the implementation of specific
  strategies for effecting change.

26
Considering Quality of Indicators

• Scientifically valid/theoretically sound
• Representative of community interests
• Responsive to interventions (e.g., programs and
  policies)
• Relevant to stated goals and objectives
• They are relevant to the needs of potential users
• Accurate, accessible, available data.
• Understandable by potential users.
• Provide early warnings regarding phenomena of
  interest.
• Comparable to thresholds or targets.
• Comparable with other jurisdictions.
• Cost effective to collect and use.
• Unambiguous.
• Useful for public relations and community
  mobilization.
• Potential for public participation in their
  development.
• Applicable to diverse populations.

27
Considering Quality of Indicators

• Sensitive to change over a reasonably short
  period of time.
• A balance profile and do not over emphasize any
  one group or condition
• Relevant to policy and program planning
• Make sense to people.
• Measure an important health determinant or an
  important dimension of health.
• Measure things that people care about.
• Powerful, they carry social and political punch.
• Ability to trigger action or policy development
• Foster and support equity.
• Comparable over time.
• Provide a basis for public debate and action.
• Address issue of substantial health impact.
• Have multi-level trackability.

28
Step 3

• Linking of Indicators/Measures to Standards of
• Acceptability

29
Setting Standards for Healthful Environments
Arbitrary, Experiential, Community, Utility
Historical, Scientific, Normative
Propriety, Feasibility, Administrative
From Green Kreuter, 1991 Judd, Frankish
Moulton, 2001
30
Perception-Based Standardsfor Healthful
Environments

• Arbitrary standards are simply a declared or
  expected level of change. They are most often put
  forward by individuals/groups in authority.
• Experiential standards recognize the value
  utility of local, indigenous knowledge.
• Utility standards are intended to ensure that any
  evaluation will serve the information
  decision-making needs of community stakeholders,
  practitioners government (Judd et al., 2000).

31
Data-Based Standardsfor Healthful Environments

• Historical standards are, by definition, based on
  previous performance data.
• Normative standards are usually based on what
  other health promotion programs in similar
  primary care settings have achieved.
• Scientific standards are developed from outcomes
  achieved in controlled studies generally based
  on systematic reviews of available literature.
  Scientific standards may be empirically and/or
  theoretically based (Judd et al., 2000).

32
Administrative Standardsfor Healthful
Environments

• Propriety standards are intended to ensure that
  program are conducted legally, ethically with
  regard to the welfare of participants.
• Feasibility standards are intended to ensure that
  evaluation will be realistic, prudent frugal
  considers cost effectiveness, political viability
  practical procedures (Judd et al., 1999).
• "Model" standards (APHA, 1991 Judd et al., 1999)
  incorporate elements of each of the other types.

33
Implications of Adopting Explicit Criteria for
Measurement of Community-Level 'Health

• Adoption of explicit criteria for CLIs could lead
  to new approaches to funding of community
  initiatives
• Greater emphasis on explicit criteria for CLIs
  could lead to new approaches to treating illness
  and promoting health.
• Health professionals and service providers may
  need to develop new capacities and skills.
• Adoption of explicit criteria for CLIs may
  contribute to a new "culture" in the health
  sector and greater support for disease
  prevention, health promotion and population-level
  interventions.
• New forms of management for health services and
  community health programs and policies may emerge
  from adoption of explicit criteria for CLIs.

34
Implications of Adopting Explicit Criteria for
Measurement of Community-Level 'Health

• The health sector may take on new or refocused
  functions in order to address the targets and
  goals suggested by explicit criteria for CLIs.
• Adoption of explicit criteria for CLIs may lead
  to the creation of new goals for the health
  sector.
• New objects of interest for community health
  (e.g., foci for evaluation) are likely to result
  from adoption explicit criteria for CLIs.
• Adoption of explicit criteria for CLIs could lead
  to the creation of new partnerships and broader
  intersectoral collaboration around the
  determinants of health.
• Adoption of explicit criteria for CLIs could
  contribute to a demand for new resources. It may
  also help to identify existing resources that can
  be applied through innovative programs and
  policies.

35
Implications of Adopting Explicit Criteria for
Measurement of Community-Level 'Health

• Professionals/service providers may need to adopt
  new or different roles when working under a
  community health approach. These new roles may
  require new skills, training and
  capacity-building.
• New, additional stakeholders from diverse sectors
  of government/society may become involved in
  planning, implementation and evaluation of
  health-related services, programs and policies.
• A new definition of success and standards of
  acceptability for community health services may
  emerge from consideration of explicit criteria
  for CLIs.
• Creation of new partnerships and involvement of
  diverse stakeholders may contribute to the
  creation of new structures in the health sector.
• Examination of explicit criteria for CLIs may
  lead to new targets for 'health' services,
  programs and policies.

36
Chinese Boxes Community-Level Indicators

• Systems relate to one another they do not exist
  in isolation. We liken it to Chinese Boxes - a
  conjurers nest of boxes, each containing a
  succession of smaller ones. Within localized
  structures we envisage successive levels of
  organization, each of which encompasses the next
  and simpler level, all with intimate links
• Adapted from Susser, M. Susser, E. (1996).
  Choosing a future for epidemiology II. From
  black boxes to Chinese boxes and
  eco-epidemiology, AJPH, 86(5), 674-677.

37
Contact Information

• Dr. C. James Frankish, Senior Scholar, Michael
  Smith Foundation for Health Research, Associate
  Director, Institute of Health Promotion Research
• Associate Professor, Graduate Studies Health
  Care Epidemiology
• Room 308, Library Processing Centre, 2206 East
  Mall Vancouver BC V6T 1Z3
• 604-822-9205, 604-822-9210
• frankish_at_interchg.ubc.ca, www.ihpr.ubc.caResearc
  h is the art of seeing what everyone else has
  seen, and doing what no-one else has done.
  Anonymous