Strategic Planning Session

1
Strategic Planning Session

• David Rudder, Ph.D.
• Rudder Consultants, LLC.
• May 17, 2006

2
EVALUATING THE EFFECTIVENESS OF NONPROFIT
ORGANIZATIONS

• Chapter 14
• Vic Murray

3
What is Evaluation and why is it important?

• Evaluation is the process of gathering
  information on the results of past activities for
  the purpose of making decision about them.
• Organizational effectiveness evaluation (OEE)
  occurs when this process is applied to assessing
  the state of the organization as a whole.
  Typically, this refers to how well it is
  achieving its stated mission and involves looking
  at goal attainment as well as how efficiently it
  has performed.

4
What is Evaluation and why is it important?

• The focus of most evaluations is on the
  organization, but much still boils down to
  looking at the results of the actions of many
  units and individuals.
• Increasing accountability, refers to the belief
  that nonprofits, and the people who run them,
  should be more accountable to those they are
  created to serve and those who provide the money
  to operate them. This is the primary cause of
  the growing interest in organizational
  evaluation.

5
Politics

• Politics is inevitable in evaluation because
  there is so much room for subjectivity that
  differences can easily arise between the parties
  involves. Key parties are
• Evaluators
• Evaluates
• Other interested stakeholders

6
Stages of the Evaluation Process

• The Design Stage What is the purpose of
  organizational evaluation? What exactly will be
  measured inputs, activities or processes,
  outputs or outcomes?
• Implementation Stage How will the information
  be gathered?
• Interpretation Stage What will be considered a
  success or a failure? If an evaluation
  measure reveals problems, there is a question of
  drawing conclusions about why these occurred in
  order to make decisions about the future.
• Application Stage How will the information be
  used in subsequent decision making?

7
Problems with Evaluations

• Works best when the measurements can be compared
  to clearly stated goals, objectives, or standards
  that a given organization is trying to achieve.
• Often when goals are vague and ambiguous, the
  development of valid measures of these kinds of
  things is technically challenging, costly, and
  subjective.
• A design that is measuring one level of an
  organization but generalizing to another.

8
Problems with Evaluations

• Even when everyone focuses on outcomes and agrees
  on what should be evaluated, there are inevitable
  difficulties over the extent to which outcome
  measures really capture the goals they are
  intended to measure.
• Most evaluation systems are unable to provide
  conclusive analyses of why the results came out
  as they did. Most outcomes have multiple causes,
  and opinions can easily differ over which are the
  most important.

9
The United Way Approach

• The outcome information is intended to be used by
  UW to help member agencies improve program
  performance, to identify and achieve UW
  priorities (funding allocation criteria), and to
  broaden the base of financial and volunteer
  support.
• Implementation of the outcome measurement system
  is divided into six stages
• Building agency commitment and clarifying
  expectations
• Building agency capacity to measure outcomes
• Identifying outcomes, indicators, and data
  collection methods
• Collecting and analyzing outcome data
• Improving the outcome measurement system
• Using and communicating outcome information

10
United Way Continued

• Agencies are not expected to establish targets
  until they have at least one year of baseline
  data.
• The system discourages the use of benchmark-based
  relative standards or those that involve
  comparison with similar programs that are
  considered exemplary until accurate outcome data
  are available.
• It is generally understood that in the first few
  years of an outcome measurement system, the data
  often say more about what is wrong with the
  evaluation system than about what is taking place
  in the programs.

11
The Balanced Scorecard

• A multiattribute system for conceptualizing and
  measuring performance. It assumes that the
  primary goal of a business is long run profit
  maximization. It argues that this will be
  achieved through a balanced scorecard of
  performance attributes grouped around four
  perspectives
• 1) The funders perspective, measuring various
  financial performance indicators or primary
  interest to shareholders
• 2) The client perspective, comprising measures of
  client satisfaction
• 3) The internal business perspective, which means
  internal efficiency and quality
• 4) The innovation and learning perspective, which
  attempts to measure the organizations ability to
  adapt to changes required by a changing
  environment
• In the case of nonprofit organizations, their
  mission statement becomes the endpoint to be
  reached through these perspectives. The process
  starts with defining what that is and identifying
  outcome indicators that will reveal the extent to
  which it is being achieved.

12
CCAF/FCVI Framework

• Puts forward 12 attributes of effectiveness,
  suggesting that organizations can be audited in
  terms of how well they manifest these attributes.
  They are

• Management direction
• Relevance
• Appropriateness
• Achievement of intended results
• Acceptance
• Secondary Impacts
• Costs and Productivity
• Responsiveness
• Financial Results
• Working environment
• Protection of Assets
• Monitoring and Reporting

13
Conclusion

• There is no tried and tested evaluation system
  that can be applied by most nonprofit
  organizations to provide a valid picture of how
  well the organization is performing.
• We should focus on trying to improve the dialogue
  around the evaluation process.
• If a prior relationship of trust does not exist
  before evaluation begins, it must consciously be
  worked on as the process is developed,
  particularly those who are to be evaluated.

14
Conclusion Continued

• All parties involved (evaluator, evaluates, and
  stakeholders) should have a voice in deciding
  the following six questions
• What is the purpose of the evaluation?
• What should be measured?
• What evaluation methods should be used?
• What standards or criteria should be applied to
  the analysis of the information obtained?
• How should the data be interpreted?
• How will the evaluation be used?