Title: Bond Presentation
1Assessing Institutional Effectiveness The
Mission Engagement Index (MEI) as a Measure of
Progress on Mission Goals
Ellen Boylan, Ph.D. ? Marywood University NEAIR
35th Annual Conference ? Providence, Rhode Island
? Nov. 1 4 , 2008
2Scope of Research
- Purpose
- Develop a measure (inventory) of student
perception of institutional mission. - Test instrument reliability.
- Uncover constructs (factors) and observe them
longitudinally. - Exploratory multiple regression Develop a
measure (index) that allows institutions to
compare their predicted with actual outcomes on
mission constructs.
- Research Questions
- 1. Is the Mission Perception Inventory
- (MPI) a valid, reliable measure of
- student perception of institutional
- mission?
- 2. What are the factors in the
- Mission Perception Inventory?
- 3. Do factors recur with equal reliability in
repeat administrations of the MPI? - 4. Does the Mission Engagement Index (MEI)
describe causal relationships among variables
that affect mission perception?
3 Study Design
- Select the National Survey of Student
- Engagement (NSSE) as the vehicle
- for delivery of the research
- questions.
- Assemble a NSSE consortium to
- jointly engage in research to explore
- student perception of mission.
- Develop consortium questions that
- are then given as an attachment
- to NSSE analyze results via reliability,
factor analysis, and correlation. -
- Repeat annually. Once a sufficient
- volume of institutional data is
- gathered, conduct regression analysis to
develop the index (MEI).
4Developing the question items
- Conduct a qualitative analysis of mission
statements. - Distill common constructs.
- Draft question items based on those constructs.
- Vet with participating institutions, experts, and
prospective subjects.
The University roots itself in the principle of
justice and a belief that education empowers
peopleoffers students a welcoming and supportive
community that encourages men and women of all
backgrounds to shape their lives as leaders in
service to others. Proud of its liberal arts
tradition and host of professional disciplines,
challenging students to broaden their
understanding of global issues and to make
decisions based on spiritual, ethical, and
religious values
Concepts common to consortium colleges mission
statements, like this one, were drawn out,
reduced, and formulated into 20 question items.
5Concepts became question items...
6A peek at the questionnaire
See consortia questions by year
http//nsse.iub.edu
7Administration
The NSSE survey with attached Mission
Perception Inventory (MPI) questions has been
administered 145 times to more than 30,000
first-year and senior students at 83 colleges
across the United States every year since
2004.
8Is the Mission Perception Inventory (MPI)
valid and reliable?
- Reliability analysis of the mission questions
- (20 items) is performed.
- Low-loading
- items kicked
- out.
- A Mission Perception Inventory (MPI) is
- produced (17 or 18 items).
9What are the factors in the Mission Perception
Inventory?
- Factor analysis of MPI is conducted each year.
- By year, three or four subscales are produced
- Sense of mission.
- Respect for diversity.
- Individual values.
- Religious practice.
10Derivation of Mission Perception Inventory (MPI)
and subscales
11Sense of Mission Scale (8 items ? .867)
- The mission of this institution is widely
understood by students. - Social and personal development is an important
part of the mission. - Ethical and spiritual development of students is
important. - This institution offers opportunities for
volunteering and community service. - This institution offers opportunities for
developing leadership skills. - There are opportunities for students to
strengthen their religious commitment. - This institutions religious heritage is evident.
- Professors here discuss the ethical implications
of what is being studied.
12Respect for Diversity Scale (5 items ? .854)
- The faculty, staff, and students here
- respect different religions
- respect different races and
- cultures
-
- Students free to express individual
spirituality. - Different sexual orientations are accepted.
- The environment encourages appreciation of
- diversity.
13Count of institutions and respondentsby year
respondents
14Do the factors recur in repeated administrations
of the revised Mission Perception Inventory (MPI)
15Are the factors equally reliable over time?
16Teagle Foundation Research Continues
2008
- Develop the Mission Engagement Index.
- Obtain NSSE 2008 consortia data
- Test using reliability analysis to
- produce factors
- Distribute MPI reports
- Compare consortia results
- Conduct regression analysis to create the new
index - Produce Mission Engagement Index (MEI) Reports
for consortia institutions.
17Does the Mission Engagement Index describe causal
relationships among variables that affect mission
perception?
- Dependent variable (Institution score)
- Mission Perception Inventory
- Sense of Mission scale
- Respect for Diversity scale
- Independent variables (need 15 cases per)
- Selectivity 15 institutions
- Enrollment 30 institutions
- Urbanicity 45 institutions
- Resident 60 institutions
- Female 75 institutions
- Another? 90 institutions
a recommended ratio of subjects to IVs of at
least 15 to 1 will provide a reliable regression
equation (Stevens, 1992).
18Selecting IVs for regression analysis
19Participating institutions by region
2008 2 consortia, 54 institutions
20Urbanicity of participating institutions 2008
21Participants by gender and class rank by year
22Considering elegance
- Consistency of factors is affirmed.
- Institution sizes reasonably varied.
- Data sufficient to conduct analysis.
- Variables, initially selected
- on an a priori basis, tested.
- Variability investigated.
- Useful.
ßeta juice
23Mock Mission Engagement Index Report
- The MEI will show an institutions actual versus
predicted scores on mission constructs. - Progress on mission effectiveness can be assessed
by comparing MEI outcomes to institutional goals.
24Assessing Institutional Effectiveness The
Mission Engagement Index (MEI) as a Measure of
Progress on Mission Goals Ellen Boylan,
Ph.D. Marywood University Scranton PA
eboylan_at_marywood.edu available http//www.maryw
ood.edu/instresearch/activity.stm
Supported by a grant from
25(No Transcript)