Getting the data off the shelf

1
Getting the data off the shelf

• Jimmy Davis, Ph.D. Vice Provost
• Kimberly Thornbury, Ph.D. Dean of Students
• Union University

2
Where We Were

• What Got Us Started in Assessment
• Institutional Strategic Planning
• Long Range Planning (began 1982)
• Five-Year Planning Institutional Research
  Office (began 1994)
• Regional Accreditation (SACS)
• Institutional Effectiveness
• ongoing, integrated, and institutional-wide
  planning and evaluation processes
• provides evidence of improvement based on
  analysis of assessment results

3
The deciding factor in assessing the
effectiveness of any institutional effectiveness
is evidence of the extent to which it achieves
its goals and objectives. The necessity of
seeking such evidence continually is inescapable
one of the primary hallmarks of faculty,
administration, and governing boards is the skill
with which they raise questions about
institutional effectiveness, seek answers, and
significantly improve procedures in light of
their findings. (Commission on Higher
Educations Standards for Accreditation, 1982, pp
17-18.)
4
Without assessment, student affairs is left only
to logic, intuition, moral imperatives, goodwill,
or seredipity in justifying its existence.
(Upcraft and Schuh, Assessment in Student
Affairs, pg 12.)
5
We in student affairs can no longer presume that
our jobs are important and effective simply
because we think they are or because weve always
done them or even because were doing the right
thing. There must be evidence gathered from a
comprehensive assessment program to answer these
questions. (Upcraft and Schuh, Assessment in
Student Affairs, pg. 16.)
6
Survey Instruments

• How to choose
• How to focus
• How to report

7
Why We Changed

• CAP FOCUS

Year Fall Spring
2000-2001 CIRP
2001-2002 SSI/IRP CSS
2002-2003 CIRP
2003-2004 SSI/IRP CSS
2004-2005 CIRP Faculty Survey
2005-2006 SSI/IRP CSS, STI
2006-2007 CIRP
2007-2008 SSI/IRP CSS
8
Astins I-E-O Model

• I refers to Inputs
• The personal qualities that students bring to the
  educational program
• E refers to Environment
• The students actual experience during college
• O refers to Outputs
• Talents that are developed in the educational
  program
• Astin, A. W. (1993). Assessment for excellence
  The philosophy and practice of assessment and
  evaluation in higher education. Phoenix The Oryx
  Press.

9
What Surveys does Union University Employ?

• Inputs
• Cooperative Institutional Research Program (CIRP)
  Freshmen Survey
• Environment and Outputs
• College Student Survey (CSS)
• Student Satisfaction Index (SSI)
• Adult Student Priorities Survey (ASPS)
• National Survey of Student Engagement (NSSE)
• HERI Faculty Survey

• Outputs
• Alumni Survey
• General Education
• College BASE
• Major Areas
• Major Field Tests

10
Group DiscussionWhat Survey Instruments Do You
Employ?
11
How we collected the data

• CIRP part of orientation schedule
• SSI (random group selected, come at 1000 or
  300 hour.)
• NSSE our first experience with online surveys
• CSS the draw of food
• Faculty survey January workshop
• Adult Noel Levitz Program directors
• Alumni

12
Group Discussion How do you collect your data?
(e.g. service days, in class distribution?)
13
How we distributed the data

• Most people wont read raw data. The importance
  of interpretation and presentation.
• Most people need trends, themes, highs and lows.
• Most people need categories (we used the SSI
  categories for SSI, NSSE, CSS).

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
Assessment Plunge What we looked for

• CSS Looks are the differences between men,
  women, and our students versus other religious 4
  year and all private four year schools. (6
  difference gets our attention.)
• NSSE Looks at first year students and seniors
  versus other CCCU institutions and masters
  schools, and also compares data with all NSSE
  participants (5 difference gets our attention.
• SSI Looks at our students compared to private
  four year schools. We look at items with a
  Performance Gap gt 1.0, and if our scores were
  less than NGM. We also look at the items that
  are most important (already listed) as well as
  the items ranked low in satisfaction.

18
Distribution

• Provost council
• January Faculty Workshop
• In the Academy session
• SGA open student senate
• Student Life divisional workshop/retreat
• Open for other divisional retreats
• President received hard data

19
Feedback categories

• Academic Advising Effectiveness
• Campus Climate
• Campus Life
• Campus Support Services
• Concern for the Individual
• Instructional Effectiveness
• Recruitment and Financial Aid Effectiveness
• Registration Effectiveness

• Responsiveness to Diverse Populations
• Safety and Security
• Service Excellence
• Student Centeredness
• Spiritual Growth Decisions
• Factors Affecting Decision to Enroll
• Extent of Student Involvement in the Community
• Overall/General

20
Academic Advising Effectiveness
21
Academic Advising Effectiveness assesses the
comprehensiveness of your academic advising
program. Academic advisors are evaluated on the
basis of their knowledge, competence and personal
concern for student success, as well as on their
approachability.
22
CSS
23
CSS
24
CSS
25
SSI
26
SSI
27
Campus Life
28
Campus Life assesses the effectiveness of student
life programs offered by your institution,
covering issues ranging from athletics to
residence life. This scale also assesses campus
policies and procedures to determine students
perceptions of their rights and responsibilities.

29
Strengths
30
CSS
31
CSS
32
CSS
33
CSS
34
CSS
35

• SSI Seniors
• better than ngm

36
SSI
better than ngm
37

• SSI Seniors
• better than ngm

38

• SSI Seniors
• better than ngm

39
Areas for Improvement
40
CSS
There was a 13 drop in this physical health
rating from freshman to senior year. (same
students)
41

• SSI Seniors
• better than ngm

42

• SSI Seniors
• Same as ngm

43

• SSI Seniors
• Same as ngm

44

• SSI Seniors
• Same as ngm

45
Importance of context

• Helpful to compare your strengths and
  weaknesses with the overall CCCU data

46
What is most important to your students?
CCCU SSI Averages (15 Highest Importance Scores)
Item No. Imp. Score Description
8 6.63 The content of the courses within my major is valuable.
16 6.61 The instruction in my major is excellent.
68 6.58 Nearly all of the faculty are knowledgeable in their field.
58 6.54 The quality of instruction I receive in most of my classes is excellent
66 6.52 Tuition paid is a worthwhile investment.
39 6.52 I am able to experience intellectual growth here.
33 6.50 My academic advisor is knowledgeable about requirements in my major.
29 6.50 It is an enjoyable experience to be a student on this campus.
34 6.49 I am able to register for classes I need with few conflicts.
74 6.46 Being on this campus is contributing to my spiritual growth.
17 6.46 Adequate financial aid is available for most students.
75 6.44 My understanding of God is being strengthened by classroom and/or campus experiences.
69 6.43 There is a good variety of courses provided on this campus.
7 6.43 The campus is safe and secure for all students.
25 6.41 Faculty are fair and unbiased in their treatment of individual students.
47
Where are your students most satisfied?
CCCU SSI Averages (15 Highest Importance Scores)
Item No. Sat. Score Description
68 5.91 Nearly all of the faculty are knowledgeable in their field.
51 5.81 This institution has a good reputation within the community.
78 5.79 This campus provides adequate opportunities for involvement in ministry.
77 5.79 Males and females are treated with equal respect on this campus.
72 5.78 On the whole, the campus is well-maintained.
39 5.77 I am able to experience intellectual growth here.
2 5.75 The campus staff are caring and helpful.
6 5.73 My academic advisor is approachable.
45 5.70 Students are made to feel welcome on this campus.
3 5.69 Faculty care about me as an individual.
65 5.69 Faculty are usually available after class and during office hours.
29 5.67 It is an enjoyable experience to be a student on this campus.
33 5.66 My academic advisor is knowledgeable about requirements in my major.
75 5.66 My understanding of God is being strengthened by classroom and/or campus experiences.
7 5.65 The campus is safe and secure for all students.
41 5.65 There is a commitment to academic excellence on this campus.
48
Where are your students least satisfied?
CCCU SSI Averages (15 Lowest Satisfaction Scores)
Item No. Sat. Score Description
21 3.66 The amount of student parking space on campus is adequate.
38 4.32 There is an adequate selection of food available in the cafeteria.
42 4.61 There are a sufficient number of weekend activities for students.
17 4.68 Adequate financial aid is available for most students.
11 4.70 Billing policies are reasonable.
71 4.72 Channels for expressing student complaints are readily available.
57 4.74 I seldom get the run-around when seeking information on campus.
73 4.76 Student activities fees are put to good use.
40 4.77 Residence hall regulations are reasonable.
24 4.82 The intercollegiate athletic programs contribute to a strong sense of school spirit.
36 4.88 Security staff respond quickly in emergencies.
23 4.90 Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air conditioning, telephones, etc.
67 4.95 Freedom of expression is protected on campus.
66 4.95 Tuition paid is a worthwhile investment.
19 4.96 My academic advisor helps me set goals to work toward.
49
Where are your students expectations not being
met?
CCCU SSI Averages (15 Largest Performance Gaps)
Item No. Gap Score Description
21 2.37 The amount of student parking space on campus is adequate.
38 1.79 There is an adequate selection of food available in the cafeteria.
17 1.78 Adequate financial aid is available for most students.
66 1.57 Tuition paid is a worthwhile investment.
34 1.50 I am able to register for classes I need with few conflicts.
23 1.44 Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air conditioning, telephones, etc.)
11 1.37 Billing policies are reasonable.
57 1.37 I seldom get the run-around when seeking information on this campus.
36 1.37 Security staff respond quickly in emergencies.
73 1.32 Student activities fees are put to good use.
40 1.28 Residence halls regulations are reasonable.
71 1.24 Channels for expressing student complaints are readily available.
12 1.23 Financial aid awards are announced to students in time to be helpful in college planning.
5 1.21 Financial aid counselors are helpful.
69 1.18 There is a good variety of courses provided on this campus.
50
Institutional Strengths

• The content of the courses within my major is
  valuable. 6.63/5.61
• The instruction in my major field is excellent.
  6.61/5.63
• Nearly all of the faculty are knowledgeable in
  their field. 6.58/5.91
• The quality of instruction I receive in most of
  my classes is excellent. 6.54/5.57
• I am able to experience intellectual growth here.
  6.52/5.77
• My academic advisor is knowledgeable about
  requirements in my major. 6.50/5.66
• It is an enjoyable experience to be a student on
  this campus. 6.50/5.66
• Being on this campus is contributing to my
  spiritual growth. 6.46/5.57
• My understanding of God is being strengthened by
  classroom and/or campus experiences. 6.44/5.66
• The campus is safe and secure for all students.
  6.43/5.65

51
Institutional Strengths (contd)

• My academic advisor is approachable. 6.40/5.73
• There is a commitment to academic excellence on
  this campus. 6.40/5.65
• The campus staff are caring and helpful.
  6.39/5.75
• This institution shows concern for students as
  individuals. 6.38/5.61
• Students are made to feel welcome on this campus.
  6.36/5.70
• Males and females are treated with equal respect
  on this campus. 6.35/5.79
• Given where I am spiritually right now, this
  campus is a good fit for me.
• Faculty are usually available after class and
  during office hours. 6.37/5.69
• Faculty care about me as an individual.
  6.30/5.69
• This institution has a good reputation with the
  community. 6.23/5.84
• On the whole, the campus is well-maintained.
  6.22/5.78
• This campus provides adequate opportunities for
  involvement in ministry. 6.12/5.79

52
Priorities for Intervention

• Tuition paid is a worthwhile investment.
  6.52/4.95
• I am able to register for classes I need with few
  conflicts. 6.49/4.99
• Adequate financial aid is available for most
  students. 6.46/4.68
• Living conditions in the residence halls are
  comfortable (adequate space, lighting, heat, air
  conditioning, telephones, etc.) 6.34/4.90
• Security staff respond quickly in emergencies.
  6.29/4.88
• Financial aid awards are announced to students in
  time to be helpful in college planning.
  6.24/5.02
• Faculty provide timely feedback about student
  progress in a course. 6.23/5.08
• Financial aid counselors are helpful 6.21/5.02
• There are adequate services to help me decide
  upon a career. 6.12/5.12
• I seldom get the run-around when seeking
  information on this campus. 6.12/4.74
• Library resources and services are adequate.
  6.11/5.13
• How satisfied are you that this campus
  demonstrates a commitment to meeting the needs of
  evening students? 4.98

53
Matrix
Union University Union University Union University Union University
Student Satisfaction Index Student Satisfaction Index Student Satisfaction Index Student Satisfaction Index
Matrix for Prioritizing Action Matrix for Prioritizing Action Matrix for Prioritizing Action Matrix for Prioritizing Action
Very Important Very Important Very Important Very Important
Priorities for Intervention Institutional Strengths
17. Adequate financial aid is available for most students. 79. Given where I am spiritually right now, this campus is a good "fit" for me.
23. Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air, etc.) 75. My understanding of God is being strengthened by classroom and/or campus experiences.
66. Tuition paid is a worthwhile investment. 74. Being on this campus is contributing to my spiritual growth.
68. Nearly all of the faculty are knowledgeable in their field.
29. It is an enjoyable experience to be a student on this campus.
39. I am able to experience intellectual growth here.
Very     Very
Dissatisfied Areas that have Low Status for Students Consider Redirection of Resources Satisfied
42. There are a sufficient number of weekend activities for students. 9. A variety of intramural activities are offered.
24. The intercollegiate athletic programs contribute to a strong sense of school spirit.  
13. Library staff are helpful and approachable.  
 
Very Unimportant Very Unimportant Very Unimportant Very Unimportant
54
How We Used Data

• Strategic Planning Union 2010
• Eight Priorities with Goals, Outcomes, Assessment
  Techniques

55
How We Used Data Strategic Planning

• Assessment Result
• SSI Seniors Library Resources and services are
  adequate (weakness at )
• Strategic Plan response
• Priority 3 Cultivate a climate of excellence
  and innovation based on our distinctive vision
  and mission with continuous and measurable
  improvement, in all aspects of University life.
  (Student Services Committee)
• The University will develop its library and
  computing resources and services to meet the
  scholarship needs of the entire learning
  community, consistent with the Technology Master
  Plan. (Associate Vice Presidents of Academic
  Resources and Information Services)
• The University will develop library physical
  facilities on both the Jackson and Germantown
  campuses which will serve as academic commons of
  learning. (Associate Vice President of Academic
  Resources)

56
How We Used Data Strategic Planning

• Assessment Result
• CSS 62 of our seniors did not have a course
  that included community service/service learning
  since entering college compare to other religious
  (39) and all private (46).
• Strategic Plan response
• Priority 3 Cultivate a climate of excellence
  and innovation based on our distinctive vision
  and mission with continuous and measurable
  improvement, in all aspects of University life.
  (Student Services Committee)
• The University will focus on student learning
  through additional opportunities in 1)
  international and intercultural experiences, 2)
  service-learning courses, and 3) excellence in
  the classroom through 21st century pedagogical
  strategies such as web enhanced learning.
  (Provost/Academic Deans/Quality Enhancement
  Planning Team)

57
Other uses of data

• Food Services improvements move to a block meal
  plan
• Bookstore manager now attends national
  merchandise meeting
• Marketing team
• Financial Aid hired additional staff

58
Who else requested data?

• Examples include
• Minority Task Team
• First year experience task team
• Bookstore

59
Discussion

• Who are your audiences? How many times do you
  present it? How are the results tailored for each
  audience?
• Does your university have a limit to how many
  issues you tackle based on the data? Pros/cons?
• How accessible is your data on your campus? (e.g.
  website, hard copies, etc.)

60
Thank you