Cultivating Assessment on Your Campus

1
Cultivating Assessment on Your Campus

• NASPA Region III
• Summer Symposium
• June 1, 2008
• Pre-Conference Session

2
Session Overview

• Introduction
• Defining Assessment
• Categorizing Assessment
• Creating an Assessment Plan
• Conducting the Assessment
• Sharing and Using Results

3

• We are all too busy to be doing assessment if
  its only purpose is filling shelves. However,
  effective research can often save us time and
  make us more effective in our work with
  students.
• - Randy Swing, Ph.D Co-Director Senior
  Scholar, Policy Center on the 1st Year of
  College
• July 2004, Opening Remarks at the Summer
  Institute on First-Year Assessment

4
Intended Outcomes

• Participants will
• Expand knowledge of assessment and evaluation
• Learn what current research says about student
  affairs assessment
• Learn creative techniques to conduct and share
  assessment
• Share ideas for developing and supporting
  research within campus student affairs
  departments
• Gain practical knowledge and skills for
  application on their campus

5
Defining Assessment

• Research
• Systematic investigation of a subject aimed at
  uncovering new information (discovering data)
  and/or interpreting relations among the subjects
  parts (theorizing). (Vogt, 1999)
• Evaluation
• The process of determining the merit, worth, or
  value of something, or the product of that
  process. (Scriven, 1991)
• Assessment
• Assessment is any effort to gather, analyze, and
  interpret evidence which describes institutional,
  divisional, or agency effectiveness. (Upcraft
  Schuh, 1996)

6
Evolution of Assessment

• Accountability to public constituents
• Poor reflections of college graduates
• Increasing costs, decreasing quality
• Under-representation of diverse groups
• Necessary for Accreditation
• Constant improvement/evaluation of goals and
  objectives
• Critical Issue in Student Affairs

7
Creating an Assessment Culture

• Divisional Commitment
• Department Commitment
• Student Involvement
• Relate to institutional strategic plan and
  mission
• Accountability

8
Purpose of Assessment

• Internal
• Decision Making
• Program Evaluation Improvement
• Budget allocations
• Marketing Education
• Interventions

• External
• Accreditation reports
• Grant reports
• Benchmarking
• Professional Development
• Publicity for recruiting, alumni news, local
  media, etc.

9
Categorizing Assessment

• Program Outcomes
• Provide data concerning activities, services, or
  processes that the program undertakes in order to
  meet campus needs
• Student Learning and Development Outcomes
• Provide evidence of actual impacts, benefits or
  changes for students

10
Standards

• Professional Organization Standards
• CAS Standards
• Institutional strategic plan
• Institutional mission
• Department goals and objectives
• Program outcomes
• Student learning and developmental outcomes

11
Methods for Assessment

• Quantitative
• Qualitative

12
Value of Quantitative Method(Keeling, R., Wall,
A. Underhile, R. Dungy, 2008)

• Economy
• Quick and low-cost
• Generalizability
• Representative sample population
• Reliability and Validity
• Rigorous criteria
• Random sampling, response rates, replicatability,
  internal validity

13
Types of Quantitative Data(Keeling, R., Wall, A.
Underhile, R. Dungy, 2008)

• Institutional research
• Department of Education
• Test and grading data
• SAT, GPA
• Large survey data
• Cooperative Institutional Research Program
  (CIRP) College Learning Assessment (CLA)
• Local survey data
• Campus administrative/department surveys

14
Value of Qualitative Method (Keeling, R., Wall,
A. Underhile, R. Dungy, 2008)

• More focus on measuring impact
• Skills development
• Defining and affirming why
• Method closely tied to purpose of assessment
• Produce in-depth results
• Better understanding of participants reasoning
  behind responses

15
Types of Qualitative Assessment (Keeling, R.,
Wall, A. Underhile, R. Dungy, 2008)

• Biography
• Interviewing students lives and learning
• Phenomenology
• Interviewing students specific learning from
  experience
• Grounded Theory
• Interviewing sampling to understand overall
  experiences

16
Types of Qualitative Assessment (Keeling, R.,
Wall, A. Underhile, R. Dungy, 2008)

• Ethnography
• Interviewing and artifact reviews Understand
  student experience
• Case Study
• Interviewing and artifact reviews Measure impact
• Self-assessment
• CAS Standards
• Frameworks for Assessment for Learning and
  Developmental Outcomes (FALDOS)

17
Instruments and Measures

• Criteria and rubrics
• Interviews and focus groups
• Peer reviews
• Observations
• Surveys (paper and online)

18
Creating an Assessment Plan

• Purpose of the assessment
• Target population
• Method
• Dissemination of results
• Timeline

19
Planning Steps of a Good Assessment
Program(Schuh Upcraft, 2001)

• Define the Problem
• Determine the Purpose of the Study
• Determine Where to Get the Information Needed
• Determine the Best Assessment Methods
• Determine Whom to Study
• Determine How the Data Will Be Calculated

• Determine What Instruments Will Be Used
• Determine Who Should Collect the Data.
• Determine How the Data Will be Analyzed.
• Determine the Implications of the Study for
  Policy Practice
• Report the Results Effectively.

20

• The best assessment programs share timely
  information in varied ways
• Palomba Banta, 1999, p. 15

21
Sharing and Using Results

• Who needs the results?
• Internal audiences
• Wider campus community
• Student community
• External audiences

22
Internal Uses

• Communicate the reasons why we do the work we do
• Make the information pertinent to continue
  engagement of staff
• Can present only the research because the
  audience understands the background
• Creative ways to use results
• Staff Newsletters - Interesting Facts
• Staff Retreats/Meetings same page

23
Interesting Facts Simple Example
24
Campus Community Uses

• Student Affairs is difficult to understand
• Chance to share important information to gain
  validity
• Opposite approach share information, but backed
  by research
• Creative ways to use results
• Letter to Faculty with important facts
• Support to change policy

25
Student Uses

• Learn information through interaction or quick
  information
• Just the basics with the option to learn more
• Creative ways to reach students with stats
• Question of the Week
• Newspaper Ads
• Website Driven

26
Question of the Week
27
External Uses

• Need background information first along with
  research
• Increase the town-gown relationships
• Share ideas and achievements among peers
• Creative ways to use results
• Snapshots
• Newspaper Ads
• Website Driven

28
Snapshot
29
FSU Web Sites

• Office of Research, Office of the Vice President
  for Student Affairs http//dsaresearch.fsu.edu
• Research and Communications, Oglesby Union
  http//union.fsu.edu/education

30
References

• Banta, T. W., Lund, J. P., Black, K. E.
  Oblander, F. W. (1996). Assessment in practice
  Putting principles to work on college campuses.
  San Francisco Jossey-Bass Inc.
• Palomba, C.A. Banta, T.W. (1999). Assessment
  essentials Planning, implementing, and improving
  assessment in higher education. San Francisco
  Jossey-Bass Inc.
• Schuh, J. H. Upcraft, M. L. (2001). Assessment
  in practice in student affairs. San Francisco
  Jossey-Bass Inc.

31
References

• Scriven, M.  (1991). Evaluation Thesaurus (4th
  ed.). Thousand Oaks, CA Sage Publications.
• Upcraft, M. L. Schuh, J. H. (1996) Assessment
  in student affairs A guide for practitioners.
  San Francisco Jossey-Bass Inc.
• Vogt, W.P. (1999). Dictionary of statistics
  methodology A nontechnical guide for the social
  sciences (2nd ed.). Thousand Oaks, CA Sage
  Publications

32
Contact Us

• Allison Hawkins Crume, Ph.D.
• acrume_at_fsu.edu
• Associate Director, Oglesby Union, Florida State
  University
• Tamara Bertrand Jones, Ph.D.
• tbertrand_at_fsu.edu
• Research Coordinator, Division of Student
  Affairs, Florida State University