The Role of Institutional Research

1
Assessment
The Role of Institutional Research
"What Should I Be Doing?"
2
Your mission, if you choose to accept it is to
pull all the pieces together.
levels of assessmentplanning and
assessment direct and indirect measuresprogram
review accreditationresourcesgroups
3
Overview

• Definitionsfor context
• Roles
• Facilitator/Advisor
• Leadership
• Planning
• Traditional institutional research indirect
  assessment
• More context
• Developing a culture of inquiry and climate of
  trust
• Organizational/staffing considerations

4
Definitions
Assessment is an ongoing process aimed at
understanding and improving student learning. It
involves making our expectations explicit and
public setting appropriate criteria and high
standards for learning quality systematically
gathering, analyzing, and interpreting evidence
to determine how well performance matches those
expectations and standards and using the
resulting information to document, explain, and
improve performance. When it is embedded
effectively within larger institutional systems,
assessment can help us focus our collective
attention, examine our assumptions, and create a
shared academic culture dedicated to assuring and
improving the quality of higher education.
(Thomas A. Angelo)
5
Direct Assessment of Student Learning
OutcomesCognitive Skills, Knowledge, and
Abilities
Definitions
Plan - identify objectives
College Context Communication Dialogue
Assess/evaluate Review, analyze, feed back
Implementation Curriculum and pedagogy
6
Facilitator/advisor/expert
Roles

• Learn, learn, learn about assessment
• definitions, models, strategies
• conferences, workshops, literature

• Become a central resource for
• professional expertise
• providing standard, consistent definitions,
• literature and searches
• coordination and communication/internal
• external
• facilitation

7
Leadership
Roles

• Initiate assessment programs, strategically
• Bring new information to the table, identify
  connections, e.g., new brain, learning research
• Encourage participation in training opportunities
• Do the work
• Identify organizational linkages, e.g
• program review
• accreditation
• staff development
• Promote the dialogue, ask the questions

8
?
?
?
9

• What evidence along students careers would
  document students
• progress towards expected learning outcomes?
• Do students achieve the expected student
  learning outcomes? (summative
• evaluation)
• What helped or hindered students in achieving
  the SLOs? (formative
• evaluation)
• What strategies are effective for using
  assessment to improve learning?
• Are the connections among assessment, planning,
  delivery, and funding
• evident?
• What do you talk about during your
  college/committee/department meetings?
• Do faculty and staff ask questions?

10
Planning
Roles

• Integrate assessment into all levels of
  planning
• mission, goals, departmental

Plan - identify objectives
College Context Communication Dialogue
Assess/evaluate Review, analyze, feed back
Implementation Curriculum and pedagogy
11
Direct Assessment of Student Learning
Outcomes Sam Pull College Framework
Plan-Identify objectives
Assessment methods
College-wide General Education Student
Learning Outcomes (SLOS)
Discipline/Program Specific Student Learning
Outcomes (SLOS)
Achievement tests Capstone experience
Competencies/Criteria Levels (Beginner,
Developed, Accomplished)
Competencies/Criteria Levels

Implementation
For each course Level, curriculum, pedagogy
For each course Level, curriculum, pedagogy
Rubrics Term papers Research Projects
What do you want students to be able to do? to
know? to value?
12
Palomar Colleges Core Skills, Benchmarks,
Competencies, and Levels A. Communication

• Students will communicate effectively in many
  different situations, involving diverse people
• and viewpoints. (core skill)
• Listening Students will listen actively and
  respectfully to analyze the substance
• of others comments. (benchmark)
• Demonstrated Competencies for Each Level
• Beginner
• Avoid interrupting the speaker
• Summarize speakers main points when called upon
  to do so.
• Developed
• Develop a framework for organizing the message.
• Accomplished
• Differentiate between denotation and connotation
  recognize irony, metaphorical language, and
  intentionally misleading language

13
Palomar Colleges Core Skills, Benchmarks,
Competencies, and Levels (continued)

• Reading (benchmark)
• Beginner
• Developed
• Accomplished
•  3. Writing
• Speaking
• B. Cognition (core skill)
• 1. (benchmark)
• 2.
• C.  Information Competency (core
  skill)
• 1.

14
Traditional Institutional ResearchIndirect
Assessment
Roles
15
DIRECT AND INDIRECT MEASURES OF STUDENT LEARNING
OUTCOMES
16
Organizational/Staffing Considerations
More Context

• Nature of existing IR is consideration
• An Assessment Coordinator/Director?
• Where does it fit organizationally?
• Notion that assessment crosses silos and
• areas
• Increasing expectations of institutional
• research, quality and quantity
• Should all research analysts be assessment
  experts?
• Workload issues
• Imperative and common for institutional
• research to be involved

17
Creating a Culture of Evidence and a Climate of
Trust
More Context

• Implement creative and multiple strategies for
  infusing information into environment
• Continually identify conclusions and connections
  and integrate them into the dialogue
• Taking risks is encouraged and failure is ok
• Spend time with faculty, staff, groups
• Honesty, responsiveness, and feedback is
  important, as well as diplomacy
• Managed conflict is constructive