Assessment

1
Assessment

• A New Way of Thinking and Seeing Things

2
A FEW Issues Facing Academic Libraries

• Assessment
• Contributing to student success

• Focus on learning more than imparting
• How do students learn?
• Information Literacy
• ACRL competencies
• http//www.ala.org/ala/acrl/acrlstandards/informat
  ionliteracycompetency.htm

3
Quotation

• Assessment and evaluation are intended as means
  to document educational quality and institutional
  effectiveness, foster institutional improvement,
  and demonstrate accountability.
• Unfortunately, without a clear sense of the
  purpose for assessment, knowledge of what is to
  be assessed, and understanding how the results
  will be used, assessment efforts,

4
Quotation (continued)

• all too often, become an end in themselves.
• If the results of assessment are not used for
  information planning and decisions, colleges and
  universities often find themselves in positions
  of being data rich and information poor.
• Ronald L. Baker, Outcomes Assessment in Higher
  Education, p. 12.

5
NEASC STANDARD SEVEN Library and Information
Resources

• 1 The institution makes available the library
  and information resources necessary for the
  fulfillment of its mission and purposes. These
  resources support the academic and research
  program and the intellectual and cultural
  development of students, faculty, and staff.
  Library and information resources may include the
  holdings and necessary services and equipment of
  libraries, media centers, computer centers,
  language laboratories, museums, and any other
  repositories of information or technological
  systems required for the support of institutional
  offerings. Clear and disseminated policies govern
  access, usage, and maintenance of the library,
  information resources, and services. The
  institution ensures that students use these
  resources as an integral part of their education.
• 2 Through the institution's ownership or
  guaranteed access, sufficient collections,
  information technology systems, and services are
  readily accessible to students wherever programs
  are located or however they are delivered. These
  collections, systems, and services are sufficient
  in quality, level, diversity, quantity, and
  currency to support and enrich the institution's
  academic offerings. The institution provides
  facilities adequate to house the collections and
  equipment so as to foster an atmosphere conducive
  to inquiry, study, and learning among students,
  faculty, and staff.

6
NEASC STANDARD SEVEN (continued)

• 3 The institution provides sufficient and
  consistent financial support for the effective
  maintenance and improvement of the institution's
  library, information resources, and services. It
  makes provision for their proper maintenance,
  preservation, currency, and security. It
  allocates resources for scholarly support
  services compatible with its instructional and
  research programs and the needs of faculty and
  students.
• 4 Professionally qualified and numerically
  adequate staff administer the institution's
  library, information resources, and services. The
  institution provides appropriate orientation and
  training for use of these resources, as well as
  instruction in basic information literacy.
• 5 The institution participates in the exchange
  of resources and services with other institutions
  and within networks as necessary to support and
  supplement its educational programs. It provides
  appropriate support for distance learning
  students and faculty, such as on-line reference
  service and contractual access to relevant
  off-campus library resources.
• 6 The institution regularly and systematically
  evaluates the adequacy and utilization of its
  library, information resources, and services and
  uses the results of the data to improve and
  increase the effectiveness of these services

7
NEASC STANDARD FourPrograms and Instruction

• Undergraduate Degree Programs
• Graduate Degree Programs
• Scholarship and Research
• Instruction
• Admissions and Retention
• http//community.pmc.edu/neasc/standards/Default.h
  tm

8
Student Outcomes versus Student Learning Outcomes

• Student outcomes are aggregate statistics on
  groups of students (e.g., graduation rates,
  retention rates, transfer rates, and employment
  rates for a graduating class)
• Such outcomes are institutional outcomes and are
  used to compare institutional performance
• They do not measure changes in students
  themselves due to their college experience
• these outcomes are outputs and reflect what the
  institution has accomplished they do not reflect
  what (or how much) students learned

• Student learning outcomes development of
  students demonstrable acquisition of specific
  knowledge and skills
• How well do students
• Transfer and apply concepts, principles, ways of
  knowing, and problem solving across their major
  program of study?
• Integrate their core curriculum, general studies,
  or liberal studies into their major program or
  field of study?
• Develop understanding, behaviors, attitudes,
  values, and dispositions that the institution
  asserts it develops?
• Maki (2004)

9
Overview QuestionsDoes the Library Have a Role
to Play?

• What should students learn?
• How does the content of one course relate to
  another?
• How well are they learning it?
• Across courses
• What evidence do we gather to answer these
  questions?
• Beyond graded assignments, course evaluations
• How do we use that evidence to improve learning
• Looking a courses as comprising a program

10
Frameworks
Course Evaluation Grades Student course
evaluation Course Assessment Formative
Summative Program Assessment
Program as a set of courses with some
interconnection
11
Examples of Different Perspectives
Taxpayers Cost, efficiency, affordability
12
Evaluation Defined

• The process of identifying and collecting data
  about specific services or activities,
  establishing criteria by which their success can
  be assessed, and determining both the quality of
  the service or activity and the degree to which
  the service or activity accomplishes stated goals
  and objectives.

13
Assessment Defined

• The ongoing process of
• Establishing clear, measurable expected outcomes
  of student learning
• Ensuring that students have sufficient
  opportunities to achieve those outcomes
• Systematically gathering, analyzing, and
  interpreting evidence to determine how well
  student learning matches our expectations
• Using the resulting information to understand and
  improve student learning at both the course and
  program level Suskie
  (2004, p. 3)

14
Purposes of Assessment

• Improving student learning
• Work collaboratively thinking beyond the set of
  courses you teach

• Accountability and accreditation
• Meeting institutional mission
• Your mission is
• Demonstrating institutional effectiveness

15
Institutional Effectiveness

• Defined as how well an institutions achieves its
  mission and major institutional goals
• Since student learning is the heart of most
  institutional missions, the assessment of student
  learning is a major component of the assessment
  of institutional effectiveness.
• However, institutional effectiveness examines
  other aspects such as scholarship and research,
  community service, etc.
• Suskie, 9-10

16
Peggy I. Maki defines learning as

• Knowledge leading to understanding
• but also
• Abilities
• Habits of mind
• Ways of knowing
• Attitudes
• Values
• Other dispositions that an institution and its
  programs and services assert they develop. (p.
  3)

17
Student Learning Outcomes
Skills Oral/written communication Foreign
language communication Technological
sophistication Quantitative reasoning
ability Other
Conceptual Leadership Critical Thinking Problem
Solving Information Literacy Global
Citizen Values (moral, etc.)
Do satisfaction and service quality impact
performance?
18
What Is a Rubric?

• A scoring guide a list, chart, or guide that
  describes a set of the criteria (with levels of
  achievement along explicit dimensions) that
  enable us to score or interpret learning as
  students progress through the program
• Perhaps using outside raters
• A rubric translates a statement of a student
  learning into that set of criteria
• Maki, Chapter 5 Hernon and Dugan, 274-88

19
Verb Choices for Phrasing Student Learning
Outcomes

• Analyze
• Apply
• Demonstrate
• Develop
• Differentiate
• Identify
• Recognize
• Synthesize

• NOT
• Discuss
• Understand
• Etc.

20
Examples of Student Learning Outcomes

• Students will think logically and critical in
  solving problems be able to evaluate, critique,
  and apply the thinking of others and reach an
  appropriate conclusion.
• Novice, Intermediate, Advanced
• Students will demonstrate leadership abilities
  (Leadership abilities evidenced in learning
  contexts include team building and shared
  decision making, strategic planning, advocacy,
  and consensus building and collaboration)
• Novice, Intermediate, Advanced

21
Direct Methods

• Embedded course assessment (performance on
  assignments, etc. minute paper)
• Portfolio assessment
• Performance (internships, practicum, student
  teaching)
• Professional jurors or evaluators
• Capstone course/experience

• Experimental research designs), with pre- and
  post-testing
• Use of standardized tests
• Think-aloud protocol
• Directed conversation
• Videotape/audiotape evaluation
• Analysis of theses/dissertations/
• senior papers (content analysis, interviews,
  or oral defense)

22
Indirect Methods

• Surveys (self-reporting) and self-assessments
• Curriculum and syllabus evaluation
• Exit interviews
• Observation

• Other

23
Summary Outcomes Assessment

• Standard 4 student outcomes and student learning
  outcomes
• Student learning outcomes measures changes in
  library users as a result of their contact with
  the librarys programs, resources and students
• Institutional and program improvement, not
  external benchmarking however, might be of some
  interest in best practices

24
Any questions or issues I can help with today OR
25
Bibliography

• Hernon, P., Dugan, R. E.(2004). Outcomes
  Assessment in Higher Education Views and
  Perspectives. Westport, CT Libraries Unlimited.
• Hernon, P., Dugan, R.E., Schwartz, C. (2006).
  Revisiting Outcomes Assessment in Higher
  Education. Westport, CT Libraries Unlimited.
• Hernon, P., Calvert, P. Improving the Quality
  of Library Services for Students with
  Disabilities. Westport, CT Libraries Unlimited.
• Maki, P. L. (2004). Assessing for Learning
  Building a Sustainable Commitment across the
  Institution. Sterling, VA Stylus.
• Maki, P. L., Borkowski, N. A.(2006). The
  Assessment of Doctoral Education Emerging
  Criteria and New Models for Improving Outcomes.
  Sterling, VA Stylus.
• Suskie, L. (2004). Assessing Student Learning A
  Common Sense Guide. Bolton, MA Anker.