Fred Trapp, Ph.D.

1
Comprehensive Assessment Reports

• Fred Trapp, Ph.D.
• Administrative Dean, Institutional
  Research/Academic Services (Retired)
• Long Beach City College
• Cambridge West Partnership, LLC
• Robert Pacheco, Ed.D.
• Director Of Institutional Planning, Research and
  Resource Development
• Barstow College

2
Outcomes for the Session

• The participant will be able to
• Describe the comprehensive assessment report
  concept.
• Locate best practice examples from other colleges
  through web links.
• Discuss how the comprehensive report idea can be
  used as part of the institutions learning
  process and as a means by which the institution
  provides quality assurance to the public.
• Indicate national trends and efforts of
  consortia/national organizations to provide
  quality assurance about student experiences and
  learning outcomes.
• Please hold questions until the end.

3
(No Transcript)
4
Curriculum Map
4
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
ACCJC Institutional Effectiveness Rubric

• Part III Student Learning Outcomes
• Proficiency
• Comprehensive assessment reports exist and are
  completed on a regular basis.

9
What Did We Look At, With Whom Did We Consult?

• ACCJC. Institutional Effectiveness Rubric
• ACCJC. 2002 Standards
• ACCJC. Themes
• ACCJC. Guide to Evaluating Institutions
• Professional literature
• Efforts of national groups/institutes
• Institutional web sites, listservs and colleagues

10
Guiding Questions

• How can the report writing experience
• Help faculty explore the student learning
  process?
• Determine the extent to which the curriculum is
  working?
• Where can time, energy and/or money be allocated
  for continuous improvement in learning?
• Exploit the writing process and dialogue about
  results to gain broader institutional learning
  experiences?
• Help meet our quality assurance pledge to the
  community?

11
Illustration Selections

• Council for Higher Education Accreditation (CHEA)
• Annual Award for Outstanding Institutional
  Practice in Student Learning Outcomes
• Demonstrated commitment to developed highly
  effective practice to use SLO assessment
• Willingness to share the practices they developed
• Selection committee
• Selection criteria
• Articulation evidence of outcomes
• Success with regard to outcomes
• Information to the public about outcomes
• Using outcomes for improvement

12
Illustration selections (continued)

• Cited by scholars and peers
• Schools with assessment work cited in scholarly
  books and articles
• Schools with assessment work selected for
  presentation at national conferences
• Web presentations publicly available for you to
  ease drop upon
• Prominent national movements/initiatives
  regarding learning outcomes assessment
  documentation (including public quality assurance)

13
What Might Be Included?

• Assessment focus- course, program, general ed,
  etc.
• What outcomes were assessed?
• How and when were they assessed?
• Who was assessed?
• What were the results?
• Who reviewed the results, made sense of the them
  and what conclusions were reached?
• What are the implications for practice and/or
  policy or future assessment work?

14
CC of Baltimore County (MD)Course-level Reporting

• Middle States Commission on Higher Education
• Community College Futures Assembly, Bellwether
  Award, 2008
• Instructional Programs Services for High Impact
  Course Level Assessment
• CHEA award winner, 2006
• Institutional Progress in Student Learning
  Outcomes
• National Council on Student Development (NCSD)
  Exemplary Practice Award Winner

15
CC of Baltimore County (MD)Course-level Reporting

• Projects are at least three semesters long
• Individual and high-impact courses (all sections)
  included
• Project proposal by a faculty group
• Measurable objectives
• External review approval in selecting
  methods/instrument analyzing results.
  Benchmarking should be included if possible.
• Controls and sample size considered.
• Course improvements based on data analysis
• Reassessment expected
• Results/report shared across the college and web
  posted

16
CC of Baltimore County (MD)

• Learning Outcomes Assessment Final Report
  Template
• Design proposal for the LOA project
• Implementation of design data collection
• Redesign of the course to improve student
  learning
• Implementation of course revisions reassessment
  of student learning
• Final analysis and results
• eavesdropping
• http//www.ccbcmd.edu/loa/CrseAssess.html
• Two-page executive summaries available

17
CC of Baltimore County (MD)

• CHEM 108
• An initial failure turned to success and
  collaboration with a four-year school
• HLTH 101
• Addressing an achievement gap with professional
  development and increased communication with
  students
• CRJU 101 and 202
• Statewide group assessment development effort and
  creativity in the interventions used

18
Program-level Reporting

• North Central Association of Colleges Schools,
  Higher Learning Commission
• CHEA award winner, 2008
• Institutional Progress in Student Learning
  Outcomes

19
Hocking College (OH)

• Learning outcomes data collected in a student
  E-portfolio
• Direct internal and external evidence (1 to 10
  measures)
• Indirect evidence (1 to 4 measures)
• Evidence drawn from samples of student work for
  faculty to apply an agreed upon holistic rubric
• Eight general education outcomes (student success
  skills)
• Discipline-specific exit competencies or outcomes
• See the Hocking wheel

20
(No Transcript)
21
Hocking College (OH)Program-level Report

• eavesdropping
• Cloud reference, not college URL as links are
  broken there
• Various reports available in each program profile
• Curriculum matrix
• Criteria statements (exit competencies)
• Instructional Program Outcomes (assessment plan)
• Trend Charts for performance criteria

22
Hocking College (OH)Program-level Report

• Example reports and analysis
• Culinary Arts Technology
• Forestry Management Technology
• Nursing Technology

23
Mesa College (AZ)General-Education Reports

• North Central Association of Colleges Schools,
  Higher Learning Commission
• CHEA Award winner, 2007
• Institutional progress in Student Learning
  Outcomes

24
Mesa CollegeGeneral Education Report

• Multiple outcomes assessed
• Annually
• Annual Report elements
• Executive Summary
• Methodology
• Results observations (GE workplace)
• Indirect measures findings
• Appendices of past results

25
Mesa Community College (AZ)

• General education studies completed 2007-08
  2005-06
• Numeracy
• Scientific inquiry
• Problem solving/critical thinking (2008-09)
• Information literacy
• Workplace skills (CTE) (2009-10)
• General education studies completed 2006-07
  2004-05
• Arts humanities
• Cultural diversity
• Oral communication
• Written communication

26
Mesa College (AZ)General Education Reports

• eavesdropping
• http//www.mesacc.edu/about/orp/assessment/index.h
  tml
• Annual reports and summaries available
• Nine years of history and experience
• 14 years of assessment work

27
Capital CC (CT)General-Education Reports

• New England Association of Schools and Colleges,
  Commission on Institutions of Higher Education
• Cited in the Art and Science of Assessing General
  Education Outcomes A Practical Guide (AACU,
  2005)
• eavesdropping
• http//www.ccc.commnet.edu/slat/
• Annual reports and summaries available

28
Portland CC (OR)General Education Reports

• Northwest Accrediting Commission
• eavesdropping
• http//www.pcc.edu/resources/academic/learning-ass
  essment/
• One general education theme a year
• Learning Assessment Focus for 2009-10- Critical
  Thinking Problem Solving
• Physical Science, Geology and General Science
• Bioscience Technology
• Management and Supervisory Development
• Culinary Assistant Program

29
Truman State University (MO)Various Reports

• Southern Association of Schools and Colleges,
  Commission on Colleges
• eavesdropping
• Assessment work began in 1970
• http//assessment.truman.edu/
• Assessment Almanac- A compilation of results from
  each years assessment work (versions from 1997
  to 2009 are posted)
• General Education outcomes are assessed in the
  context of the major field of study
• Portfolio Project- required of all seniors to
  show best work assessed by faculty for the nature
  quality of the liberal arts and sciences
  learning outcomes (versions from 1997 to 2008 are
  posted)

30
Authorship

• Course-level, program-level general education
• Teaching faculty study team with technical
  assistance from
• institutional research or assessment committee
• No lone ranger authors
• Institutional summary
• Academic administrator with assistance from
• Learning outcomes coordinator or assessment
  committee
• Compilation of work accomplished in one or two
  academic years across the institution

31
Putting the Reports to Work

• To all affected participants
• Campus committees
• Curriculum, assessment, resource allocation
  group, unit (department) leadership, general
  academic and college leadership
• Campus fairs, brown-bag lunches, poster sessions
  for information sharing
• Faculty professional development programs
• Accreditation self-study committee work groups
• Local governing board presentation
• College web site for the public

32
Reports a Learning Organization

• Learning organization
• Environment that promotes a culture of learning
• Individual group learning enriches enhances
  the organization as a whole
• Systematic problem solving using data for
  decisions
• Learning from experiences in assessing
  organizational performance
• Comparing yourself to others (benchmarking) and
  borrowing ideas
• Adriana Kezar ed. Organizational Learning in
  Higher Education New Directions for Higher
  Education. No. 131, Fall 2005. Jossey-Bass.

33
Reports as Institutional Learning Resource
Allocation

• The assessment data sense-making process a
  faculty learning experience
• Linking results to future interventions a
  learning experience
• Using results to inform an intervention, then
  reassess a learning experience (accomplished
  one or more terms later)
• Reference for future assessment work and other
  groups on campus

34
Reports as Institutional Learning Resource
Allocation

• Hocking College (OH)
• Annual summary
• Improvements in the program in the previous year
  brought on by study of assessment results
• Expenditures of time, money materials for the
  assessment program
• Requests for assistance in implementing
  assessment
• Recommendations for altering the institutions
  assessment process
• Transition from evaluating individual students to
  assessing groups of students the curriculum
  experience

35
Reports as Institutional Learning Resource
Allocation

• Community College of Baltimore County (MD)
• Learning Outcomes Assessment Advisory Board
• Links findings in assessment reports to other
  college-wide initiatives and professional
  development opportunities
• Use of assessment processes and (findings)
  results
• Challenged faculty to reexamine prompts used in
  assessment
• Clarity of written prompt extent it supports
  program goals
• Common assignment options and common rubric
  increases faculty understanding and buy-in
• Builds faculty unity toward common goals
• Public web page enhances communication and
  accessibility to information

36
Reports as Institutional Learning Resource
Allocation

• CHEA award winner, 2010
• Institutional Progress in Student Learning
  Outcomes

37
Reports as Institutional Learning Resource
Allocation

• Feedback recognition
• Feedback rubric for annual assessment reports
• Conversations and action
• Collection and analysis of evidence
• Implementation of findings
• Recognition (achievement excellence)

38
Reports as Institutional Learning Resource
Allocation

• Seal of Assessment Achievement
• Academic programs earning this recognition have
  demonstrated in their annual report that
• learning outcomes have been assessed through two
  or more methods, and
• findings have been discussed among
• the faculty.

39
Reports as Institutional Learning Resource
Allocation

• Seal of Assessment Excellence
• Academic programs earning this recognition have
  demonstrated
• a thorough implementation of assessment plan(s)
• the reporting of meaningful assessment data
• the discussion of findings among faculty
• and perhaps students
• the use of findings to showcase
• student achievements and
• to make curricular adjustments.

40
Reports as Institutional Learning Resource
Allocation

• Mesa College (AZ)
• Results Outreach Committee
• Promotes use of outcomes data in relation to
  faculty development, pedagogy and academic
  climate
• Groups of faculty offer a proposal for summer or
  academic year work above the course level
• Resulting report placed on the web and used for
  campus discussion and action

41
Report as Quality Assurance

• NILOA
• 2010 Webscan report Exploring the Landscape
  What Institutional Websites Reveal About Student
  learning Outcomes Assessment Activities
• 2010 Connecting State Policies on Assessment with
  Institutional Assessment Activity
• 2011 Transparency Framework (online)
• eavesdropping
• www.learningoutcomeassessment.org

42
Report as Quality Assurance

• Promising Vehicles for Expanding Information to
  the Public
• Brief narrative report from annual assessment
  reports
• Simple statistical reports on learning outcomes
  or surveys
• Best practices stories supported by assessment
• Peter Ewell. Accreditation the Provision of
  Additional Information to the Public about
  Institutional and Program Performance, CHEA, May
  2004

43
Quality Assurance to the Public

• Voluntary System of Accountability
• APLU AASCU (520 public institutions, award 70
  of bachelors degrees in the US each year)
• College Profile (includes learning outcomes
  links to campus)
• Proactive initiative to document learning gains
  and average institutional scores (choice of 3
  national instruments)
• Proactive initiative to illustrate unique campus
  learning outcomes assessment work
• Promoting a learning institution
• eavesdropping
• http//www.collegeportraits.org/

44
Quality Assurance to the PublicVSA Example, CSU
Monterey Bay

• http//www.collegeportraits.org/map
• Cal Poly Pomona
• http//www.collegeportraits.org/CA/CPP/learning_ou
  tcomes
• Learning Assessment Examples

45
Quality Assurance to the Public

• National Association of independent Colleges and
  Universities
• Assessment programs on campus tied to
  institutions mission
• eavesdropping
• http//www.naicu.edu/special_initiatives/accountab
  ility/Student_Assessment/id.514/default.asp
• Pepperdine University
• http//services.pepperdine.edu/oie/learning-outcom
  es/learning-outcomes-overview.aspx

46
Where Can I Go?Resource

• Filesanywhere.com
• http//www.filesanywhere.com/fs/v.aspx?v8a69668b5
  c6773a96f6d

47
Contacts Questions

• Robert Pacheco (Barstow College)
• Rpacheco_at_Barstow.edu
• Fred Trapp (Cambridge West Partnership)
• fredtrapp_at_gmail.com
• Questions and Comments

48
Session EvaluationOutcomes for the Session

• The participant will be able to
• Describe the comprehensive assessment report
  concept.
• Locate best practice examples from other colleges
  through web links.
• Discuss how the comprehensive report idea can be
  used as part of the institutions learning
  process and as a means by which the institution
  provides quality assurance to the public.
• Indicate national trends and efforts of
  consortia/national organizations to provide
  quality assurance about student experiences and
  learning outcomes.