Using WIDS to Document How Colleges Assess Learning Outcomes

1
Using WIDS to Document How Colleges Assess
Learning Outcomes
2
Overview

• What is assessment?
• What are the drivers?
• How does curriculum fit in?
• How does WIDS help?

3
Assessment of Learning
NCA Higher Learning Commission

•  Assessment of student learning is a
  participatory, iterative process that
• Provides data/information you need on your
  students learning
• Engages you and others in analyzing and using
  this data/information to confirm and improve
  teaching and learning
• Produces evidence that students are learning the
  outcomes you intended
• Guides you in making educational and
  institutional improvements
• Evaluates whether changes made improve/impact
  student learning, and documents the learning and
  your efforts.

From Student Learning, Assessment and
Accreditation Criteria and Contexts, presented
at Making a Difference in Student Learning
Assessment as a Core Strategy, a workshop from
the Higher Learning Commission, July 26-28,
2006. 
4
Assessment of Learning

• Informed judgments about achievement of intended
  learning outcomes
• Based on evidence of what students can do and
  their capacity to apply what they know (data) at
  the completion of the learning experience
  (course, certificate, program, or other
  credential)
• Valid, reliable, and fair
• Built into the plan for teaching
• Data-driven evaluation that is actively used for
  continual improvement of teaching and learning

5
Sound Assessment

• Fair
• Valid
• Reliable
• Learners informed of expectations up front!
• Feedback to learners

• Reliable
• Performance assessment based on consistent
  rubrics, scoring guides, and rating scales
• Consistent process - each learner is assessed in
  same way as other learners

• Valid
• Outcomes based on standards (industry)
• Measures intended outcomes
• Measures application and critical thinking

6
Summative/Formative Assessment
Formative Assessment
Summative Assessment
Continuous Improvement
7
What are the drivers?

• Learning continuous improvement of teaching and
  learning
• Accreditation (NCA, AQIP, Industry)
• Carl Perkins IV

8
NCA Higher Learning Commission

•  
• HLC/AQIP asks 5 fundamental questions
• How are your stated learning outcomes appropriate
  to your mission, programs, students and degrees?
• How do you ensure shared responsibility for
  student learning assessment of student
  learning?
• What evidence do you have that students achieve
  your stated learning outcomes?
• In what ways do you analyze and use evidence of
  student learning?
• How do you evaluate and improve the effectiveness
  of your efforts to assess and improve student
  learning?

From Student Learning, Assessment and
Accreditation Criteria and Contexts, presented
at Making a Difference in Student Learning
Assessment as a Core Strategy, a workshop from
the Higher Learning Commission, July 26-28,
2006. 
9
Carl Perkins IV The New Law
The What Both secondary and postsecondary must
develop a process for measuring technical skills
that lead to an industry-recognized credential,
or state-recognized credential, or
locally-developed credential The Why Both
secondary and postsecondary education systems
must provide, as a separate measure of
accountability, the extent of skill development
in Career and Technical Education (CTE)
programs The How Both secondary and
postsecondary education must develop valid and
reliable assessments
10
Carl Perkins IV The New Law

• Postsecondary Indicators
• Technical Skill Attainment
• Credential, Certificate, or Diploma
• Student Retention or Transfer
• Student Placement
• Nontraditional Participation
• Nontraditional Completion

11
Carl Perkins IV The New Law
WTCS will move toward a balance of silver and
gold assessment of learning outcomes.
12
Carl Perkins IV The New Law
Summary of Standards for Assessment of Technical
Skill Attainment

• Gold Standard (ex. NCLEX, ASE, Barber/Cosmetology)
  
• External, third-party assessment that objectively
  measures student attainment of industry
  recognized skills upon graduation.
• Silver Standard
• WTCS approved assessment that objectively
  measures student attainment of industry
  recognized skills upon graduation.
• May be developed and/or implemented by the state
  system.
• May be developed and/or implemented locally
  according to system guidelines and with industry
  (advisory committee) approval.
• Must be valid and reliable.
• Bronze
• Non-assessment related indicators such as
  indirect measures.
• GPA and/or Course Completion
• Program Completion Graduation
• Teacher-developed exams that do not meet silver
  standard (not externally approved).

DRAFT
13
Curriculum

• Plan
• Establish outcomes
• Build assessments
• Design learning

• DO
• Deliver
• Assess
• Document

• ACT
• Adjust design
• Adjust delivery
• Adjust assessment

Assessment Process and Curriculum

• CHECK
• Analyze assessment data
• Propose improvements

14
The Learning Assessment Design Technology Tool
Kit
People Soft Banner Datatel
TracDat Blackboard
iWebfolio eFolio
Questionmark
15
WIDS Learning Design System populates assessment,
electronic portfolio, enterprise, and assessment
systems with learning outcomes and performance
standards.

• WIDS
• Establish Learning Outcomes
• Build Assessments
• Design and Plan Learning

• Assessment Software
• (ex. Questionmark)
• Author Q A Assessments and Surveys
• Deliver Assessments/Surveys
• Record Learners Performance (In terms of outcomes)

Learning Outcomes

• Electronic Portfolio
• (ex. iWebfolio/MSNCU eFolio)
• Present evidence of individual learning/profession
  al experience
• Document connection between artifacts and
  outcomes
• Owned by the student

Learning Outcomes Rubrics
Learning Outcomes
Learning Outcomes Curriculum Documents
Continuous Improvement Feedback
Student Learning Documentation
Web-based Electronic Portfolio System presents
evidence of individuals learning and
professional experience.

• Enterprise System
• (ex. PeopleSoft DataTel or Banner)
• Manage Student Records
• Manage Enrollment/Scheduling
• Manage Budget
• Etc.

• Assessment Mgt Software
• (ex. TracDat)
• Describe Outcomes (Learning/Org)
• Build Assessment Plans
• Track Continual Improvement Activities
• Document/Report Results

Assessment Management Software provides feedback
loop for continual improvement of learning design
using WIDS.
Student Data
16
WIDS Model

• Who are the learners?

• What do they need to be able to achieve?

• How will we know and show when theyve achieved
  it?

• How will they get there?

17
Integrating the Outcomes
External Standards (Research-based)
Job Task Analysis (DACUM)
External
Exit Learning Outcomes
Program-Level
Program Outcomes (summative assessment)
Core Abilities (summative assessment)
Gen Ed Outcomes (summative assessment)
Linked Competencies (performance assessments)
Linked Competencies (performance assessments)
Linked Competencies (performance assessments)
Course-Level
18
Dental Hygiene Example
ADA Dental Hygiene Education Standard
Graduates are competent in applying ethical,
legal and regulatory concepts to the provision
and/or support of oral health care services.
External
Dental Hygiene Program Outcome
Incorporate into dental hygiene practice
professional laws, regulations and policies
established by the licensing state and regulatory
agencies
Program-Level
Dental Hygiene Competency
Respond to a request to perform a task that is
not legally permitted to be delegated to a dental
hygienist.
Course-Level
19
Challenges

• How do we communicate WTCS long-standing
  commitment to industry input and standards?
• How do we continue WTCS commitment to performance
  assessment whenever possible (versus testing?)
• How do we ensure that we use quality assessments?
  (That we dont regress to testing regurgitation
  of information?)
• How do we ensure that we are doing what our model
  and assessment plans imply?

20
Design. Document. Align
Using WIDS Program Design

• Create Program Design files for all programs
• Develop Exit Learning Outcomes
• Core Abilities (mission level)
• Program Outcomes (discipline level)
• General Education Outcomes (Optional)
• Determine summative assessments
• Define criteria for summative assessments
• Link courses to exit learning outcomes

21
Design. Document. Align
Using WIDS Learning Design

• Create Performance Assessment Tasks (with scoring
  guides and rubrics) to support summative
  assessment
• Embed in program courses, using the program file
  as the master file
• Create a separate Summative Assessment course
  file for program

22
Design. Document. Align
Using WIDS Analyzer

• Generate Analyzer reports
• Learning Outcomes Matrix
• Program Course List Matrix
• Assessment Task Outcomes Matrix

23
Learning Outcomes Matrix
24
Assessment Task Outcomes Matrix
25
Maximize the potential of WIDS to support
assessment planning

• Create/maintain Program Outcome Summaries for all
  new and existing programs.
• Develop/maintain Course Outcome Summaries for all
  courses.
• Design Performance Assessment Tasks for course
  level (formative) and beyond-the-course
  (summative) assessments.
• Use WIDS Analyzer to build Learning Outcome
  Matrices that document how outcomes are linked
  with programs and courses.
• Present WIDS documentation as a central feature
  in accreditation.