Quality Matters: Peer Review of Online Courses - PowerPoint PPT Presentation

1 / 28
About This Presentation
Title:

Quality Matters: Peer Review of Online Courses

Description:

This session will address how faculty and administrators could use the Quality ... I.3 Netiquette expectations with regard to discussions and email communication ... – PowerPoint PPT presentation

Number of Views:60
Avg rating:3.0/5.0
Slides: 29
Provided by: CFr45
Category:

less

Transcript and Presenter's Notes

Title: Quality Matters: Peer Review of Online Courses


1
Using the Quality Matters Rubric to Improve
Online Coursesed bowen, Executive Dean Distance
Learning, Dallas TeleCollege
This session will address how faculty and
administrators could use the Quality Matters
rubric to improve their online course(s).
Participants will become familiar with the rubric
tool developed by the nationally recognized,
FIPSE-funded Quality Matters (QM) project. The QM
rubric provides a research-supported framework
with annotations and examples for applying
quality practices to specific course design
standards.
2
Why Adopt
What's Involved
Implementation Strategies
3
Quality Matters Is a Process A Set of Standards
4
Quality Matters Peer Course Review Process
Faculty Course Developers
Institutions
National Standards Research Literature
Course
Rubric
Faculty Reviewers
Training
Peer Course Review
Feedback
Blueprint for improvement
Instructional Designers
5
Underlying Principles of QM
  • The QM toolset and process are
  • based in national standards of best practice, the
    research literature and instructional design
    principles
  • designed to promote student learning
  • integral to a continuous quality improvement
    process
  • part of a faculty-driven, peer review process
  • Course does not have to be perfect but better
    than just good enough. (Standards met at about
    85 level or better.)

6
Underlying Principles of QM Continued
  • Process designed to ensure all reviewed courses
    will eventually meet expectations
  • Collegial review process, not an evaluation
    process
  • Review team must include an external peer
    reviewer
  • Course faculty or instructor considered part of
    the review team

7
Whats In It For Institutions
  • External validation
  • Strengthen accreditation package
  • Raise QA as a priority activity
  • Access to a sustainable, replicable, scalable QA
    process
  • Inform online course training practices
  • Provide professional development activities

8
Whats In It For Faculty
  • Improve online courses
  • External quality assurance
  • Expand professional community
  • Review other courses and gain new ideas for own
    course
  • Participation useful for professional development
    plan and portfolio
  • Receive 150 for each completed peer course review

9
(No Transcript)
10
Why Adopt
What's Involved
Implementation Strategies
11
Quality MattersPeer Course Review Process
1. Course Selected
2. Trained Review Team Assigned
3. MOUs and Instructor Worksheet
4. Pre-Review Discussion
5. Individuals complete reviews
6. Team Discussion(s)
7. Final Review Report
12
Quality Matters Peer Course ReviewerCertification
Process
Quality Matters Certification
Course Review Experience


Training
  • Name on QM website
  • Use of QM Logo
  • Eligible for Peer Reviewer Pool

Assigned to Peer Review Team
Attend QM Training
Submit Report
Demonstrate Competencies
Trained
Certified
Kane 1/15/05
13
The Rubric
  • Eight general standards
  • Course Overview and Introduction
  • Learning Objectives
  • Assessment and Measurement
  • Resources and Materials
  • Learner Interaction
  • Course Technology
  • Learner Support
  • Accessibility

Key components must align.
14
Key Standards That Must Align
  • Objectives
  • Standard II.1 Measurable outcomes
  • Standard II.2 Content mastery, critical
    thinking, learning skills addressed.
  • Assessment and Measurement
  • Standard III.1 Measures objectives consistent
    with learning activities
  • Learner Interactions and Activities
  • Standard V.1 Help students achieve the
    objectives
  • Course Materials
  • Standard IV.1 Deep and comprehensive enough for
    students to achieve the objective
  • Course Technology
  • Standard VI.1 Tools and media support the
    objectives

15
General Standard ICourse Overview and
Introduction
  • 1.1 Navigational instructions make the
  • organization of the course easy to
  • understand.
  • 1.2 A statement introduces the student to the
  • course and to the structure of the student
  • learning, and, in the case of a hybrid
  • course, clarifies the relationship between
  • the face-to-face and online components.

16
Rubric Scoring
  • Team of three (3) reviewers
  • One score per standard based on team majority
  • Assigned point value not sliding scale

17
The Peer Review Team
  • 3 Faculty Peer Reviewers
  • All must be experienced online instructors
  • All must attend QM training
  • One MUST be external to the course developers
    institution
  • One must be a subject matter expert (SME) The SME
    could also be the external reviewer.
  • AND
  • Faculty Course Developer
  • access to rubric prior to review
  • involved in pre-review discussions
  • consulted during review

18
Faculty Developer
  • Part of the review team.
  • Provides access to the course.
  • Completes Instructor Worksheet
  • Part of the initial team discussion
  • Receives compiled report
  • Returns Faculty Response Form

19
Peer Reviewers
  • Establish Team Calendar
  • Review the course individually
  • Complete the online web review form
  • Discuss review with Team as needed
  • Complete an Exit Interview
  • Receive
  • Recognition as Certified Peer Reviewer.
  • Compensation

20
Team Chair
  • Reviewer Roles Responsibilities plus.
  • Organizes Team calendar
  • Confirm Instructor Worksheet is used
  • Creates draft report from compiled reviews
  • Convenes Team discussions
  • Reviews, edits and submits Team Report
  • Receives
  • Recognition as Certified Peer Reviewer and Chair.
  • Compensation

21
Overall Course Review Results
  • Upon initial review
  • 51 meet expectations
  • 19 do not meet expectations - missing at least
    one essential 3-point element(s)
  • 30 do not meet expectations - missing at least
    one essential 3 point element(s) and a minimum of
    68 points

22
Why Adopt
What's Involved
Implementation Strategies
23
Range of Implementation Options
Formal QM Peer Review
Internal QM Peer Review
Internal Review
Dean Review
Colleague Review
Cost
Self Administered
24
(No Transcript)
25
  • Additional Optional Services and Fees
  • Independent (Subscriber-Run) Peer Reviews
  • Subscribers may manage their own MOL-recognized
    Peer Reviews after training. Institutions are
    responsible for selecting and managing peer
    reviewers per Quality Matters program guidelines
    and for compensating peer reviewers. Subscribers
    are given access to the Quality Matters Rubric
    for the review team and recognition in the
    Quality Matters Registry of courses that meet
    standards. The annual subscriber fee includes
    registration of up to 30 independent reviewed
    courses for Large Institutions, 20 for Mid-Sized
    Institutions, and 10 for Small Institutions.
    (Subscribers pay a 50 per-course registration
    fee for any additional courses.)
  • To be eligible for recognition in the Quality
    Matters Registry, an on-line course must have
    successfully completed a subscriber-run review
    that has met all Quality Matters guidelines
    (which may or may not involve additional costs to
    the Institution).
  • Guidelines include
  • Review managed by a QM-trained Institutional
    Representative (one seat in training included in
    annual subscriber fee)
  • Review completed by a review team of three
    Qualified Quality Matters-Trained Reviewers. Team
    make-up must include a reviewer from outside the
    institution a reviewer trained as a Quality
    Matters Master Reviewer and a reviewer assigned
    as team chair. (A single reviewer may fill one or
    all of these roles.)
  • Reviewers used the Quality Matters materials and
    followed the Quality Matters process (provided
    electronically to subscribers as part of annual
    subscriber fee)
  • Upon successful completion of review and
    submission of documents to MOL, course
    institution has paid 50 per course for each
    registration over those included in annual
    subscriber fee
  • MOL provides access to list of trained Peer
    Reviewers to subscribers. Subscribers must
    negotiate the peer reviewer stipends paid if any.
    (MOL pays 150 per reviewer, 250 to team chair)

26
  • MOL-Managed Peer Reviews
  • MOL will manage the course review process for
    subscribing and non-subscribing institutions on a
    course-by-course basis. This service includes
    Peer reviewers assigned, managed, and paid by
    MOL compiled final report and recognition of a
    course in the Quality Matters Registry upon its
    meeting Quality Matters program review standards.
  • Quality Matters Program Subscribers
    750/course
  • Non-Subscribers
    1,000/course

27
Range of Implementation Options
Formal QM Peer Review
Internal QM Peer Review
Internal Review
Dean Review
Colleague Review
Cost
Self Administered
28
Multiple Uses of QM
  • Guidelines for initial online course development
  • Quality assurance of existing courses
  • Ongoing faculty professional development
  • Institutional reaccredidation packages
  • Formation of distance learning policies
    steering committees
Write a Comment
User Comments (0)
About PowerShow.com