MERLOT: The Peer Review Process

1
MERLOT The Peer Review Process

• George Heake

2
Discipline Communities
Info Tech
Biology
Engineering
Psych
Health Sciences
Teacher Ed
Business
Math
Chemistry
Teaching w/Tech
History
Music
World Languages
Physics
3
Editorial Boards

• Support development of discipline communities
  (14)
• Editorial Review Board Members (150)
• Editors/Co-Editors (25)
• Associate and Assistant Editors
• Peer Reviewers (?)

4
Board Qualifications

• Expertise in scholarship of their field
• Excellence in teaching
• Experience in using technology in teaching and
  learning
• Connections to professional organizations
• Experience in conducting peer reviews of online
  learning resources

5
Board Responsibilities

• Expand and manage the collection
• Implement the peer review process
• Post peer reviews
• Recruit and train peer reviewers
• Education and outreach to the community of
  educators

6
MERLOTs Business Discipline

• Accounting
• Business Law
• Economics
• E-commerce
• Finance
• General

• Information Systems
• International Business
• Management
• Marketing

7
TYPES OF MODULES

• Simulations
• Tutorials
• Animations
• Drills Practice

• Quiz/Tests
• Lecture/Presentations
• Collections
• Reference materials

8
The MERLOT Learning Profile

• Title, author and affiliation
• Peer Review Link
• User Comments Link
• Type of learning material
• Location (URL for the module)
• Subject classification
• Description
• Submitter
• Audience

9
Evaluation Process

• Stage 1
• Cursory review to identify worthy modules
• Post triage comments and triage value online
• Stage 2
• Editor assigns worthy materials to reviewers
• Reviewers apply MERLOT standards to write reviews.

10
MERLOT follows the model of peer review of
scholarship
Individual Review 1
Composite Review
Individual Review 2
11
Evaluation Process

• Editor sends review to author for feedback and
  permission to post
• Authors can elect to modify materials and request
  review be modified
• Authors can request module be pulled from the
  repository
• Authors can request 2 letters from MERLOT
  summarizing peer review process and report to 2
  people of their choice.
• Peer review is posted

12
Standard Evaluation Criteria (Strengths
Concerns)
Ease of Use
Potential Effectiveness
Quality of Content
13
Quality of Content

• Current and relevant
• Accurate information
• Clear and concise
• Informed by scholarship
• Completely demonstrates concepts
• Flexibility
• Integrates/summarizes concept well

14
Potential Effectiveness

• Specifies learning objectives
• Identifies prerequisite knowledge
• Is very efficient
• Reinforces concepts progressively
• Builds on prior concepts
• Demonstrates relationships between concepts

15
Ease of Use

• Is easy to use
• Has clear instructions
• Is engaging
• Has visual appeal
• Is Interactive
• Uses effective navigation techniques
• All elements work as intended

16
Star Rating System
Excellent all around
Very good w/few minor concerns (4.0-4.9)
Meets/exceeds
standards with some
significant concerns
(3.0-3.9)
Standards not met, some limited value (2.0-2.9)
Not worth using at all (1.0-1.9)
17
Reviews can be developed online through your
workspace
18
The review can be completed in different time
segments
19
To become a peer reviewer, an individual must

• be an instructor at an institution of higher
  learning
• demonstrate expertise in the discipline
• be recognized for excellence in teaching
• have experience using technology in teaching, and
  
• have participated in the activities of the
  discipline.

20
Standards for Scholarly Work

• Endeavors require high level of discipline
  expertise
• Breaks new ground and is innovative
• Is of significance
• Can be replicated or elaborated upon
• Can be documented
• Has the potential to be peer reviewed