EDER 677 Telecommunications in Education Web Based Assessment Activities

1
EDER 677 Telecommunications in Education- Web
Based Assessment Activities -

• Norm Vaughan

2
Housekeeping Items

• Second review and final mark for your web based
  student portfolios this weekend
• Questions about final project presentations
  Thursday April 4 and 18

3
Presentation Overview

• This presentation will
• Explore the issues around designing, developing
  and utilizing web based assessment techniques
• Discuss the importance of formative and summative
  program/course evaluation

4
Assessment and Evaluation - Definitions

• Assessment - determine changes in learner
  performance
• Evaluation - make judgments and decisions about
  learners
• Program or course evaluation - appraise the
  success of your program or course

5
Assessment and Evaluation - Activity

• List examples of web based assessment activities
• What types of student learning do these
  activities support (e.g. Blooms Taxonomy
  knowledge, comprehension, application, analysis,
  synthesis, evaluation)?
• Identify issues surrounding the use of web based
  assessment activities

6
Assessment and Evaluation - Activity
7
Triangle of Assessment

• Multidimensional approach to assessment

Knowledge Transfer and Performance
Knowledge Constructs and Reflection
Concrete Knowledge Acquisition
Blooms Taxonomyhttp//www.coun.uvic.ca/learn/pro
gram/hndouts/bloom.html
8
Concrete Knowledge Acquisition

• Concept
• "Factual Foundation Knowledge" - provides the
  ground floor in a learning hierarchy
• Working knowledge or language of a subject
  discipline

9
Concrete Knowledge Acquisition

• Working Knowledge Examples
• Calculus
• recognize and count numbers
• add, subtract, multiply and divide
• Literary criticism and creative writing
• alphabet and basic rules of grammar

10
Concrete Knowledge Acquisition

• Assessment Techniques
• Measuring student mastery of the factual
  foundation knowledge
• Drill and Practice
• multiple choice
• true and false
• matching
• ordering
• short answer

11
Concrete Knowledge Acquisition

• Assessment Ideas
• Preview quiz on preparatory textbook, article
  or online readings
• Review quiz on lecture content
• Collaborative quiz online discussion about the
  questions and answers on a quiz
• Students developing their own quiz questions
  learning how to design tests
• Stronger students in the class writing the
  mid-term and final exam questions receiving
  credit for this as an assignment
• Online quizzes - Integrity test question first
  or last question Did you do this quiz on your
  own?

12
Concrete Knowledge Acquisition

• Web Based Tools
• Hot Potatoes
• http//web.uvic.ca/hrd/halfbaked/
• Quiz Lab
• http//www.funbrain.com/quiz/index.html
• TeAch-nology.com's Crossword Puzzle Maker
• http//www.teach-nology.com/web_tools/crossword/
• Discovery School's PuzzleMaker (Word Searches)
• http//www.puzzlemaker.com/

13
Concrete Knowledge Acquisition

• Web Based Tools Examples
• Macromedia Course Builder
• http//www.macromedia.com/solutions/executive/lear
  ning/center/coursebuilder.html
• Blackboard, WebCT and other course development
  tools

14
Concrete Knowledge Acquisition

• Web Based Resources - Text Book Publishers
• McGraw-Hill Ryerson Canada
• http//www.mheducation.com/
• Nelson-Thomson Learning
• http//school.nelson.com/
• Pearson Education Canada
• http//www.pearsoned.ca/

15
Knowledge Constructs and Reflection

• Concept of Knowledge Constructs
• Process of students constructing and/or
  discovering their own knowledge
• Internalization and integration of new
  information with previously held knowledge

16
Knowledge Constructs and Reflection

• Concept of Reflection
• Students reflecting on the construction of their
  knowledge
• Comparing thoughts and ideas with others or
  against accepted principles

17
Knowledge Constructs and Reflection

• Assessment Techniques
• Structured journals (e.g. responding to specific
  questions)
• Open-ended reflection journals (e.g. responding
  to broad or open questions)
• Collaborative discussions and summaries

18
Knowledge Constructs and Reflection

• Assessment Techniques for Online Discussions
• Establishing discussion guidelines and etiquette
• Developing an assessment rubric for student
  postings
• Tips for online discussion forums

19
Knowledge Constructs and Reflection

• Assessment Examples
• Example One - Class Discussion Scoring Guide
• http//www.tageducation.org/webct/webct/level1/6di
  scussrubric.htm
• a rubric for a 100-point project (60 for
  responses to weekly readings, 40 for responses
  to classmates postings)
• Example Two - Grading Online Discussions
• http//www.oznet.ksu.edu/dp_iet-edtech/TechforTeac
  hing/grading.htm
• various models based on support/clarity/helpfulnes
  s, an A-F scale, and groups grading each other

20
Knowledge Constructs and Reflection

• Assessment Examples
• Example Three - Evaluation and Grading Criteria
  for WebBoard Discussion Assignments
• http//www.ivcc.edu/eng1001/handout_evaluating_dis
  cussions.htm
• an outline of four evaluation criteria for
  discussion assignments
• Example Four - Discussion Guidelines and Grading
  Standards
• http//www.stedwards.edu/ursery/discuss.htm
• discussion grade guidelines on an A to F scale,
  as well as a 10-point guideline for participation
  

21
Knowledge Constructs and Reflection

• Web Based Computer Mediated Communication Tools
• Asynchronous
• E-mail
• File transfers
• Threaded discussion forums
• Synchronous
• Virtual chats
• Shared whiteboards, URLs, and applications
• MUDs(multi-user dimensions) and MOOs (multi-user
  dimension - object oriented)

22
Knowledge Constructs and Reflection

• Web Based Resources
• Using Asynchronous Conferencing to Promote
  Critical Thinking
• http//crlt.indiana.edu/publications/crlt99-8.pdf
• Instructional Design Principles for Teaching in
  Computer Conferencing Environments
• http//www.cvm.tamu.edu/wklemm/instruct.html
• Shaping Cyberspace Into Human Space
• http//css.sfu.ca/update/vol6/6.3-harasim.main.htm
  l
• Handbook for Instructors on the use of Electronic
  Discussions
• http//www.osu.edu/education/ftad/Publications/ele
  cdisc/pages/index.htm
• Check for Plagirismhttp//www.turnitin.com

23
Knowledge Transfer Performance

• Concept
• Performance-based assessment - students
  demonstrating the behavior to be measured
• Authentic assessment - students demonstrating the
  acquisition and mastery of a 'real world' skill
  or concept
• Learners applying knowledge and skills rather
  than simply remembering facts

24
Knowledge Transfer Performance

• Assessment Techniques
• Case Studies (University of Virginia)http//curry
  .edschool.virginia.edu/go/ITcases/home.html
• Creation of web based artifacts (e.g.
  WebQuests)http//www2.mtroyal.ab.ca/nvaughan/web
  questwinter2002.htm
• Portfolios
• http//www2.mtroyal.ab.ca/nvaughan/studentweb200
  2.htm

25
Knowledge Transfer Performance

• Web Based Resources
• Performance Assessment
• Performance Assessment Educational Issues Series
• Strategic Questions - What to Consider When
  Planning for Electronic Portfolios
• Electronic Teaching Portfolios
• Electronic Portfolio Checklist
• Portfolio Assessment
• Student Portfolios Classroom Uses
• Student Portfolios Administrative Uses

26
Assessment - Resources

• Web sites
• Alternative Assessment and Technologyhttp//www.e
  d.gov/databases/ERIC_Digests/ed365312.html
• Assessment Terminology A Glossary of Useful
  Termshttp//www.newhorizons.org/assmtterms.html
• Collected Assessment Resources Center for
  Research on Evaluation, Standards, and Student
  Testinghttp//www.cse.ucla.edu/

27
Program/Course Evaluation - Model
28
Program/Course Evaluation - Definitions

• Formative Evaluation
• ongoing analysis of your program/course design
  during development and implementation
• Summative Evaluation
• conducted to determine whether your course or
  program is worthy of adoption or continuation
• leads to a go/no go decision

29
Program/Course Evaluation - Formative

• On-going process at each stage of the course
  development process
• have colleagues and students constantly reviewing
  the development and implementation of your web
  course site internal review
• develop an evaluation checklist or form
• ensure that each module is piloted
• make the necessary revisions based on informal
  and formal feedback mechanisms

30
Program/Course Evaluation - Formative

• Resources
• Using Midterm Evaluations and Other Sources of
  Student Feedback on Teaching http//www2.mtroyal.
  ab.ca/nvaughan/studentfeedback.htm
• Free Assessment Summary Tool (FAST)
• http//www.getfast.ca
• Zoomerang
• http//www.zoomerang.com/
• SurveyMonkeyhttp//www.surveymonkey.com/

31
Program/Course Evaluation Summative -

• Judging the worth or merit of a program/course
• Conducted after a program/course/design has been
  in place for a complete cycle external review
• Key Questions
• Effectiveness Do students learn or not -
  usefulness?
• Appeal Does the program/course/design engage
  learners and students do they like it?
• Efficiency Is the program/course cost
  effective?

32
Program/Course Evaluation - Resources

• Article
• Shrock, S.A. and Geis, G.L. (1999). Evaluation.
  In H.D. Stolovitch and E.J. Keeps (Eds.) Handbook
  of human performance technology (2nd ed.) (
  Chapter 10 Evaluation, pp. 185-209). San
  Francisco, Jossey-Bass Pfeiffer.
• Web sites
• Finding a Great Evaluative Question The Divining
  Rod of Emotionhttp//www.tltgroup.org/resources/F
  _divining_rod.html
• Evaluation for Distance Educators
• http//www.uidaho.edu/evo/dist4.html

33
To-Do for Next Session

• Contribute to this weeks discussion forum Web
  Based Assessment Techniques 
• Prepare for our next session Module Twelve
  Readings The Future of Telecommunications
• Read the 2 papers by Dr. Bill Hunter
• Review the PowerPoint slides for this session
  (Rory McGreal)