A Decade of Research on Internet Enhanced Education

1
A Decade of Research on Internet Enhanced
Education
D.A.Kashy, M. Thoennessen, G. Albertelli, E.
Kashy
Supported by
Alfred P. Sloan Foundation
Andrew W. Mellon Foundation
Michigan State University
National Superconducting Cyclotron Laboratory
National Science Foundation
2
Project Timeline

• 1992 Initial implementation of the CAPA System
  Using Telnet, printed individualized
  assignments, quizzes and exams.
• 1995 Web implementation for students.
• 1995-1999 Sloan Foundation support (ALN
  implementation in large on-campus courses)
• 1999-2002 Mellon Foundation support (MSU/TAMU
  Cost and educational effectiveness) CAPA ? Open
  Source
• 1999 Initiation of CAPA LectureOnLine
  collaboration into an open source, experimental
  version of a content course management system
  (LON-CAPA)
• 2000-2005 NSF Information Technology grant for
  research with the experimental LON-CAPA system

3
(No Transcript)
4
(No Transcript)
5
Concerns in Large Lecture Classes

• Impersonal nature of instruction
• Large differences in student preparation
• Achieving and maintaining high standards
• Large human resources required for grading and
  assessment
• Timely recognition of students problems and
  difficulties
• Copying assignments - cheating on exams

6
Computer-Assisted Personalized Approach

• Individualized homework, quizzes, exams, and
  supplementary exercises
• Problems can be numerical or conceptual in nature
  
• Facilitates communication among students and
  between students and teaching staff
• Automatic grading and record keeping
• Online statistical analysis and feedback for both
  the instructor and student

7
Research Topics Encountered

1. Student evaluations of the technology
2. Educational effectiveness in Physics
3. Gender differences in Physics
4. Educational outcomes in Statistics for Psychology
   
5. Student feedback and system development
6. Students teaching students
7. Teaching chemistry
8. Cost effectiveness of technology in education
9. Effectiveness of individualized interactive
   content
10. Educational impact of Web-facilitated cheating

8
Typical results examining student evaluations of
CAPA
9
Evaluations of CAPA by students in Physics for
Scientists and Engineers
CAPA as Tool for Learning/Understanding
10
(No Transcript)
11
(No Transcript)
12
Time-on-task with and without CAPA
13
Evaluations of CAPA effectiveness by students in
Statistics for Psychology on scales from 1-10

• Mean SD Range
• Homework educationally
• effective and motivating 8.48 1.53
  5-10
• Quizzes effective in learning
• and motivating attendance 7.56 2.29
  2-10
• Exam Corrections effective
• in learning material missed 9.20
  1.17 5-10
• on exams

14
(No Transcript)
15
2. Educational Effectiveness in Physics
16
Technology Impact
Prof. Danielewicz (Physics)
17

• Distribution of grades in Introductory Physics
  for Scientists and Engineers
• 1992-1994 w/o CAPA
• 1996 with CAPA

18
3. Gender differences in Physics
19

• Grade distributions for male and female students
  for a two-semester course (Phy231 and Phy232).
• Phy231 was taught the traditional way and in
  Phy232 CAPA was used.

20

• Relative difference between female and male
  performance on exams in Introductory Physics for
  Scientists and Engineers

21
Educational outcomes in Statistics for Psychology
22
Do students need to be computer savvy to succeed?

• Measured Computer Self-efficacy on the first day
  of classes
• Correlation w/Computer
• Self-efficacy
• Final Exam -.16 n.s.
• Homework -.02 n.s.
• Quizzes -.02 n.s.

23
Additional educational outcomes

• Similar to results found in Physics
• The vast majority of students achieve very high
  grades on the homework (e.g., 60 of students are
  above 90)
• Performance on homework predicts success on
  midterm and final exams
• Class attendance is generally high (above 75)
  and correlates positively with performance

24
Student and faculty feedback Impact on system
developmentExample Printing
25
(No Transcript)
26
Web-Only Assignments e-mails For 1 Against
88 PHY183, Fall 98.
Harder to work in front of screen
56
43
Printed HW more accessible
Trouble to print, quality of printing
27
Web access required
18
Work on-line too long
11
Paid for it
7
N 88
Printed HW is a reminder
4
4
Computer Access Problems
Prefer printed copy- No reason
4
0
10
20
30
40
50
Number of Responses
Class Poll, Web-Only Assignment (1 week) For
6 Against 420
27
Students teaching students
28
Cost Effective Use of Undergraduates as Teaching
Assistants

• Members of the Honor College
• An H (next to grade) must be earned for a
  minimum number of some courses
• Get badges to wear in PLC
• The H implies additional achievement in the
  course
• Used in Phy183 course Students Teaching Students
• Agree to have Assignments done 2 or 3 days ahead
  of deadline.
• Write an exit essay on experience

29
Undergraduate T.A.s - I

• The teaching was good experience and I felt that
  through teaching, I learned the material better
  myself.
• I thought the way the honors option was set up
  was great because not only was I helping other
  students I was also helping my self study and
  learn the material better.
• I think, we were, in a way, more approachable
  than the TA's
• I thought it was a good thing to do as an honors
  option because it not only helped me but others
  also.

30
Undergraduate T.A.s - II

• After working in the PLC, I truly believe that
  there is no higher form of learning and
  understanding than teaching someone else.
• For some time, Ive been considering teaching as
  a career, and this has strengthened the
  inclination.
• I learned that everyone learns things
  differently, some need pictures and diagrams,
  others need equations and examples.
• I remember a study that said, you learn 40 of
  what you hear, 50 of what you read, and 90 of
  what yu teach. Helping other students has enabled
  me to help myself in this physics class.

31
(No Transcript)
32
GRADE DISTRIBUTIONS
33
7. Teaching chemistry conceptswith Guy
Ashkenazi, Hebrew University, Jerusalem
34

• Ongoing investigation with 400 chemistry students
  
• Comparison of sections taught using
• traditional methods including in-lecture
  numerical exercises
• lectures cover only concepts
• Results on examination performance
• Numerical problems equivalent
• Significantly higher performance on questions
  involving concepts for Concept-only lecture
  sections

35
8. Cost effectiveness of technology in education
36
Cost Effectiveness Conclusions

• ALN technology can reduce costs when faculty and
  staff are involved in grading and record keeping
  (repetitive, time consuming tasks. Staff reduced
  by 30)
• Broader impact of technology improving the
  students educational experience while
  maintaining current costs.
• TAMU study of Psychology Statistics indicated
  that although larger classes are more cost
  effective, technology did not make an equal
  substitute for much smaller class sections.
• Cost savings are most likely to be in disciplines
  for which large content libraries have been
  developed Without such libraries, however, a
  considerable initial investment of faculty and
  teaching assistant time is necessary.

37
9. Effectiveness of individualized interactive
content
38
(No Transcript)
39
Mathematics
40
Numerical (Typical Algorithmic)
41
Randomly Labeled Numerical (Student 1)
42
Conceptual Textual Principe DArchimède

• 2pt Un pêcheur et sa jeune nièce sont en
  bateau sur un petit étang. Ils portent l'un et
  l'autre un gilet de sauvetage. La nièce tient la
  ficelle d'un gros ballon rempli d'hélium qui
  flotte au-dessus du bateau. Examinez séparément
  chacune des situations présentées ci-dessous et
  indiquez si le niveau d'eau de l'étang M -
  Monte, B - Baisse, R- Reste le même, I-
  Indeterminé. (Si dans le premier cas le niveau
  monte, baisse dans le second cas, et que le
  résultat est indetermié pour les autres cas,
  tapez MBIIII)
• A) Le pêcheur remplit un verre avec de l'eau de
  l'étang et le boit.
• B) Le pêcheur jette l'ancre et la laisse pendre
  verticalement à un pied audessus du fond de
  l'étang.
• C) Le pêcheur se jette lui-même à l'eau et flotte
  sur le dos.
• D) La nièce fait éclater le ballon.
• E) La nièce tombe à l'eau, lâche la ficelle du
  ballon qui s'envole.
• F) Le pêcheur renverse par-dessus bord la boite
  de pêche qui coule au fond de l'étang.

43
ConceptualFunctions GraphedRandomLabels
44
(No Transcript)
45
Applet Text
46
AppletInitial and Final ActionFrames
47
(No Transcript)
48
Covariates included ACT, College GPA, Initial
Physics knowledge (Force Concepts Inventory), and
number of absences
49
Interactive Applet Impact
50
Interactive Exercises with Network Technology

• Goals achieved
• Increased time-on-task
• Timely feedback (to both students and
  instructors)
• More positive collaboration among students and
  between students and instructors
• A greater ability to achieve and maintain high
  standards

51
10. Educational impact of Web-facilitated
cheating
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
Sure Enough Excel spreadsheets!
61
More Individualized Graphics
Version I
Version II
62
What to do?

• Have very large databases of problems
• Design to make cheating more difficult than doing
  the work
• Should we be trying to protect the students from
  cheating themselves
• 
  and/or their parents?

63
(No Transcript)
64
For More Info

• www.loncapa.org
• Informational Conference
• January 17/18 2003
• Truckee Meadows Community College
• Reno, Neveda