Ted J. Branoff, Ph.D.

1
Large Course Redesign of GC120Moving the
Foundations of Graphics Course from a
Face-to-face to a Blended Learning Format
Ted J. Branoff, Ph.D. North Carolina State
University College of Education Department of
Mathematics, Science and Technology Education
2
GC120 - Content
3
Previous Structure of GC120

• Daytime sections taught by full-time faculty
• Met 2 times per week for 110 minutes
• 2-3 sections rotated into a computer lab during
  these meetings
• Night sections taught by part-time faculty
• Met once per week for 3.5 hours
• Typically had their own dedicated computer lab

4
Issues with GC120

• Consistency between sections
• SolidWorks instruction
• Evaluating SolidWorks assignments
• Proportion of sketching and SolidWorks
  assignments
• Access to SolidWorks help outside of class
• Large number of students on wait list (about 130
  each semeter)

5
Large Course Redesign Grant

• Summer 2008 - Submitted proposal for DELTA Large
  Course Redesign Grant.
• Proposed moving all sections of GC120 to a hybrid
  or blended format.
• Proposed more laptop sections.
• Proposed help sessions in Elluminate.
• Proposed an automated SolidWorks grader.

6
Moving Sections to Blended Format

• During the Fall of 2008 content was moved from
  WebCT to Moodle.
• Currently all day-time sections of GC120 are
  blended.
• Continue to train night faculty.

7
New Course Structure

• Daytime sections meet in class once per week
• Answer students questions
• Talk about key concepts for the week
• Check homework
• Online content completed outside of class
• Voiced-Over PowerPoint Presentations
• Voiced-Over SolidWorks Demonstrations
• Videos of an instructor sketching
• Study Guides
• Low-stake Online Assessments in Moodle 10-20
  questions 2 Attempts

8
More Laptop Sections

• Gradually moving to more laptop sections.
• Have renovated Poe 106 to handle 60 students.
• Still have many issues to overcome with student
  owned computers.

9
Online Help Sessions

• Offered some help Elluminate help sessions during
  the Fall 2008 semester for exam reviews and
  project help.
• Students still seem to prefer face-to-face help
  sessions.
• We have started offering face-to-face help
  sessions each semester.

10
Automated SolidWorks Grader

• Began programming in Fall 2008.
• Made some improvements during Fall 2009 with
  departmental funding.
• Still have issues with ease of use for faculty.

11
Fall 2008 Survey

• Prior to moving to Moodle, compared 3 pilot
  sections of blended structure to F2F sections.
• Materials were organized in open web pages.
• Conducted in the 13th week of semester.
• Questions about
• Previous experience with online or hybrid courses
• Instructional preference
• General order when navigating through online
  materials

12
Fall 2008 Results

• Some previous experience with online and blended
  courses.
• Prefer blended over face-to-face.
• Prefer face-to-face over online.
• Students navigated through material in many
  different ways.

13
Fall 2008 Results

• No differences between face-to-face and blended
  sections on the midterm exam.
• Significant difference between the groups for the
  final exam.

14
Moving to Moodle

• We can now look at how students are actually
  navigating through materials.
• What are the best strategies for success?

15
Fall 2009 Study Research Questions

• In what order did students progress through the
  materials related to the textbook?
• What was the typical number of attempts at each
  assessment?
• Did students who attempted all of the online
  assessments perform better on the midterm and
  final exams than students who only attempted a
  few assessments?

16
Participants
GC120 Section Frequency
003 55 34
004 56 34
005 53 32
Total 164 100
17
Participants Academic Year
Year Frequency
Freshmen 4 2
Sophomores 115 70
Juniors 29 18
Seniors 16 10
Total 164 100
18
Participants Academic Major
Academic Major Frequency
Aerospace Engineering 17 10
Civil Engineering/Construction Man. 45 27
Mechanical Engineering 44 27
Other Engineering Majors 29 18
Education 7 4
First Year College 8 5
Other Majors 14 9
Total 164 100
19
Moodle LMS
20
Moodle Lesson Format
21
Voiced-Over Presentations
22
Sketching Videos
23
Results
In what order did students progress through the
materials related to the textbook?
24
Results Order of Navigation
25
Results
What was the typical number of attempts at each
assessment?
26
Results Number of Assessment Attempts
27
Results
Did students who attempted all of the online
assessments perform better on the midterm and
final exams than students who only attempted a
few assessments?
28
Results Midterm Means by Assessment Attempts
29
Results Exam Means by Assessment Attempts
30
Analysis Correlation between Midterm Examand
Online Assessment Attempts
Spearmans Rho Spearmans Rho Assessment Attempts Before Midterm Midterm Exam
Assessment Attempts Before the Midterm Exam Correlation Coefficient 1.000
Assessment Attempts Before the Midterm Exam Sig. (2-tailed) .
Assessment Attempts Before the Midterm Exam N 164
Midterm Exam Correlation Coefficient .233 1.000
Midterm Exam Sig. (2-tailed) .004 .
Midterm Exam N 164 164
Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed)
31
Analysis Correlation between Final Examand
Online Assessment Attempts
Spearmans Rho Spearmans Rho Assessment Attempts Before Final Exam Final Exam
Assessment Attempts Before the Final Exam Correlation Coefficient 1.000
Assessment Attempts Before the Final Exam Sig. (2-tailed) .
Assessment Attempts Before the Final Exam N 164
Final Exam Correlation Coefficient .283 1.000
Final Exam Sig. (2-tailed) .000 .
Final Exam N 164 164
Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed) Correlation is significant at the 0.01 level (2-tailed)
32
Discussion / Conclusions

• While a number of students followed the
  explicitly recommended order of material use
  (i.e., view the streaming video before attempting
  the quiz), many students took alternate
  approaches to usage order.
• In addition, this usage order also changed over
  the course of the semester.

33
Discussion / Conclusions

• The weekly online quiz assessments were only
  worth a total of 10 of students final grade and
  the midterm and final worth collectively 40 of
  the grade.
• It is our assumption that students primarily saw
  the value in the weekly assessments as preparing
  them for the larger summative assessments of the
  midterm and final.

34
Discussion / Conclusions

• The data collected seem to support the conclusion
  that those students who attempted more weekly
  assessments (and/or made use of the streaming
  videos) did better on the midterm and final exams.

35
Discussion / Conclusions

• Probably the most important finding of this study
  is that the logging tools provided in Moodle
  provides a powerful tool for instructors to
  gather and analyze data on how students make use
  of the resources provided online.
• It is now easier to chart student trends and
  performance in a more accurate way than in past
  incarnations of this course.

36
Discussion / Conclusions

• This provides instructors with better information
  to use in the redesign of course materials for
  the future.
• Since the production of the multimedia learning
  resources is a labor-intensive practice, this
  formative data provides valuable evidence as to
  whether such material is being used by students
  and whether it provides real educational value.

37
Future Research

• Develop methods for more fine-grained analysis of
  log data.
• More data points over the semester to better
  understand trends
• Analysis of first versus second tries on quizzes
• Use of SCORM-compliant learning resources in
  conjunction with Moodle that allow richer data
  collection usage (e.g., how long did they view a
  video and how many times did they stop and start
  it)

38

• Questions?