UW-ACE Survey Results: Students

1
UW-ACE Survey Results Students Experiences of
Using Technology in Learning

• Vivian Schoner and Katrina Strampel
• Centre for Learning and Teaching through
  Technology (LT3)
• University of Waterloo

2
The Setting Teaching, Technology, and UW-ACE

• Student Encounters with the Blended Learning
  Experience
• Technically Viable? Yes
• Pedagogically Sound? Yes, No, Maybe, Depends
• Survey Research Results

3
The UW-ACE Survey

• We asked students to tell us about their
  experiences with
• Online Learning Activities
• Online Collaborative Learning (small group)
• Technology Function
• Student Satisfaction
• Online Instructor Feedback

4
Students Completing Survey

• Faculty Quest Enrolment Survey
• AHS 1525 6
• Arts 3041 6
• ENG 739 5
• ENV 1414 7
• MATH 3170 4
• SCIENCE 9474 4

5
Comparisons of Scales
6
Technology Function Access Ease of Use
Navigation

• 26. logging in to be
• 27. accessing my courses to be
• 28. navigating around the UW-ACE course pages to
  be
• 29. accessing the course tools/announcements to
  be
• 30. posting, reading, responding to messages in
  the online discussion forum to be
• 31. accessing feedback on submissions to be
• 32. accessing my grades to be
• 33. submitting assignments to be
• 34. accessing quizzes to be
• 35. accessing team or small group resources to
  be

7
Scale Results

• Technology (alpha .953)

8
Technology Accessibility and Ease of Use

• quick access to grades and assignment feedback,
  access to past lecture notes, announcements,
  additional resources
• UW-ACE helped a lot in centralizing my courses
  online
• the fact that you can connect anytime/anywhere
• a safe online environment
• easy to use, helpful, good interactions with
  profs and peers
• I could sign in and have everything at my
  fingertips

9
Appreciation for Technical Potential

• it enhances the learning process by offering
  supplemental resources for course related topics
• it is certainly appropriate to post lecture
  notes and other resources, but the absolute most
  important thing to remember is not to invent uses
  just because the UW-ACE software is there.
  Legitimate uses are great, but online activities
  just for the sake of doing them are frustrating
• it is desirable because it gives students an
  alternative way to find information about the
  course, also allows them to communicate with
  others who need help and who can offer help
• if you have a question, but hate raising your
  hand and asking in front of all your peers, it
  can be asked and answered online
• I believe its an integral part of the
  university to have students using technology
  which will be useful once we are out there in the
  workforce. Developing this skill is essential for
  us to be able to learn to be technologically
  diverse

10
Overall Student Satisfaction Course
Organization Instructions Workload Technical
Support

• 36. the organization of the course content and
  activities?
• 37. the instructions for completing learning
  activities?
• 38. the number of course learning activities
  through the term?
• 39. the distribution of course learning
  activities through the term?
• 40. the technical support?
• 41. the professional look of the course web
  page(s)

11
Scale Results

• Overall Student Satisfaction (alpha .916)

12
Online Instructor Feedback Helpful Informative
Appropriate Timely

• 42. instructor feedback helped me learn and
  understand difficult concepts
• 43. the frequency of instructor feedback met my
  information needs
• 44. the quality of instructor feedback was
  helpful and informative
• 45. the timeliness of instructor feedback was
  appropriate for my needs
• 46. the amount of instructor feedback was
  appropriate for my needs

13
Scale Results

• Online Instructor Feedback (alpha .972)

14
Online Instructor Feedback

• I got assignments back sooner and easier with
  the site
• the professor always kept us informed of news by
  posting announcements frequently on the UW-ACE
  website
• the immediate feedback from quizzes written as
  well as the correct answers is a definite asset
• took too long for marks to be posted
• not being able to see answers on actual quizzes
• grades are not properly recorded
• there was an announcement that marks for an
  important section would be posted which they
  never were

15
Online Learning Activities Learning Support
Features, Concepts, Skills, Applications
Integrated Parts Study Time Performance Level

• 1. Discussion forum activities
• 2. Self-Assessment (un-graded)
• 3. Self-assessment (graded)
• 4. Collaborative/Peer Review submissions
• 5. Individual Submissions
• 6. Lecture notes
• 7. Links to external resources
• 8. Information literacy/library modules
• 9. Lab modules/activities
• 10. helped me learn the new skills and concepts
  for this course
• 11. helped in that it reduced the amount of time
  spent studying text or notes in order to grasp
  the material to be learned
• 12. helped me to better analyze and apply key
  points in the material to be learned
• 13. encouraged me to take responsibility for my
  own learning
• 14. I found that working online improved my
  performance on graded course assessments
• 15. I found that the online component(s) were
  well integrated with the other course components
  not offered online

16
Scale Results

• Online Learning Activities (alpha .965/.935/.919)

17
Online Learning Activities

• Im sure there are situations where it would be
  desirable to use technology to support teaching
  and learning, but that in itself is too broad a
  goal. You should be using technology to solve a
  particular problem, or to compensate for a
  recognized weakness in existing methods. In the
  case of my course, I cant see how UW-ACE has
  added anything to the course. I found the written
  quizzes far more helpful than the online ones,
  and I prefer to read printed course notes rather
  than getting course material from the web (you
  cant highlight/underline/scribble on a webpage)
• it helped me develop the habit of studying for
  the course regularly. It allowed me to learn
  course material through practice questions. A
  chance to do the same quizzes again and again
  which helped me learn and understand course
  concepts better. Unlimited time to do the quiz
• a good way to practice concepts learned in
  class. Particularly I found the fill in the
  blank type exercises to be very helpful
• the information was very complete and organized.
  I feel that the textbook for the course would not
  have been sufficient on its own to learn all the
  course material. The resources offered on UW-ACE
  enhanced my ability to learn the material

18
Online Collaborative Learning Learning
Activities Peer Interaction Reflective
Learning Community

• 16. Team or small group graded assignments
• 17. Whole class discussion activities
• 18. Peer review submissions
• 19. Team or small group un-graded assignments
• 20. Giving/receiving peer feedback
• 21. I was able to develop a better understanding
  of the material because of my interaction with my
  peers
• 22. I put more thought into my comments when
  interacting with others online
• 23. I felt part of a learning community when
  working online
• 24. I found that working online encouraged me to
  take responsibility for my group contribution
• 25. I am more comfortable asking questions to
  members of my group in an online setting

19
Scale Results

• Online Collaborative Learning (alpha
  .961/.964/.895)

20
Online Collaborative Learning Benefits

• the group discussion page made it a lot easier
  to communicate with my group members since we are
  all in different classes and communication for
  projects is an issue
• I dont mind the discussions online, Im more
  comfortable since it gives the sense of anonymity
  (privacy thing)
• It depends on what type of activities I think
  group tutorials are a good way to learn, whereas
  group projects are hard to realize
• they allow me to communicate with others,
  evaluate their thoughts and compare them to my
  own
• the addition would benefit learning because it
  would provide a place for quick review and
  knowledge testing as well as the style of the
  lecturers questions

21
Construct Validity Pearson Correlation Matrix
22
Identifying Patterns

• Science
• Technology (85) and Student Satisfaction
  (77) 81
• Instructor Feedback (64), Learning Activity (57),
  Collaborative Learning (53) 57
• Applied Health Sciences
• Technology (82) and Student Satisfaction
  (77) 80
• Instructor Feedback (67), Learning Activity (58),
  Collaborative Learning (53) 59
• Environmental Studies
• Technology (83) and Student Satisfaction
  (74) 79
• Learning Activity (62), Instructor Feedback (55),
  Collaborative Learning (49) 55
• Arts
• Instructor Feedback (72), Technology (83),
  Student Satisfaction (74) 76
• Learning Activity (57) and Collaborative Learning
  (54) 56
• Math
• Technology (74) and Student Satisfaction
  (62) 68
• Learning Activity (49), Instructor Feedback (43),
  Collaborative Learning (39) 44
• Engineering
• Technology (67), Student Satisfaction (67),
  Learning Activity (62) 65
• Instructor Feedback (54) and Collaborative
  Learning (49) 52

23
All Faculties, All Scales
24
General Patterns

• Technology function and student satisfaction
  scales represent the more straightforward scales
  group addressing technical support and student
  workload in all Faculties
• Online Learning Activity, Online Collaborative
  Learning (Small Group) and Instructor Feedback
  scales represent the teaching and learning
  pedagogy group in four of the six Faculties
• Exceptions
• Instructor Feedback, a pedagogy scale, grouped
  with technical support and student workload
• Learning Activity, a pedagogy scale, grouped with
  technical support and student workload

25
Faculty Scales
26
Faculty Scales
27
Faculty Scales
28
Survey Results 865 Students 113 Courses 6
Faculties
29
Percent Student Response Ratings by Faculty
30
Overview of Student Commentary

• Students are aware of the benefits of technology
  in the classroom and commented frequently on
  instructors under use of this resource.
• They responded positively to technology use
  perceived to add real value to learning outcomes
  as evidenced in improved performance on quizzes,
  tests, assignments or projects.
• Students provided suggestions for improvements
  and/or described new features they would like to
  see developed for UW-ACE.
• They did not see value-added in working in small
  groups or teams online when they could readily
  interact with their peers in person.

31
Recommendations for Online Pedagogy

• We need to further explore ways to improve
  instructional design and teaching techniques for
  online learning and collaborative learning
  activities. (LT3 Centre interface)
• Some of the needed changes suggested by the
  students should be considered for implementation
  either in the technology itself, using the
  technology more fully to support learning, and
  reducing the ambiguity (undecided) noted in all
  the Scales except Technology Function.

32
Looking Ahead

• We need to think how to reduce the undecided
  rates,
• Rethink providing this as a response choice
• Forced choice is one option
• Use intermediate instead of undecided
• Provide instructors with key information on the
  pedagogy of online teaching and learning
• Encourage instructors to direct their students to
  the online survey available in students course
  pages
• Analysis of the features of individual course
  profiles should be undertaken (Liaisons)

33
THE Capstone Statement

• Learning technologies should be used to
  support, enhance, supplement and/or expand
  learning opportunities Technology should NEVER
  be used to replace teaching and learning in the
  classroom setting.