The CMS moodle: A Heuristic Evaluation

1
The CMS moodle A Heuristic Evaluation

• Jay Melton
• Prefectural University of Kumamoto
• jay_at_pu-kumamoto.ac.jp

2
Some Definitions

• HCIhuman computer interaction
• also known as CHI (cf. Rozanski Haake, 2003)
• Heuristicsa set of rules designed to solve
  problems (cf. predictive vs. evaluative)
• CMScourse management system
• moodle (Dougiamas, 2004)
• cf. Web CT (2004) Blackboard (2004)
• AKA as content management system
• AKA LMS, learning management system
• Usabilityreally just means making sure that
  something works well (Krug, p. 5, 2000).

3
Evaluation

• Interface evaluation is very important
• (Colace, Santo, Vento, 2002 Iding,
  Auernheimer, Crosby, Klemm, 2002)
• Interfaces should have a basic level of usability
• (Chalk, 2002 Silius, Tervakari, Pohjolainen,
  2003)
• This is especially important for learners of
  another language
• (Wang Beasley, 2002)
• A straightforward evaluation method
• (Preece, Rogers, Sharp, 2002)

4
Nielsens Ten Heuristics

• Visibility of system status
• Match between system and the real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose, and recover from
  errors
• Help and documentation
• (Nielsen, 1994b)

5
Visibility of system status

• Dependent on the browsers signals for status
• Will vary from browser to browser
• User will need to pay attention
• moodle has some signals
• Chat
• Recent activity
• Online Users

6
Match between system and the real world

• Language should be simple, familiar to user
• moodle can be localized in 40 languages (and in
  any combination)

• Often 1 or 2 words used for links
• Note the use of icons

7
User control and freedom

• There are many ways to move out of a particular
  area.

• Top-left corner, hierarchy

• Bottom of each page

• Emoticons (cf. Carter, 2003)

8
Consistency and standards

• moodle has many modules
• This list is growing
• Tested modules maintain consistency
  (forums/journals)

• May be variation, but that is the nature of such
  a robust package

9
Consistency and standards, cont.

• Problem with links

• Navigation is a problem in forums

• Compare with

10
Error prevention

• Registration page
• Only one prevention mechanism located
• Red text
• Clear messages
• Will not allow user to continue

11
Recognition rather than recall

• Should not have to remember what to do
• Should be clear what to do
• Weekly, topic, social outlines
• Activities, Weeks
• Upcoming Events
• Recent Activity

12
(No Transcript)
13
Flexibility and efficiency of use

• Some overlap with previous features
• Many features allow for efficient use
• Activities
• Upcoming events
• Power users may use them
• Jump to
• Search is also useful
• (forums only)

14
Aesthetic and minimalist design

• moodles interface is simple
• Colors basic

15
Help users recognize, diagnose, and recover from
errors

• Users will get messages alerting them to trouble
• Unless they check this, they will not know
• Timeouts were a problem
• These have been fixed

16
Help and documentation

• No help available from the main page
• Users need to go into a section
• Help available in some areas
• Forums
• Not many other places

17
Conclusions

• Good first step in evaluation
• Caveats
• You are only one of the users
• May have different results with other users
• Nielsen (1994a) recommends using 3-5 experts
  for a heuristic evaluation
• Ask your colleagues
• Run a usability test with potential users

18
References

• Blackboard. (2004). Blackboard. Retrieved
  January 31, 2004 from http//blackboard.com/
• Carter, K. A. (2003). Type me how you feel
  Quasi-nonverbal cues in computer-mediated
  communication. ETC, 60(1), 29-39.
• Chalk, P. D. (2002). Evaluating the use of a
  virtual learning environment for teaching aspects
  of HCI. Paper presented at the 7th Annual
  Conference on Innovation and Technology in
  Computer Science Education, Aarhus, Denmark.
• Colace, F., Santo, M. D., Vento, M. (2002).
  Evaluating on-line learning platforms a case
  study. Paper presented at the 36th Hawaii
  International Conference on System Sciences, Big
  Island, Hawaii.
• Dougiamas, M. (2004). moodle (Version 1.3).
  Perth, Australia. Retrieved May 25, 2004 from
  http//moodle.org/
• Iding, M. K., Auernheimer, B., Crosby, M. E.,
  Klemm, E. B. (2002). Guidelines for designing
  evaluations of web-based instructional materials.
  Paper presented at the 36th Hawaii International
  Conference on System Sciences, Big Island,
  Hawaii.
• Krug, S. (2000). Don't make me think! A common
  sense approach to web usability. Indianapolis,
  IN New Riders.

19
References, cont.

• Nielsen, J. (1994a). How to conduct a heuristic
  evaluation. Retrieved January 28, 2004 from
  http//www.useit.com/papers/heuristic/heuristic_ev
  aluation.html
• Nielsen, J. (1994b). Ten usability heuristics.
  Retrieved January 28, 2004 from
  http//www.useit.com/papers/heuristic/heuristic_li
  st.html
• Preece, J., Rogers, Y., Sharp, H. (2002).
  Interaction design Beyond human-computer
  interaction. Hoboken, NJ John Wiley Sons.
• Rozanski, E. P., Haake, A. R. (2003).
  Curriculum and content The many facets of HCI.
  Paper presented at the 4th Conference on
  Information Technology Curriculum on Information
  Technology Education, Lafayette, Indiana, USA.
• Silius, K., Tervakari, A.-M., Pohjolainen, S.
  (2003, 28 June1 July). A multidisciplinary tool
  for the evaluation of usability, pedagogical
  usability, accessibility and informational
  quality of web-based courses. Paper presented at
  the Eleventh International PEG Conference, St.
  Petersburg, Russia.
• Wang, L.-C. C. Beasley, W. (2002). Effects of
  learner control and hypermedia preference on
  cyber-students performance in a Web-based
  learning environment. Journal of Educational
  Multimedia and Hypermedia, 11(1), 71-91.
• WebCT (2004). WebCT. Retrieved January 31, 2004
  from http//webct.com/