Open Learner Modelling

1
Open Learner Modelling

• Alice Kerly
• alk584_at_bham.ac.uk
• http//postgrad.eee.bham.ac.uk/alk584

2
Outline

• Who am I? What is this study about?
• Open Learner Modelling
• Assessment for Learning
• Examples
• Studies/results
• Parallels between OLM and AfL

3
Introduction

• Who am I?
• My name is Alice Kerly. Im researching for a
  PhD at the University of Birmingham,
  investigating issues in Educational Technology,
  in particular, an assessment and reflection
  technique known as Open Learner Modelling.
• What is this study all about?
• Open Learner Modelling provides a means for
  learners to reflect on what theyve learnt, what
  they know or dont know, and how they can
  progress. This study aims to investigate the
  opinions of teachers and other education
  professionals in an effort to better understand
  how Open Learner Modelling might be used in
  schools.
• Open Learner Modelling as a method for
  facilitating learning fits particularly well with
  current Assessment for Learning (AfL) approaches.
  Therefore, this presentation draws parallels
  between Open Learner Modelling and AfL intentions
  and methods.

4
Open Learner Modelling (1)

• In order to provide adaptive, personalised
  content and feedback to their users, computer
  teaching environments or Adaptive Learning
  Environments construct a learner model. This
  model holds information about the current,
  evolving knowledge, difficulties or
  misconceptions of the individual learners and is
  used in decisions about what personalised
  educational intervention to make. It is not a
  simple score or percentage.
• Learner models have not been commonly accessible
  to the learners being modelled, but it has been
  argued that allowing the learner to see
  representations of their understanding can
  provide opportunities for learner reflection and
  for self-assessment, increasing learners
  awareness of their developing knowledge,
  difficulties and the learning process. Thus,
  open to the learner.

5
Open Learner Modelling (2)

• The aims of Open Learner Modelling (OLM) include
• Promote learner reflection on knowledge and
  understanding
• Help learners plan and/or monitor their learning
• Encourage self-directed learning
• Improve accuracy of the learner model by allowing
  learners to contribute information
• Facilitate collaboration and/or competition
  between learners
• Give learners greater control over their learning
• Using the learner model as an assessment of the
  learner

6
Assessment for Learning

• Following the publication of Inside the Black
  Box 1 and Beyond the Black Box Assessment
  FOR Learning 2 there has been extensive work
  in schools to encourage pupil participation in
  the assessment process
• At its heart, Assessment FOR Learning is a way of
  informing and involving the learners themselves
  in the process of assessmentto enable the pupils
  to learn more effectively through playing an
  active role 3
• Involves pupils being active looking for
  information about how well they are doing and
  what steps to take next
• May involve discussion, one-to-one reviews, self,
  peer or buddy evaluation, using self-evaluative
  tools
• Open Learner Models can support all of these

7
Open Learner Model variety

• Open Learner Models have been developed to
  support learners at all stages through Primary
  School to University in a range of subjects.
  Therefore, there is wide variety in
• presentation methods
• level of detail and complexity shown
• amount and type of functions available to the
  learner
• level of access and control offered to the
  learner
• The following examples illustrate some of the
  features that Open Learner Models can offer to
  their learners. (Not all designs would be
  appropriate for all learners)

8
Open Learner Model examples

• Wandies Magical World of English
• For 7-8 year olds
• Simple display for young learners
• Displays the learner model as coloured (bronze,
  silver, gold, or red for misconceptions) magic
  wands (to match the theme of wizardry used in the
  system)
• Uses colour to indicate the childs level of
  knowledge.
• Explanatory text appears as the mouse is moved
  over a wand.
• Children were keen to collaborate and to help
  each other to achieve gold wands.
• (Mangat, 2005 4)

9
Open Learner Model examples

• Subtraction Master (8-9 year olds)
• Another simple presentation for younger children
• Shows children their level of knowledge
  represented as smiley faces
• The yellow screen shows a childs own level of
  knowledge
• The pink screen shows the childs understanding
  compared to the average knowledge of classmates
• Teachers can view more detail (right, grey
  screen), to see what misconceptions a child holds
• Children understood the meaning of these
  representations, and were generally keen to view
  both their own learner model and the comparison.
• (McKay, 2004 5)

10
Open Learner Model examples

• OLMlets
• Presents a simple overview of knowledge level in
  the various topics and concepts of a module, in
  five formats
• Skill meters are the most commonly used view
• Provides textual descriptions of misconceptions
  a comparison of the learners knowledge of the
  topics with the knowledge of other users and a
  comparison of their knowledge to the knowledge
  level expected by the lecturer, for the current
  stage of the course.
• (Bull, 2005 6)

11
Open Learner Model examples

• UMPTEEN
• A simple OLM showing knowledge level in a number
  of stars
• Students can choose to open their model to their
  instructors and/or peers, in named or anonymous
  form to engage their comparative progress and in
  seeking collaborative partners
• Students have different patterns of sharing their
  OLM, and of viewing the models of peers Some used
  their own model only others found viewing the
  models of peers helpful.
• (Bull, in press 7)

12
Open Learner Model examples

• Flexi-OLM
• presents the learner model data in a choice of
  seven formats (3 shown here)
• Students can edit their learner model if they
  believe it to be inaccurate (e.g. if they have
  had a lecture since last using the system)
• They can also try to persuade the system to
  change the learner model contents, by offering
  evidence of their knowledge state (e.g. by
  answering additional test questions)
• (Mabbott, 2006 8)

Your understanding of the following topics is
excellent Arithmetic operators, Keywords, If
construct, Initialisation Your understanding of
the following topics is moderate Logical
operators Your understanding of the following
topics is somewhat limited Comments, Control of
flow statements, Primitive types, Prefixes,
Expressions, Operator precedence Your
understanding of the following topics is very
limited While loop, scanf, Strings You may have
misconceptions about the following topics
Arrays The system does not have enough data to
assess your understanding of the following
Prototypes, Arguments, Scope, Relational
operators, Functions, printf, Format specifiers,
Type conversion, Macros,
13
Open Learner Model examples

• CALM-System
• Conversational Agent for Learner Modelling
• Allows the learner to compare their own beliefs
  about their understanding with the beliefs held
  by the system calculated from the learners
  answers to questions
• Learners may view pages showing their own belief,
  the systems beliefs, or a comparison page (shown
  here)
• Learners may also answer more questions to update
  their model, or negotiate with the system
  conversational agent about their understanding
• (Kerly, 2006 9)

14
Studies with OLM

• Laboratory trials showed success in engaging
  learner reflection 10
• Learners want to help improve model accuracy 10
  
• Learners are interested to see the contents of
  their learner model 10
• The learner model can change learners views of
  their understanding 9
• Children with a range of abilities could
  understand their open learner model 11

15
Parallels OLM and AfL

• Learners should be helped to develop the
  capacity and the habit of self-reflection so that
  they can increasingly become self-motivating and
  self regulating. 12
• There is nothing special about the techniques we
  can use for involving learners in reviewing in
  the classroom. What is special is the belief that
  this process actually helps learning. 13
• When pupils do acquire such an overview, they
  then become more committed and more effective as
  learners their own assessments become an object
  of discussion with their teachers and with one
  another, and this promotes even further that
  reflection on ones own ideas that is essential
  to good learning. 1
• Metacognition is the process of being aware of
  ones own learning good learners monitor their
  learning and thinking processes through
  self-monitoring. It focuses the pupils
  evaluation on his or her own performance rather
  than in comparison with others, which we know is
  more likely to maintain motivation. 14
• These quotes about AfL are all goals of Open
  Learner Modelling

16
Summary

• Open Learner Modelling offers substantial
  benefits to learners in terms of reflection,
  motivation, planning, engagement and
  understanding
• Learner models can be presented to learners in
  many ways, to suit the age of learners, and the
  purpose of opening the model in the learning
  scenario
• Current Assessment for Learning initiatives
  emphasize the benefits to learners in developing
  self-assessment, reflection and reviewing skills
• For further information about OLM, visit here
• For additional examples of OLMs, visit here

17
References

• 1 Black, P. Wiliam, D.Inside the black box
  raising standards through classroom assessment.
  London King's College London, School of
  Education, 1998. ISBN 1871984688
• 2 The Assessment Reform Group Beyond the Black
  Box. University of Cambridge School of Education,
  Cambridge, 1999 (free download from
  www.assessment-reform-group.org.uk)
• 3 Association for Achievement and Improvement
  through Assessment. Self Assessment. AAIA North
  East Region, 2002 (free download from
  www.aaia.org.uk/pubs.asp)
• 4 Wandies Mangat, 2005
• 5 Subtraction Master - Bull, S. McKay, M.
  (2004). An Open Learner Model for Children and
  Teachers Inspecting Knowledge Level of
  Individuals and Peers, in J.C. Lester, R.M.
  Vicari F. Paraguacu (eds), Intelligent Tutoring
  Systems 7th International Conference,
  Springer-Verlag, Berlin Heidelberg, 646-655.
• 6 OLMlets - Bull, S. Mabbott, A. (2006).
  20000 Inspections of a Domain-Independent Open
  Learner Model with Individual and Comparison
  Views, in M. Ikeda, K. Ashley T-W. Chan (eds),
  Intelligent Tutoring Systems 8th International
  Conference, Springer-Verlag, Berlin Heidelberg,
  422-432.
• 7 Umpteen Bull, in press
• 8 FlexiOLM - Mabbott, A. Bull, S. (2006).
  Student Preferences for Editing, Persuading and
  Negotiating the Open Learner Model, in M. Ikeda,
  K. Ashley T-W. Chan (eds), Intelligent Tutoring
  Systems 8th International Conference,
  Springer-Verlag, Berlin Heidelberg, 481-490.
• 9 Kerly, A., Hall, P. Bull, S. (2006).
  Bringing Chatbots into Education Towards Natural
  Language Negotiation of Open Learner Models, in
  R. Ellis, T. Allen A. Tuson (eds), Applications
  and Innovations in Intelligent Systems XIV -
  Proceedings of AI-2006, 26th SGAI International
  Conference on Innovative Techniques and
  Applications of Artificial Intelligence,
  Springer.
• 10 Bull, S. Pain, H. (1995). 'Did I Say What
  I Think I Said, And Do You Agree With Me?'
  Inspecting and Questioning the Student Model, in
  J. Greer (ed), Proceedings of World Conference on
  Artificial Intelligence and Education, AACE,
  Charlottesville VA, 501-508.
• 11 Bull, S. McKay, M. (2004). An Open Learner
  Model for Children and Teachers Inspecting
  Knowledge Level of Individuals and Peers, in J.C.
  Lester, R.M. Vicari F. Paraguacu (eds),
  Intelligent Tutoring Systems 7th International
  Conference, Springer-Verlag, Berlin Heidelberg,
  646-655.
• 12 QCA Review of assessment arrangements
  Assessment for Learning, 2001
• 13 Sutton, R. Assessment for Learning, RS
  Publications, 1995
• 14 Stobart Gipps Assessment a teachers
  guide to the issues. Hodder Stoughton, 1997