Whither learning design tools: an evaluation of LAMS

1
Whither learning design tools? an evaluation of
LAMS

• Stuart Lee Liz MastermanLearning Technologies
  Group,Oxford University Computing Services
• With acknowledgements toHelen Beetham Sarah
  Knight (JISC)Russell Francis (Oxford University)

2
Overview

• Learning design and LAMS
• Description of the LAMS trial
• Results
• Profile of participants
• Response to designing and running learning
  activities in LAMS
• General reflections
• Related projects in progress

3
What is learning design?

• Potential to improve technology-mediated teaching
  and learning
• Basic principles (Britain 2004)
• Focus on activity
• Sequencing of activities
• Re-use and sharing of activities
• IMS LD specification

4
LAMS and learning design

• Background
• Developed at MacQuarie University, Sydney
• Web-based
• Written in Java, XML, and Flash open source
• IMS LD Level A compliance this year
• Key features
• Focus on collaborative activities, not
  single-learner content and tasks i.e.
  complements VLEs
• Visual authoring environment
• Runtime environment for learners
• Monitoring tool for teachers

5
The LAMS authoring environment
6
The LAMS learning environment
7
The LAMS monitor (i)
Screenshot to follow
8
The LAMS monitor (ii)
9
The JISC LAMS trial

• July 2004-March 2005
• Post-16 education institutions FE, HE, ACL
• Server hosted by SST
• Research question
• Does the use of a learning design tool such as
  LAMS support effective practice in designing for
  learning?
• Acceptable to practitioners?
• Satisfactory experience and outcomes for
  learners?
• Promote reflection and sharing of practice?
• Goal provide recommendations

10
Evaluating the LAMS trial Time-table and
instruments

• 2004 milestones
• July Pre-training questionnaires LAMS training
• October Distribution of design questionnaires
• 2005 milestones
• January Workshop
• March-April Completion of design questionnaires
• End of April Final report
• Continuous activities
• EPED-LAMS mailing list
• Interviews and observations
• LAMS sequences made available by participants

11
Profile of participants

• 40 respondents from 24 institutions and 9
  agencies
• 38 in FE
• 35 in HE
• e-Learning experience
• 29 were experienced users/champions
• Technologies most widely used were
• Web research tool curriculum-specific pages
• VLEs
• Discussion boards
• Learning software, virtual tutorials

12
Results key data collected

• Course information
• Expected benefits of LAMS
• Reflections on the design process
• Reflections on use with students
• Summing up

13
LAMS usage

• 21 participants completed design questionnaires
• Examples of subjects
• Motor vehicle maintenance (FE)
• Adult numeracy (FE special needs)
• Marketing (FE)
• History (ACL, FE)
• Biochemistry (HE)
• e-Learning (ACL, HE)
• Summary of experiences (no. of sequences)
• Planned only 22
• Planned and designed 18
• Planned, designed and ran 14 ? 11
• ? 3

14
LAMS and effective practice Acceptability to
practitioners

• Designing a sequence
• Addition of new activities esp. collaborative
  ones
• Linearity of sequences
• Running a sequence
• Collaborative resource-sharing activities
• Ability to monitor progress
• Linearity of sequences
• Judgements
• 17 out of 18 felt LAMS supported their design
  approach either very well or quite well
• 10 out of 16 anticipated ? workload
• 12 out of 15 likely to use LAMS again

15
LAMS and effective practice The learning
experience outcomes

• The learning experience
• Motivation was main purpose for using LAMS and
  students were motivated
• Ability to work at own pace
• ? Participation in comparison with F2F
• Variable impact on quality of student
  interactions
• Inability to return to discussions
• Lack of feedback on performance
• Preference for F2F over virtual conversations
• Lack of IT skills and/or introduction to LAMS
• Learning outcomes achieved in 12 out of 14 cases

16
LAMS and effective practice Supporting
reflection and sharing

• Not a big issue for participants
• Reflection
• Choice of activities, relationship between them
• Different learning styles, needs
• Sharing and re-use
• 12 out of 15 said re-use is important
• Re-use can save time
• Issues
• Context-dependence
• Transferability across institutions, sectors
• No culture of re-use

17
General reflections (i)

• Support for different pedagogical approaches
• Teachers can choose activities to match their
  preferred approach
• LAMS claims to support collaborative learning
  but
• Participants use of collaborative activities
  varied
• Participants provided limited opportunities for
  learners to contribute resources
• LAMS does not allow teacher to give dynamic
  feedback or draw threads together
• BUT some conversions!
• I think it has made me want to be less
  instructivist.

18
General reflections (ii)

• LAMS as mediator of new types of teaching and
  learning experience
• Criterion experiences must be difficult or
  impossible to achieve with other tools
• Teachers new dimensions
• Combine online research discussion in an
  ordered manner
• Analyse online discussions
• Monitor students progress
• Learners new opportunities (if receptive)
• Review online discussion ? reflection ? revision
  of (mis)conceptions
• Participate anonymously

19
General reflections (iii)

• Conditions conducive to the successful deployment
  of LAMS
• Teachers
• If subject teacher enthusiasm for IT. If IT
  teacher support from subject teachers
• Support from management
• Understanding of what is likely to work well (or
  not) for learners
• Creativity willingness to invest time
• Learners
• Clear relevance of LAMS activities to their
  learning
• Readiness to profit from enhanced opportunities
• Adequate induction into LAMS

20
General reflections (iv)

• Evaluating a tool which is still under
  development
• Balance between fostering interest in learning
  design providing robust product
• LAMS participants stayed on board because
• Experienced users of IT
• Able to discern the potential of LAMS
• Maintained dialogue with each other and LAMS
  development team

21
General reflections (v)

• Supporting participants in a longitudinal trial
• Retention during the trial
• Net attrition rate of 50 normal or not?
• Variety of reasons for failure to complete
• Solutions
• RemunerationFund supply staff
• Ongoing support
• The last thing I want to do is potentially
  excite tutors about this and then say to them
  Well sorry, youre not going to be able to use
  this because we cant offer it.

22
Final Report

• http//www.jisc.ac.uk/index.cfm?nameelp_lams
• Recommendations for JISC servers, continued
  support, new LAMS studies, investigations of this
  nature
• Practitioners using LAMS
• Developers of LAMS
• Developers of LD Tools

23
The Learning Technologies Group, learning design
LAMS

• English Subject Centre Oxford Brookes,
  Leicester (LTSN)
• Re-use of learning objects
• LAMS
• eLISA Greenwich (JISC)
• Distributed e-learning
• LAMS, Moodle
• Learning Design Tools (JISC)
• Processes and representations in effective
  practice
• Generic tools