Measures for assessing the impact of ICT use on attainment

1
Measures for assessing the impact of ICT use on
attainment

• Ian Stevenson
• University of Leeds

2
Structure of talk

• What Impact 2 tells us
• Aims of the Study
• Focus of the Study
• Modelling teaching and learning
• Data Collection
• Measured developed
• Observations based on the study

3
What Impact 2 tells us

• ICT usage can have positive effects on high
  stakes examinations
• Childrens ICT home experience can have a
  significant impact on their attainment
• Difficult to establish a causal link between ICT
  experience and attainment
• Context of learning is significant

4
Aims of the Becta Study

• Building on ImpaCT2 to develop a set of measures
  capable of collecting snapshot data on
  relationship between ICT and attainment
• Examine aspects of resourcing, content, practice
  and other factors that play a part in governing
  the likely impact that ICT may have on attainment.

5
Project Team

• Jeremy Higham (Steering Group)
• Isobel Jenkins (Observer)
• John Monaghan (Steering Group)
• Ian Stevenson (Director)
• John Threlfall (Steering Group)
• Stephen Wilkinson (Observer)

6
Focus of the study

• What ICT-related factors are associated with
  development of
• Curricular achievement?
• ICT skills?
• Personal factors
• Motivation?
• Self-confidence?
• Collaboration?
• Autonomy?

7
Modelling teaching and learning
Outcomes
Purpose
Structure
Conditions
Resources distribution, site policy, curriculum
8
Tools

• leader using ICT
• learners using ICT individually, initiated by the
  leader
• learners using ICT in collaborative tasks
  initiated by the leader
• learners using ICT individually, initiated by
  themselves
• learners using ICT in collaborative tasks
  initiated by themselves
• learners interacting via ICT, initiated by the
  leader
• learners interacting via ICT, initiated by
  themselves
• learners creating using ICT (visual arts, music,
  etc).

9
Organisation of teaching and learning

• leaders lecturing/explanation
• leaders' teamwork
• learners' teamwork
• learners' working individually
• learners' reporting or presenting own material.

10
Roles of leaders and learners

• leader giving information
• leader directing questions and answers to
  reproduce facts
• leader directing conversation
• leader stimulating reflections or other critical
  analysis
• learners directing conversation with peers or
  leaders.

11
Data Collection and Organisation

• Focus on teaching and learning activities
• Observation and interview with activity leaders
• Survey of sites, leaders and learners
• Use relational database for organisation and
  analysis of the data collected

12
Data Collected

• Leeds average LEA (Ofsted)
• 24 sites visited between February and April 2003
  selected to cover location, phase,size, and
  socio-economic background
• 61 activities observed across all KS
• 48 leaders and 50 groups of learners

13
Measures Developed

• Extent to which an activity met its intended
  outcomes (rated 1-6)
• Leaders confidence in using ICT for teaching and
  learning
• Structure of an activity
• Potential for attainment

14
Outcomes achieved?
15
Leaders confidence with ICT

• Score was based on a survey of leaders ICT
  background and usage for teaching, learning, and
  administration
• Three groups of leaders confidence scores
• Group A (11) long experience of teaching, but
  who made comparatively little use of ICT for
  teaching or administration.
• Group B (47) long teaching experience but who
  made extensive use of ICT for teaching or
  administration
• Group C (42) used ICT for all of their
  relatively short teaching career.

16
Structure of teaching and learning

• Activities fell into two main types
• Leader-centred approach
• leader using the ICT
• asking learners questions to check facts or to
  confirm understanding
• Learner-centred approach
• learners using the ICT
• directing the conversation as they worked on the
  task that they were set or chose.
• Leaders asked questions to encourage critical
  reflection or to confirm understanding

17
Potential for Attainment

• Potential for contributing to attainment, based
  on
• Hardware configuration
• Leader confidence score
• Structure of teaching and learning
• Outcome rating for the activity

18
Structure of teaching and learning

• Nearly 90 of activities with highest PA score
  had a learner-centred structure
• Learners had the necessary ICT skills for the
  activity
• Learners had the opportunity for sustained and
  meaningful work using ICT, either on their own or
  with others.

19
Leader Confidence

• 75 of activities with the highest PA score were
  led by members of leader group B, with the rest
  in group C
• Activities had clear outcomes and were carefully
  planned, using ICT appropriately
• Leaders were skilled in engaging learners in
  reflection and analysing their own work with ICT,
  both informally and with whole groups

20
Further Work

• Expand the database to cover a larger range of
  sites
• Use the measures to monitor ICT and attainment
  over a period of time at a variety of sites
• Focus on learners
• Relationship between specific types of attainment
  and ICT applications used in different settings
• Role of ICT experience in non-formal settings for
  developing attainment
• Value-added analysis of learners ICT use and
  their attainment, over a period of several years

21
Understanding ICT and pedagogy

• Mediational tool Purpose, context, meaning
• Activity theory System and structure

Activity Structure
Activity System
22
Structure of Activities
Affordance and Constraint
23
Community and Context

• Tools and Communities
• Rules and ontology
• Pedagogy and structure