PISA Project on Learning and Teaching Strategies in Schools

1
PISAProject on Learning and Teaching Strategies
in Schools

• A Video Based Classroom Study in
• Mathematics
• Science
• Language Arts

Funded by The Norwegian Research Counsel
2
Research Group

• 3-4 Senior Researchers
• Pedagogy, Science Education, Text Comprehension
  (reading)
• 1 Post Doc
• Science Education
• 4 PhD Students
• Text Comprehension (reading), Science Education,
  Math Education, Pedagogy
• 1-2 Technicians

3

Motivation for PISA

• To pursue problematic PISA findings for
  Norway
• The achievement within all domains (reading,
  mathematics, and science) are near the OECD
  average, and has declined in 2003.
• Relatively high spread in student achievement.
• Increasing gender difference in reading literacy
  in favour of girls.
• A particularly narrow repertoire of learning
  strategies.
• Low level of pressure to achieve, and high level
  of noise and disorder, according to reports from
  both students and principals.

4
Motivation for PISA cont.

• To deepen our understanding of the findings
  from the Norwegian Curriculum Evaluation (R97)
• High pressure on learning activities low
  pressure on learning goals.
• Low and unclear standards for students
  achievements and performances.
• Individual seatwork is the most frequent activity
  in all classrooms.
• Little time is used by teachers for explaining
  concepts and topics.
• Big differences in time allocation and grouping
  procedures between primary level and upper
  secondary level.
• An inclusive and informal climate between
  students and teachers in all classrooms.

5
Focus for PISA

• Study offered learning activities, actions, and
  experienced learning activities, meaning in
  mathematics, science and language arts(reading).

6
Main Research Questions

• How can we understand and interpret the general
  achievement level and patterns in the Norwegian
  PISA results?
• How can we understand the pedagogical processes
  that shape this reality?
• How can we transform some of the PISA findings
  into concrete suggestions for improving Norwegian
  education in the perspective of life-long
  learning?

7
Design

• PISA is inspired by the design and framework
  of Learners Perspective Study. Our data are
• Observations and videorecordings (teacher, class,
  student)
• Interviews (teacher, student)
• Copies of student work

8
Mixed Methods Research Design
PISA
Evaluation of R97

Quantitative research
Qualitative research
PISA
Qualitative research objective
Collect qualitative data
Perform qualitative analysis
Perform quantitative analysis
9
Data Collection

• Variation as sampling strategy
• 6 schools chosen on the basis of demographic
  and pedagogical criteria.
• 3 weeks at each school.
• Recording/observing lessons in mathematics,
  science and language arts.
• Interviews with selected students after each
  observed lesson in mathematics and science.
• 2-3 interviews with teachers in the involved
  subjects during the three week period.

10
Technical Expertise

• A data engineer working at the University of Oslo
  has been in charge of ordering adequate technical
  equipment. This has been done in close
  cooperation with external expertise and us, the
  users.
• He has consulted both sound-experts and
  video-experts.
• He is still connected to the project as a
  technical consultant.

11
Technical Equipment

• A powerful computer workstation with specialised
  software and hardware, recording in MPEG2
  (videoformat).
• This computer has 4 videotracks and 8
  soundtracks. All used tracks are directly
  synchronized with the help of the installed
  software.
• A mixing board to control the sound, and two
  screens to monitor the videofilming.
• Laptops for student interviews.
• 4-5 External harddiscs (250 GB).

12
The Computer Workstation

• The technician monitors
• Video recording
• Sound recording
• Remote controlled camera
• Sound feedback

Photo Ola Sæther
13
Technical Equipment cont.

• Four small video cameras one teacher camera with
  remote control, one focus group camera, one
  whole-class camera and one camera in reserve.
• One handheld video camera.
• One whole-class microphone and bugs on both
  focus students and teacher.
• Audioreceivers and headphones for the observers
  with the opportunity to listen to either the
  teacher or the two focus students during
  recording.

14
Technical Equipment cont.

• Two screens for monitoring the videorecording.
• Four special cables, one for each camera, with a
  socket outlet box for the video signals and the
  electric current. The cables vary in length from
  25 to 50 meter.
• A lot of smaller accessories like stop watches,
  tripods, magic arms, super clamps, sucking discs,
  strips, tape, batteries etc.
• One flightcase for the work station and two other
  flightcases for the rest of the equipment.

15
Info About Data Size

• Recording one lesson with three cameras occupies
  7,5 GB diskspace.
• We have recorded approximately
• -150 video filmed lessons
• - 75 student
  interviews
• - 25 teacher
  interviews
• All together 2,6 TB

16
Data Storage

• We have purchased our own server (capacity 5,6
  TB) which is used exclusively for storing PISA
  data.
• We have bought several external harddiscs (250
  GB), to be used for temporary storing of
  video-data. These harddiscs also makes
  transportation easy. Every workday we would copy
  files from the computer workstation to these
  harddiscs.
• Once or twice a week we would copy files from the
  external harddiscs to the main server, which had
  its own backup system, located at the University
  of Oslo

17
Data Storage cont.

• After double-checking the copying of datafiles
  from the external harddiscs to the server, these
  master-copy files were locked for editing and
  turned into read only.
• Only persons involved in PISA have access to the
  data-material on this server.

18
Data Storage Principles

• Our most important principle throughout the
  process was
• Be very particular about keeping account of
  where the data is stored.
• This was done by elaborating clear and unique
  labels for datafiles and keeping a logbook which
  was updated on a daily basis.
• For safety reasons all the material would always
  be stored in minimum two places, until it was
  safely stored on the main server.

19
Log for video-recording at fieldwork
20
LOG FOR PLACEMENT OF FILE
21
Student Interviews

• The videofiles were copied to an external
  harddisc immediately after the end of a lesson.
• This harddisc was then connected to a laptop for
  the video-stimulated student interview.
• We would mainly use the recordings from the
  teacher camera and the focusgroup camera for this
  student interview.
• The interview was recorded and directly saved on
  a file on the computer workstation.

22
Major Advantages With Our Technical Solution

• No need for time-consuming synchronizing of the
  recordings.
• All the recorded data are directly digitalized.
• Small sized, invisible equipment inside the
  classroom leads to minimal interference and
  disturbance of the lesson.
• Researchers (2) remain seated during class in
  order to cause minimal disturbance.
• Technical crew are situated outside the classroom
  for the same reason.

23
Major Advantages With Our Technical Solution cont.

• Video-stimulated student interviews can be
  carried out shortly after the end of a lesson.
  Recordings from all three cameras are available
  for the interviews.
• Our experience was that students were positively
  motivated by high-tech equipment.
• All datafiles can be easily retrieved by the
  involved researchers.

24
Some Disadvantages

• Rigorous planning of the daily recordings.
• A lot of extra work when teachers were changing
  classroom.
• High demand on cooperation.
• Meticulous daily logbook writing.
• A lot of technical equipment to keep track of.
• Some heavy equipment.
• A lot of cable stretching.

25
A Rough Estimation of Expenses

• Approximately 100 000 Euro for all technical
  equipment.
• Approximately 50 of this was spent on the
  server.
• The other 50 was used on the computer
  workstation, the cameras and the accessories.

26
Remotely controlled teacher camera
27
Focus group camera
28
Wholeclass camera