SETDA

1
FETC 2006 Research on the Effectiveness of Two
Models of Implementing Educational Technology
SETDA
2
FETC 2006

• Barry Golden, Project Director, Instructional
  Media and Technology Team, WDPI
• Dr. Randall Ryder, Director of Research,
  UW-Milwaukee

3
ESEA NCLB Title II Part D Enhancing Education
through Technology

• Goals of the program
• Improve student achievement through technology
• Assist all students in becoming technologically
  literate by the end of eighth grade
• Integration of technology and curriculum
  development to establish successful
  research-based instructional methods

4
Title II Part D - Ed Tech

• Evaluating States Education Technology Programs
  (ESETP)
• Research Based
• Nine States 15,000,000 10 grants
• Wisconsin 1,600,000
• Does Technology Make A Difference?

5
Model Selection

• Competitive Process
• Model Identification Committee
• Selected from 16 applications

6
Selected Two Models 1

• Information Literacy, Technology, Problem-Based
  Learning Using the Big6 Model
• Technology Enhanced Writing to promote
• Science and Social Studies learning
• using 61 Traits Writing Model
• Control Group

7

• Project Objectives
• Evaluate the effectiveness of the models on
  student achievement and teacher efficacy by
  randomly assigning schools to either one of two
  experimental groups or a control group
• Measure student achievement in multiple ways,
  including observation, standardized achievement
  test score, work sampling and surveys

8
Incentives

• Free Professional Development
• Laptop computer and software per teacher
• Data Projector per teacher
• Opportunity to participate in national research
• 2000 Experimental teachers 1000 Controls
• Control Group Training and equipment in 2006

9
Number of Schools
10
Evaluation Design

• Subjects 7th and 8th Grade Classrooms in rural,
  urban, suburban school districts
• 34 Schools
• 96 Teachers
• 8500 Students
• 7th and 8th grade Science and Social Studies only

11
Demographics
12
Professional Development

• Experimental Groups
• 4 Days of training each year
• 2 Days during the school year each year
• 5 Days for collaboration each year
• Learning communities using BlackBoard
• Weekly dialog, pre/reflections on activities
• Onsite visits by training consultants

13
Instructional Expectations

• Implement at least 2 Big6 units integrating
  technology
• Implement 6 Traits Writing integrating technology
• Participate in ongoing staff development
• Submit Data

14
Data Gathering Methods

• 7th Fall 2004 Achievement 8th Fall 2005
• 8th Spring 2006
• Teacher Observation for fidelity
• Teacher On-Line Survey Instrument (Becker)
• Student On-Line survey
• Student Work Samples assessment

15
Big 6

• 1. Task Definition
• 1.1 Define the information problem
• 1.2 Identify information needed
• 2. Information Seeking Strategies
• 2.1 Determine all possible sources
• 2.2 Select the best sources
• 3. Location and Access
• 3.1 Locate sources (intellectually and
  physically)
• 3.2 Find information within sources
• 4. Use of Information
• 4.1 Engage (e.g., read, hear, view, touch)
• 4.2 Extract relevant information
• 5. Synthesis
• 5.1 Organize from multiple sources
• 5.2 Present the information

16
6 1 Trait Writing

• Idea/Content No paper has value without ideas.
  Ideas should be presented clearly and developed
  fully with reasons, examples and support from a
  text. Insightful development of interesting ideas
  makes a paper great.
• Organization Writing should be organized
  logically and be easy to follow.
• Word Choice Precise and effective word choice
  adds clarity to writing.

17
6 1 Trait Writing

• Sentence Fluency Sentences should be varied and
  effective in sound and in purpose. When read
  aloud, a paper should sound fluent and even
  rhythmic. Short, stilted sentences impair
  readability.
• Voice This is the personality and conviction of
  the writer showing through the words.
• Conventions Spelling, mechanics, usage,
  agreement are all conventions of writing. Too
  many errors in conventions impair readability.
• Presentation This is how the paper looks. Is it
  typed according to a format? Does it have a title
  page? Presentation affects readability.

18
Data Types

• Pre-Post Teacher Training Survey
• Annual Pre-Post Teacher Survey
• Teacher Model Fidelity Surveys (4X/yr.)
• Student Survey (4x/yr.)
• Annual Achievement testing
• Classroom Observations
• Work Sample Analysis (5x/yr.)

19
Evaluation of Student Work Sample

• Process
• Work samples collected on 7 dates throughout the
  2004 2005 and 2005 2006 school years.
• No restriction on type of work sample submitted.
• Teacher selected one average, one excellent
  sample.
• Each work sample scored by researchers and
  assigned a teacher knowledge and cognitive score.
• Each student sample scored and assigned a
  knowledge and cognitive score.

20
Evaluation of Student Work Sample

• Purpose
• Assess teachers level of expectations of
  cognitive and metacognitive engagement.
• Assess students understanding of each task.
• Examine teachers expectations vs. student
  performance.
• Evidence of model utilization.
• Evidence of technology use.

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
COGAVG DATA
0 lacking 1 adequate 2 strong 3 thorough
32
COGSUP DATA
0 lacking 1 adequate 2 strong 3 thorough
33
Work Samples

• Although superior students engaged in more higher
  level thinking in assignment, the quality of
  these assignment were not significantly better
  than average students
• Quality of average students performance in all
  groups improved during the course of the year
• Teachers, regardless of group, focus on lower
  level thinking tasks

34
Work Samples

• Significant progression in the use of technology
  and treatment model
• Teachers consistently assigned tasks that were
  lower in complexity for both the type of
  knowledge and thought processes
• There was no trend indicating greater frequency
  of higher order thinking tasks between
  groups-greater frequency for all groups
• End of year projects were more complex

35
Classroom Observations

• 18 teachers from 11 schools were observed
• Observation rubric measured constructivism, use
  of technology, evidence of treatment model, and
  nature of instruction

36
Constructivism

• No trends of the level of constructivist-like
  instruction
• In most classrooms students sit in desk, engage
  in seatwork, no not engage in defining lesson
  objectives or tasks, do not engage in
  collaborative tasks with other students or the
  teacher

37
Model Fidelity

• 50 of observations revealed no evidence of
  instruction of model-this improved toward end of
  year
• In 10 of the classes there was evidence of the
  model in the form of a visual device
• In 25 of observations, teachers talks about the
  model, 20 of students discuss model
• 5 of teachers discussed or clarified using the
  model elements

38
Model Fidelity

• No evidence that students discuss model with peers

39
Technology Integration

• 75 of teachers do not demonstrate software or
  hardware within the context of the model or how
  to integrate the same when using the model
• 50 of teachers have their own computer in the
  classroom
• When software use was observed, it tended to be
  presentation software

40
Technology Integration

• Use of computers in classroom is split between
  individual work and whole class
  presentationnegligible evidence of pairs or
  small groups using computers
• Only 10 of teachers used scaffolding techniques
  for computer demonstration
• Only 15 of teachers used word processing in
  meaningful lesson

41
Technology Integration

• 20 of teachers used technology for purpose of
  research

42
WiLATA

• Wisconsin
• Learning and Teaching Assessment

43
What Limits the Effect of Technology to Promote
Thinking?

• Ideology--the unfiltered acceptance of knowledge
• Vs.
• Pragmatism--filtering information from within by
  having thoughts guide themselves

44
WiLATA

• COGNITIVE DIMENSION
• METACOGNITIVE DIMENSION

45
Cognitive Dimension
46
Cognitive Dimension RatingAutomatic
Recall Synthesize Analyze Apply Judge Create

• 0Lacking
• 1Minimal
• 2Good
• 3Strong
• 4Thorough

47
Meta-Cognitive Dimension
48
Meta-Cognitive RatingNon- metacognitive
Declarative Procedural Conditional

• 0Lacking
• 1Minimal
• 2Good
• 3Strong
• 4Thorough

49
Hello I am a virus, I am a piece of hereditary
material. I mostly infect things, that us what I
do. My clothing or structure is basically
protein. I can be found in just about everything
on earth. There are many things that I can do.
I mostly invade cells and reproduce to make more
of me. I reproduce by first attaching myself to
the surface of the cell then I invade the cell
and inject myself the virus into the cell. After
I am done with all the hard work I have to copy
the gorgeous me, then release them all to do the
process all over again. I have to admit
wellthere are some good viruses but I am not one
sorry to say. One thing that can prevent these
nasty viruses is a preposterous thing called a
vaccine. Some of my close friends are AIDS,
chicken poxs, the common cold and one of my best
friends is the Mumps. These different kinds of
viruses are classified first by the kind of
organism they infect, their method of
reproduction, their shape and the kind if
hereditary material the have.Well it is time
for me to infect, invade, copy and release my
brake is over so I might see you sorry if I get u
sick it is just my job remember that!
50
(No Transcript)
51
Use of WiLATA

• Training Teachers for 21st Century Learning
• Assess students work with high degree of
  authenticity
• Generate assessment items to reflect range of
  cogntive and metacognitive tasks and content
  standards

52
Next Steps

• Create software program to provide Web based
  student reporting system
• Complete development of teacher training model
• Selection of pilot sites for training and
  implementation
• National rollout

53
Contacts

• Randall Ryder
• randall_at_uwm.edu
• Barry Golden
• Barry.Golden_at_dpi.state.wi.us