OAT Toolkit for Social Studies

1
OAT Toolkit for Social Studies
Using OAT Grade 5 Score Reports
2
Why analyze data?

• To improve student achievement results, use data
  to focus on a few simple, specific goals.
• Mike Schmoker

3
Data Analysis Cycle
Assessments Classroom, School, District, State
Standards
Standards
Compile Data
Classroom Implementation
Standards
Standards
Analysis, Interpretation and Dialogue
Determine Aligned Resources
Standards
Curriculum Design and Planning for New Learning
and Re-Teaching
Standards
4
Enhanced Class Score Report
Page one is a welcome page. One item to note on
this page is the web address for Ohios
Instructional Management System (IMS)
http//ims.ode.state.oh.us/ode/ims/ On this site
you can access released test items, model
lessons, research on social studies education and
other resources.
5
SUCCESS Web Site

• See the bottom of page 1 for information on the
  SUCCESS web site.

6
Ohio Achievement Test Results

• Compare your class average with district and
  state averages.

7
Performance Levels

• View the percentage of students who achieved each
  performance level.

8
Standard Categories
9
Standard Categories
10
Standard Categories
11
Standard Categories
12
(No Transcript)
13
Now What?
14
Interpreting the Data

• On which standards were your students most
  successful? Least successful?
• On which standards does your class exceed the
  district or state average? Are those items
  clustered around a specific skill or content?
• On which standards does your class fall below the
  district or state average? Are those items
  clustered around a specific skill or content?

15
Interpreting the Data

• Consider the OAT data using three filters
• What does this mean for

• Curriculum?
• Instruction?
• Assessment?

16
Interpreting the Data
17
Interpreting the Data
18
Interpreting the Data

19
Interpreting the Data
Question 25, Grade 5 OAT in Social Studies, 2007
Which newspaper headline would give you
information about why a town is having an
election next week? A. Councilman Quits to Take
Job in Private Industry B. Council Votes to
Increase Property Taxes C. Mayor Takes Oath of
Office D. Crime Rate Rises in City
20
Interpreting the Data
Commentary This multiple-choice question asks
students to use a source (a newspaper headline)
to identify a possible cause-and-effect
relationship to make an inference (a conclusion).
Response A is correct, because if a councilman
quits or resigns from his position, there is an
empty position on the town council, which means
that the town will have an election. Response B
is incorrect because a town council voting for
higher taxes does not involve an election.
Response C is incorrect because a mayor would not
take the oath of office before an election, but
afterwards. Response D is incorrect because
while current officials might not be re-elected
if crime goes up, an election would not occur
solely because of a rise in crime.
21
Discussion Questions

• Which benchmark is being assessed?
• How does this item test understanding of the
  benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance on
  this item?

22
Constructed Response Items
Constructed response items include both short
answer and extended response. Though few in
number, they have a significant impact on the
total score.

• How did your students fare with these items?
• Do they frequently write in the classroom?
• Are they expected to answer questions completely
  and to provide supporting details?
• Do they share examples of thorough responses?

23
Interpreting the Data
24
Interpreting the Data
Question 11, Grade 5 OAT in Social Studies, 2007
In your Answer Document, give one reason English
colonists came to Virginia and settled in
Jamestown. Describe one way their experience
was different from what they had expected. (2
points)
25
Interpreting the Data
Scoring Guidelines for Item 11 (Short
Answer) Exemplar The English who settled in
Jamestown were hoping to get rich quickly by
finding gold. In reality, there was no gold
there, and the only way the colony eventually
prospered was by planting and selling
tobacco. Other correct responses may include but
are not limited to Goals They wanted to live
like gentlemen. They wanted to acquire
land. They wanted to make money for their
joint-stock company. Results Many fell ill from
diseases. Many starved. The land was not
suitable. They had to work hard. At one
point they only survived due to food and water
provided by local American Indians.
26
Interpreting the Data
Scoring Guidelines for Item 11 (Short Answer)
2 point Response includes one explanation why the
English settled in Jamestown AND a correct
description of one way in which reality was
different from settlers expectations. 1
point Response includes one correct explanation
why English colonists settled in Jamestown OR a
correct description of one way their experience
was other than what settlers expected. The
noncredit part of the response may be incorrect,
overly general or vague, or missing. 0 point The
response does meet the criteria to earn one
point. The response indicates inadequate or no
understanding of the task and/or the idea or
concept needed to answer the item. It may only
repeat information given in the test item. The
response may provide an incorrect
solution/response and the provided supportive
information may be totally irrelevant to the
item, or possibly, no other information is shown.
The student may have written on a different
topic or written, I dont know.
27
Questions to Consider

• Which benchmark is being assessed?
• How does this item test understanding of the
  benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance on
  this item?

28
Grade-band Achievement Tests
MYTH The grade five OAT is a test designed to
measure what students have learned in their fifth
grade year.
FACT The grade five OAT test is a Grade-band
test designed to measure what students have
learned in their social studies classes in grades
three, four, and five based on Ohios Academic
Content Standards.
29
Interpreting the Data Grade bands
Question 2, Grade 5 OAT in Social Studies, 2007

• Where the press is free, and every man able to
  read, all is safe.
• Thomas Jefferson, 1816
• Which American document guarantees the freedom
  described above?
• the Articles of Confederation
• the United States Constitution
• the Emancipation Proclamation
• the Declaration of Independence

30
Interpreting the Data
Item 2 Commentary This multiple-choice question
asks students to identify from a list of
documents the one that guarantees the freedom
described in Jeffersons quote (where the press
is free). The United States Constitution
(Response B) is correct because the Constitution
includes the First Amendment which guarantees
freedom of the press, an important freedom in a
democracy. The Articles of Confederation
(Response A) provided a framework for the
government of the original thirteen colonies with
little mention of individual rights. The
Emancipation Proclamation (Response C) is the
document that freed slaves in Confederate states
during the Civil War. The Declaration of
Independence (Response D) was for the most part a
list of grievances which the colonists had and
urged independence from England.
31
Questions to Consider

• How does this item test understanding of the
  benchmark?
• What is the cognitive level of this item?
• What can we learn from student performance on
  this item?

32
Next Steps

• Review alignment of lesson plans and assessments
  with the knowledge and skills explained in the
  benchmark and grade-level indicators.
• Work with third and fourth grade teachers to plan
  together. Start by looking at how each grade
  contributes to understanding of the benchmarks.
  How can we make connections between grade levels?
  
• Review classroom resources and materials and
  identify gaps. Talk to the Media Specialist about
  supplemental texts, Web resources, or
  audio/visual materials.
• Review classroom assessments to ensure that they
  push students to explain their answers and make
  connections between ideas.

33
If we are not going to try to improve what we
do, there is little sense in assessing
it. William Glasser
34
Resources
Glasser, William. (1990) The Quality School
Managing Students Without Coercion. New York
Harper Collins. Mertler, Craig A. Interpreting
Standardized Test Scores, Strategies for
Data-Driven Instructional Decision Making. Sage
Publications, Inc. Thousand Oaks,
2007. Schmoker, M. First Things First
Demystifying Data Analysis. Educational
Leadership, vol.60 n5 p22-24 Feb 2003.