Achievement Improvement Monitor

1
Achievement Improvement Monitor

• Professional Development Workshops, 2007
• Better Learning by Understanding AIM Data

Peter Congdon
2
Introduction

• Who is responsible for the AIM?
• The Victorian Curriculum and Assessment Authority
  (VCAA) is responsible for the development and
  delivery of the AIM program.
• Who is running the professional development
  workshops?
• PD workshops are being delivered by
• Peter Congdon (Kmetrics) and the Australian
  Council for Educational Research (ACER).

3
Introduction

• Today we will cover
• The background on the history and development of
  the AIM
• How to access your results
• How the results are calculated
• A description of the different types of reports
• The additional features and options within the
  data service
• How to work with your results
• How to present results back to your school
  community
• Today I expect that you will leave with
• Confidence in using the data service
• A deeper understanding of your schools results

4
History and Development of the AIM

• The AIM developed out of the LAP
• Between them they have been used to measure and
  report student achievement in Literacy and
  Numeracy since 1995.
• AIM currently measures and reports student
  achievement in Reading, Writing, Spelling,
  Mathematics and Number.
• Students are assessed at Years 3, 5, 7 and 9.
• Reporting is conducted at the student,
  class/group, school, sector and state level.
• Results can be used as both summative and
  diagnostic assessments.

5
History and Development of the AIM

• Test development and reporting procedures involve
  a number of steps.
• Each step is crucial for the production of a
  quality measurement system.
• Steps include
• Construct definition (VELS)
• Item construction
• Trialing
• Psychometric analysis
• Test construction
• Equating
• Test Administration and data capture
• Psychometric analysis
• Reporting

6
Calculating Results, Years 3 5

• Writing
• Teacher Assessed Writing Task
• Centrally Assessed Writing Test
• Language Conventions short answer items
• To be given a score in Writing the student must
  attempt at least the Centrally Assessed Writing
  Test, or both the Teacher Assessed Writing Task
  and the Language Conventions short answer items.

7
Calculating Results, Years 3 5

• Spelling
• Dictation in the Centrally Assessed English Test
• Editing in the Centrally Assessed English Test
• Spelling in the Centrally Assessed Writing Test
• To be given a score in Spelling the student must
  attempt at least two of these components.

8
Calculating Results, Years 3 5

• Mathematics
• Teacher Assessed Mathematics Task
• Centrally Assessed Mathematics Test
• To be given a score in Mathematics the student
  must attempt at least the Centrally Assessed
  Mathematics Test.

9
Calculating Results, Year 7 9

• Writing
• Centrally Assessed Writing Test
• Language Conventions short answer items
• To be given a score in Writing the student must
  attempt at least the Centrally Assessed Writing
  Test.

10
Calculating Results, Year 7 9

• Spelling
• Editing and Short Answer or Multiple Choice items
  in the English Test
• Spelling in the Centrally Assessed Writing Test
• To be given a score in Spelling the student must
  attempt at least the Spelling items in the
  English Test.

11
Calculating Results, Year 7 9

• Mathematics
• Mathematics Test 1
• Mathematics Test 2
• To be given a score in Mathematics the student
  must attempt at least one of these tests.

12
Accessing the AIM Data Service

• Username and Password
• Sent to school principal
• Access is via the Internet
• http//aimds.vcaa.vic.edu.au

13
(No Transcript)
14
VICCOLLEGE

15
(No Transcript)
16
(No Transcript)
17
Options and Features
Extracting data
18
Understanding the Reports

• Report types
• Profiles
• Trends
• Summative
• Diagnostic

19
Box and Whisker Charts
State
Like Schools
School
Reference groups
Focus group
20
Normal Distribution
21
Skewed Distribution
22
Reference Groups

• State
• All students who sat the test
• Like School
• Concept of being able to compare the performance
  of your students to students from a similar home
  background.
• Different for schools in the different sectors
• Government like school groups are based on the
  proportion of students receiving the Education
  Maintenance Allowance (EMA) and the proportion
  for whom a language other than English (LOTE) is
  spoken at home
• Catholic like schools are all other Catholic
  schools
• Independent like schools are not available

23
Like School Groups
24
Reference Standards

• VELS
• Mathematics Level 2
• At Level 2, students model the place value of the
  natural numbers from 0 to 1000. They order
  numbers and count to 1000 by 1s, 10s and 100s.
  Students skip count by 2s, 4s and 5s from 0 to
  100 starting from any natural number. They form
  patterns
• National Benchmarks
• Represent minimum standards of performance, below
  which students will have difficulty progressing
  satisfactorily at school
• Years 3, 5 and 7, in Reading, Writing and
  Mathematics

25
Report 1, Student Level

• Individual student in each domain relative to the
  State and the VELS

26
Report 2, Student Response
27
Report 2, Student Response cont.
28
Report 3, Group Summary
29
Report 4, Short Answer Response
30
Report 5, Writing Summary
31
Report 6, School Summary
32
Report 7, Five Year Trend Data
33
Report 8, Matched School Group Comparison
Note Year 9s were assessed earlier in the year
than at Year 7
34
Report 9, Matched Student Comparison
35
Report 10, Student School Comparison
36
Report 11, Student Ranking Comparison
37
Summary of Reports Available
38
Look at Your Reports

• Report 1
• Can you find students with unexpected achievement
  levels
• Are there any with an unbalanced profile
• Report 6
• How would describe the location of your students
  against the state
• How spread out are your students in the different
  dimensions
• Report 7
• How have your results changed over time
• Consider both the location and the shape of the
  distribution
• Report 8
• Have students at your school shown the growth you
  would expect
• Are there areas you would like to target for
  further investigation
• Report 9
• Are there students with unexpected results.
• Do these data support what you know about these
  students

39
Item Level Diagnostics, Report 2
40
Item Level Diagnostics, Report 2
41
Look at Your Report 2

• Eyeball the per cent corrects Group V State
• Highlight the largest differences
• Where it was harder than expected find the common
  incorrect response
• Look at the test item
• What skill/knowledge is required to respond
  correctly
• What does the incorrect response tell you about
  where your students are at

42
Item level Diagnostics

• By comparing your students success at the item
  level, to that of all other students in the
  state, you can
• Look for relative differences as the test
  progresses
• Did our students run out of steam?
• Were they consistently above, below or similar to
  the rest of the state?
• What does this mean?
• Is language an issue what is being done
• Is testing an issue format, motivation,
  terminology
• Are there items that represent areas of the
  curriculum not yet introduced?
• How does this compare to the rest of the state?
• Are there areas taught, but not as well as
  expected?
• Consider what is happening at other Year levels.

43
Reporting Back

• Summative position, Reports 3 and 6
• Overall location of the students and subgroups
• Shape of the distributions
• Are students being left behind?
• How spread out are they?
• Trends, Report 7
• Are there identifiable factors that may be
  contributing?
• Student level effects
• Teacher level effects
• School level effects
• Growth, Report 8
• Are students maintaining their position relative
  to the state?
• Is the top growing as fast as the bottom relative
  to the state?
• Is growth even across the dimensions?
• Curriculum Mapping
• Item level Diagnostics, Report 2

44
2008 National Literacy and Numeracy Testing

• Outline
• From 2008, the AIM testing program will be
  replaced with a national testing program.
• National Literacy and Numeracy tests will
  involve students in Years 3, 5, 7 and 9 from
  all States and Territories.
• 2008 National Literacy and Numeracy tests will
  be conducted from 13-15 May.
• This provides teachers with the opportunity to
  get the results back and still have the 2nd half
  of the year to work with them.
• In Victoria, students from all Government,
  Catholic and Independent schools
• will participate in the Program. 
• For students, teachers and schools there will
  be little change from the
• arrangements currently in place for AIM tests.
• National tests will offer the same levels of
  support for students with special needs as
  currently exist in the AIM program.

45
2008 National Literacy and Numeracy Testing

• Test Content
• As with AIM tests, National Literacy and Numeracy
  tests will assess
• Reading
• Writing
• Language Conventions (spelling, grammar and
  punctuation), no dictation
• Numeracy Years 7 9 will have both a
  calculator and non calculator section.
• It is anticipated that the format of the 2008
  National Literacy and Numeracy testing materials
  will be similar to existing AIM test materials.
• Also, Teacher Assessed Tasks will not be part of
  the National testing program.

46
2008 National Literacy and Numeracy Testing

• Timetable
• 2008 National tests will be conducted for all
  four year levels on
• Tuesday 13 May Language Conventions and
  Writing
• Wednesday 14 May Reading
• Thursday 15 May Numeracy
• There will be opportunities for catch-up tests
  to be conducted until Friday 23 May, for students
  who did not do the tests on the nominated day.

47
2008 National Literacy and Numeracy Testing

• Reporting
• Results will be used for individual student
  reporting to parents, school
• reporting and aggregate reporting by States and
  Territories.
• Individual student reporting will include a
  comparison against other students at the
  national level.
• National tests will be expected to align with
  the Victorian Essential Learning Standards
  (VELS).

48
2008 National Literacy and Numeracy Testing
For further information As it becomes
available, additional information will be placed
on the VCAA website site www.vcaa.vic.edu.au .
For further information contact the VCAA Help
Desk on freecall 1800 648 637 or email
at vcaa.aim.help_at_edumail.vic.gov.au
49
Questions

• Further assistance is available, please take my
  card, send me an email.
• Copy of the presentation is available at
  www.kmetrics.com.au/AIMpd2007.ppt
• Further professional development opportunities
  can be found at www.acer.edu.au/proflearn/
• Your feed back on the data service and todays
  presentation is welcomed.
• Thank you