Using data in school inspections

1
Using data in school inspections

• General Introduction
• for group training

Version 6 July 2007
2
Using data in school inspections

• Objectives of these training modules

• To ensure consistent approaches to particular
  issues within our data work
• A particular emphasis on eradicating common
  misconceptions
• To provide materials accessible to all inspectors
• To enjoy working with and learning from colleagues

These are group training materials so they are
not differentiated. That depends on YOUR
discussion! The inspectors also dont represent
real people they are there to encourage
discussion.
3
Using data in school inspections

• Lets just remind ourselves about statistical
  significance and CVA percentiles before we start.
  These figures are for Blowy Down High School

Look at Maths 2006 what does it mean? True or
False game time!
A. Percentile 41 is above average so it was
Good.
?
B. The actual score was above average, so it is
probably Good.
?
C. Progress in maths was definitely weaker than
in previous years.
?
D. We can be confident that maths CVA was
somewhere between 999.6 and 1001.6 last year.
?
E. Progress in maths was either inadequate or
satisfactory in 2006.
?
F. Progress in maths should be an inspection
trail.
?
4
Using data in school inspections

• This is an inspection case study. It looks at
  two inspectors investigating a particular issue
  in a school.
• The point of the exercise is to allow groups to
  consider strengths and weaknesses in the practice
  of the inspectors what they do is not
  necessarily good practice that is for you to
  discuss!

5
Using data in school inspections
My first hypothesis was that this was probably a
good school what do you think?

Windy Ridge High School inspection 14/15 April.
The school has evaluated itself as Good for
Achievement and Standards and, at first, I was
tempted to agree with this judgement. It has a
Key Stage 2-4 CVA figure that is shown in
RAISEonline as significantly above average and
to have a percentile ranking of 10. However, I
noticed that the school has a consistent pattern
of sig throughout RAISEonline and in its
Fischer Family Trust reports quoted in the SEF
it also has some subjects that appear especially
strong English CVA for KS2-3 for example. The
percentile for this is 7, but I am cautious about
placing any great emphasis upon one figure and
concentrate more on KS2-4. The progress of SEN
pupils is also noticeably strong.
Do you think this is an effective analysis?
6
Using data in school inspections

• The lead inspector talks it over with her
  colleague as they arrive to start the inspection

Do you think we should consider whether the
school might be outstanding?
What makes you say that? Its only percentile
rank 10 after all.
Hmmmthey seem a bit unsure! What do you think
they should do? Youve got three minutes to come
up with a strategy for what they should do next!
You can return to previous page to check the info!
7
Using data in school inspections

• What did you decide?

Progress is at least good in all major respects
and is exemplary in significant elements.
I decided it was best to START from the guidance,
Using the Evaluation Schedule. I looked up how it
defined OUTSTANDING.
Dont forget this is all data about the PAST!
What weve already got is
KS2-4 CVA was SIG, percentile rank 10
Most other RAISEonline CVA indicators were SIG
English CVA for KS2-3 was percentile rank 7
SEN progress was SIG in RAISE
Is it possible to make a judgement at this stage?
Are there any warnings you want to give?
8
Using data in school inspections

So I am treating those percentile figures
cautiously! Both are indicated as SIG in our
schools RAISE but I am not going to place too
much emphasis on one figure being slightly higher
than another.
Cynthia is referring to the confidence bars on
the snake graph here is an example
indicated for CVA here by the horizontal bar.
Unless the school is very small, the confidence
interval for percentile rank tends to be only a
few percentage points.
It is also a smaller school, so that 10 might
actually be anywhere between 8 and 12 according
to the Confidence Interval.
9
Using data in school inspections

• The inspectors have a conversation

Lets put Achievement is good overall and
outstanding in some aspects.
Wait a minute, Dave, Im not too sure about this.
Im doing the QA on this inspection, and Im not
sure about Daves suggestions. Can you think of a
few ways in which he MIGHT be wrong?
10
Using data in school inspections

• Has Dave got it right?

Ill have to give Dave some hard messages
laterbut why is he wrong to think it cant be
Outstanding?
Firstly, he is trying to make judgements without
any idea what Achievement and Standards are like
NOW! So he is only thinking about PAST data all
he has is a hypothesis.
He isnt using the SEF though Cynthia HAS
looked at it. He has not yet seen any lessons and
he has not talked to the school about its latest
assessments.
He is treating RAISEonline as a Destination not
a Signpost. Use it to plan direction, not jump to
conclusions.
You should not make judgements like G1 or G2 just
on data this is suggesting that Achievement is
significantly above average but it does not prove
G1 or G2.
Dave is wrong to assume it cannot be Outstanding
it quite possibly MIGHT be!
11
Using data in school inspections

He is wrong to assume it CANNOT be Outstanding.
He has probably decided this because the CVA is
10th percentile. But the school matches the Grade
1 descriptor in respect of its PAST performance
good in major respects, and exemplary in some.
Dave needs to think what a consistently above
average performance tells him. What sort of
signpost is it?
Your signpost is indicating that Achievement was
probably Good or Outstanding last year. So what
should inspectors do in school? Spend a couple of
minutes agreeing your actions.
12
Using data in school inspections

• The inspectors decide to

• Discuss with the school any internal assessments
  showing childrens progress since September

• Look at the questionnaires to see what parents
  think

• Observe the quality of teaching and current
  progress in lessons

• Look in some pupils books to see if there is
  evidence of progress

• Talk to the pupils themselves and perhaps
  examine the schools own pupil questionnaire

Hmmmthat boy doesnt seem to have done any
actual WORK!
13
Using data in school inspections

• The team put all their new evidence about
  Achievement and Standards together. They have
  found that

• Teaching is good in virtually all lessons and
  outstanding in a number of them including
  English this is what the SEF said

• Progress is at least good in all lessons and
  often better and this is consistent with
  pupils books and the schools own assessments

Do we agree with Dave, though?

• Pupils and parents agree that the school sets
  high expectations and they work hard!

Actually, the combination of consistently SIG in
the past and our observations of the present,
plus some outstanding, adds up to Grade 1 to me.
This evidence seems to be supporting a GOOD
judgement!
14
Using data in school inspections

• This case study illustrates several examples of
  good practice eventually! What were they?

Identify some features of GOOD PRACTICE here that
you could pass on to a new inspector!
Top Performer!
15

• Break

Return to Menu Page
16

• This section is about clarifying what is good
  or bad practice when inspecting Achievement and
  Standards.

Its not about Maths, John, it is about using
data to support inspection judgements!
Who wants to be in my team? Im good at Maths!
You will be shown a number of statements about
inspecting using data. Should they be shown the
Red, Amber or the Green Light? Form a team with a
partner and see who wins!
In case this sounds too easy, dont be surprised
if your group leader asks for a REASON!
17
Using RAISEonline

A school is at percentile rank 93 for its Key
Stage 3 CVA in English. Thats Inadequate.
Red Light! You should not reach a 1 4 graded
judgement based on percentile ranking, but it may
well be a clear signpost for your investigations.
Use the SIG- as a signpost for talking to the
school and investigating current progress.
Generally, use KS2-4 but KS2-3 may show you
something about recent trends.
18
Using RAISEonline

Using RAISE data, it is acceptable to judge past
Achievement Standards as broadly below what
might be expected, broadly in line (or not
significant due to limited evidence), or broadly
above expectations.
Pretty Good. Youve identified what SIG- and SIG
mean. However if a figure is neither of these it
MIGHT mean there is too little data. For example,
a very small group of pupils will rarely produce
a Significant figure. By the way, you can also
use other sources of past data.
19
Using RAISEonline

I think RAISE data only tells you about the past,
but it helps you to decide what to see when you
get to the school. If English KS2-3 got
percentile rank 93, then I need to see it. Am I
right?
I agree with this. That percentile rank is a
SIGNPOST, but where has the school gone since? I
will accept the answer but I need to hear some
Inspection Strategies you would use to earn extra
points!
20
Using RAISEonline

• Inspection strategies to check English might
  include
• checking pupils progress and the quality of
  teaching in lessons
• looking at latest internal assessments results
• checking exercise books for progress and
  improving standards
• asking Headteacher how they responded to the
  data, and what has happened since
• and a few more besides..

21
Using RAISEonline

At the end of the day, RAISEonline is the most
important inspection evidence about Achievement
Standards.
RED LIGHT on this, Im afraid. RAISE is certainly
AN important source about PAST performance, but
the school may have other data like FFT which
provides further evidence and a wealth of new
data. Remember also its a signpost but what is
going on in the school NOW?
22
Using RAISEonline
The trailing edge model means your judgement
should be determined by the schools weakest CVA
figures.
This is wrong. You should balance all the
available information to make a professional
judgement. One point for this. But are there any
other factors to consider?
Pay particular attention where there is SIG -
performance for key groups (eg LDD, particular
BME groups) and follow them up on your
inspection.
23
Using RAISEonline

Im planning to use RAISE to identify possible
areas of strength and weakness to investigate in
my lesson observations.
This is a GREEN light! You are using data about
PAST performance as a signpost! You can compare
your observations with the schools internal
assessments and their own monitoring.
24
Using RAISEonline
I think RAISE helps with Capacity to Improve
because it shows if a school is generally
improving or declining.
YES! But can you give some reasons behind it?
It provides clear evidence of IMPACT on
Achievement and Standards - ie progress trends
over time.
You can see whether the school has identified
these trends in the SEF.
You can quiz the school about how it is
responding to these trends during your visit.
You can visit lessons etc to check the schools
responses are having an effect.
Dont forget CVA is RELATIVE so for ABSOLUTE
trends you need to look at the attainment data.
25
Using RAISEonline

All RAISEonline data is valuable to inspectors.
Hmmm.amber. It certainly is an important source
of data about PAST progress but you must take
account of the confidence levels. In very small
schools, or with small groups of pupils, data may
be interesting but it may not be reliable.
26
Using RAISE online
I find the pupil scatter graphs really useful in
a secondary school because it shows you the
pupils who have done very badly at school.
I agree the scatter graphs are useful but
remember they are about INDIVIDUAL pupils so you
cannot rely on single pieces of data! In your
group, think of two good reasons why pupils might
have low scores.
A low-attaining pupil might have had medical or
social problems that kept them away from school.
In a secondary school, pupils may have been
placed on relevant alternative courses that do
not score points.
27
Using RAISE online

A secondary school I visited said KS2-4 CVA is
unfair to them as pupils do well in non-GCSE
courses which are much more suitable to their
needs. I must say I tend to sympathise.
CVA and attainment data does include NVQs.  The
full scope covers GCSEs, up to Level 2
vocationally related qualifications, NVQs, Key
Skills, Basic Skills and Entry level
qualifications.  AS levels and advanced free
standing maths qualifications taken early will
also be counted.
28
Using RAISEonline

A selective school I was inspecting said that CVA
was unfair to them because of the ceiling
effect. I told them this was not correct.
Ohand for the few schools where the ceiling
effect had a large impact, comparisons between
2005 and 2006 CVA should be treated with caution,
as 2005 didnt include this adjustment.
Correct! The ceiling effect was where many of the
pupils had such high prior attainment that it was
impossible for them to improve significantly more
than would be expected. In 2006 improvements were
made to the CVA model to adjust for the ceiling
effect.  Basically, pupils with very high
predicted attainment have this capped, so it is
now possible for all pupils, hence virtually all
schools, to achieve positive CVA scores. 
 Similarly, a floor or minimum prediction is
applied if pupils have very low predictions.
 This ensures that pupils cannot achieve very
high CVA scores despite having almost nothing in
the way of attainment.
29
Using RAISEonline

My turn to ask a question I have got a really
tricky one!
I am inspecting a school tomorrow where some
pupils take GCSE maths a year early. They say
that RAISE gives them an unfair score as a
result. I think theyve got this wrong.
Well they are right! If, by taking the GCSE a
year early, the pupils did less well than they
would have by waiting a year, then CVA would be
lower.  However this can be evidence of the
school promoting progress.  Therefore CVA should
be looked at together with other evidence about
what is going on in the school, in order to get a
well-rounded picture.
30
Using RAISEonline

A school I was inspecting claimed that their CVA
was low because theyd had a high intake
mid-year. I accepted this as a reasonable view
You are right to think we should make allowances
for high rates of mobility which is why CVA is
already adjusted for it. The expected results of
these children would be modified to take account
of the increased challenge faced by the school,
to create a level playing field so you were
WRONG to make an EXTRA allowance for it!
and by the way, because of this CVA for new
schools can be misleading in the first year
especially if they are actually an amalgamation
of schools.
31

• Thats the end of the Traffic Light section
  how did you do?

Return to Menu Page
32
Judging past progress in KS2

• In this module, Cynthia and Dave are
  investigating a junior school Dave has to
  investigate what SIGNPOSTS Key Stage 2 RAISE data
  will give him about pupils past progress. He
  knows now about SIGNPOSTS but Cynthia is
  hoping he will eventually reach a DESTINATION
  his final judgement on Achievement.

I am the lead inspector and Im training Dave.
The school knows this, and they know that I am
making the final judgements.
Im hoping my friends you lot will be able to
help!
33
Judging past progress in KS2

• Which part of RAISEonline should we consult
  FIRST?
• Overall CVA percentile rank charts for the last
  three year
• CVA overall and subjects actual scores and
  SIG/SIG- last 3 years
• Quadrant graphs for last 3 years showing CVA
  versus attainment
• Pupil level scatter graphs (predicted versus
  actual scores)
• CVA graphs for groups
• CVA percentile rank snake graphs for each
  subject last year

34
Judging past progress in KS2

• The best place to start is the CVA overall and
  subjects chart, showing areas of SIG performance.

A little bit of help please what do these
figures show?
35
Judging past progress in KS2
Does this point to a Satisfactory picture?
Bad practice, Dave why?
CVA was 99.4 and percentile rank was 75. There
was no significance indicated.
The actual CVA is 95 likely to be between 98.5
and 100.3 (CVA or confidence) quite a big
range.
With nothing here SIG, the school is unlikely to
be OUTSTANDING.
36
Judging past progress in KS2
OK I have probably ruled out OUTSTANDING, so
what signposts are there here? Two minutes to
help me with some signposts!
The obvious inspection trail is English the
confidence interval is from 99.8 to 97.8 and has
been SIG- the last two years. Why is it so much
weaker than maths and science? I think I will
need to find out what progress is like NOW
because I might be looking at grades 3 or 4.
37
Judging past progress in KS2
Im going to look at the English quadrant graph
to see what else I can learn. Tell me two things!
Dave already knows that Progress was SIG- and
this shows the whiskers below the national
100.
The confidence interval for attainment is very
broad so Dave must be careful not to try to make
judgements on this data alone. Standards were in
line with national expectations nothing else is
proven.
38
Judging past progress in KS2
Does the overall Quadrant Chart give me any
helpful signposts?
The vertical line shows the confidence interval
for PROGRESS overall not significant.
Overall standards are being signposted to ABOVE
AVERAGE, but the Progress position is much less
clear because of English. Dave has plenty of
questions to ask.
The horizontal line shows attainment was SIG.
39
Judging past progress in KS2

Lets not rush what about the trailing edge
model? If English was inadequate, shouldnt
progress be inadequate overall?
I can think of three reasons why Dave is wrong
can you?
This is NOT automatic firstly, the figures are
from months ago. Secondly, you must make a
balanced judgement. Thirdly, CVA data does not
define inadequate remember it only defines
THREE categories. But you SHOULD consider
carefully where a vulnerable group are achieving
less well than others.
40
Judging past progress in KS2
Dave has found the scatter graphs this is the
overall one. What should he do with it? What are
the best uses of these graphs?
They show you patterns of performance for
individuals but in a small school it is not safe
to draw firm conclusions.
Use data like this as a signpost talk to the
school about those two pupils who did much worse
than expected! Does the SEF say anything?
You can also talk to the school about identifying
vulnerable pupils on scatter graphs.
41
Judging past progress in KS2
Can I say boys do worse than girls?
Not proven the whiskers overlap the black
line (which is the national expectation) so
difference is not significant.
42
Judging past progress in KS2
Ah, snake graphs! Perhaps these will help me
resolve my Grade 3 or 4 dilemma.
Does this give you any NEW information?
No! But it does show you in another way that the
school is broadly average overall.
43
Judging past progress in KS2
Agree two messages for Dave from the English
snake graph.
Ive heard its only grade 4 if it is 92 or
below. So they are satisfactory.
The horizontal line shows percentile rank is
roughly 89-91.
Can anyone answer this one?
The vertical whisker is below the 100 line so
progress is significantly below expectations.
44
Judging past progress in KS2

I can answer this one while I finish my coffee!
Can you?

• Firstly, RAISE data alone should NOT be used to
  make individual judgements on the 1-4 scale.

• Secondly, inspectors should not try to define any
  percentile point as a boundary between one
  grade or another the only safe guide is the SIG
  rating.

• Thirdly, Dave has forgotten that he needs to
  balance PAST figures with PRESENT assessment data
  from the school, observation of learning and
  progress in lessons etc English may now be a
  great success story!

Gosh! You lot were a bit slow my coffees
almost gone cold!
45
Judging past progress in KS2

• Cynthia now wants a clear message from Dave
  what signposts has he identified from his work on
  RAISE that will provide direction for the
  inspection?

Performance in English has been below
expectations for two years this points towards
a possible inadequate area.
Maths and science are both broadly in line with
expectations although science was much better
than this two years earlier.
Dave was then asked to check through the SEFs
commentary on these issues.
46
Judging past progress in KS2

• Daves signposts give Cynthia a clear strategy
  for the inspection. What do you think they are?

Focus initial discussions and observation on
English. You need a view on what progress is like
NOW! Do pupils feel things are better?
Investigate the schools own assessment data to
identify are things improving in English, is the
problem linked to any particular class between
year 3 and 6?
Verify current performance in other subjects
not just the core!
Set up a leadership discussion what did the
school itself identify in the SEF, what has it
done, has it been effective?
This information will allow Cynthia to reach her
destination a final judgement.
47
Inspection Data Process Chart

DISCUSS Current AS with leaders Pupils views
of their progress
READ Collect new evidence of how school responded
to RAISE and other evidence, recent progress and
standards data. Schools own judgements.
STUDY Patterns of SIG or SIG- Performance of
particular groups of learners incl most
vulnerable Trends (link to capacity.)
PIB
Fieldwork
Report
RAISEonline
SEF
INVESTIGATE Latest internal standards and
progress assmt data and other forms of
data Parents views SIP reports Other
evaluations, monitoring etc.
OBSERVE Current teaching and learning Progress in
lessons Recent progress in books where appropriate
ANALYSE HYPOTHESISE Significant signposts for
inspection trails Issues to check in SEF and with
school Identify where further evidence required
ANALYSE How does SEF evidence influence
hypotheses? What further evidence is required?
48
Inspection Data Process Chart

RAISEonline Generally SIG in key indicators
Signpost evidence indicates school standards and
achievement were above average overall. Likely to
be good or outstanding, possibly only
satisfactory if there has been a decline.

• Further evidence is required for
• ensuring current progress remains as positive
  and standards high
• ensuring there are no more detailed areas of
  concern, such as progress among small groups of
  vulnerable pupils
• ensuring there are no areas of concern about
  progress across the curriculum
• exploring possibility that AS might be
  outstanding overall

• Methodology
• compare with SEF analysis which may also contain
  additional data
• classroom observation of current progress and
  recent progress in books
• Discussion with headteacher and key managers
• observe internal assessment data, other external
  data, monitoring records etc presented by the
  school
• opinions of parents and pupils

49
Inspection Data Process Chart

RAISEonline Neither SIG nor SIG- key indicators
Signpost not sufficient evidence from the data
alone. These schools are unlikely to be
outstanding or inadequate (although this may
occur with very small schools) , but there may
turn out to be areas of particular note. Signpost
may be to Good or Satisfactory.

• Further evidence is required for
• gathering further evidence about achievement and
  standards to inform judgement
• checking current progress and standards
• ensuring there are no more detailed areas of
  concern, such as progress among small groups of
  vulnerable pupils
• ensuring progress is equally positive across the
  range of the curriculum

• Methodology further evidence is required here as
  RAISE may be inconclusive
• compare with SEF analysis which may also contain
  additional data and indicate areas of improvement
• classroom observation of current progress and
  recent progress from books
• discussion with headteacher and key managers
• observe internal assessment data, other external
  data, monitoring records etc presented by the
  school
• opinions of parents and pupils

50
Inspection Data Process Chart

RAISEonline Generally SIG- for key indicators
Signpost evidence points to a school that was
doing less well than it should have below
average. It is unlikely to be Good or Outstanding.

• Further evidence is required for
• gathering detailed evidence about areas of
  particular strength or weakness
• checking current progress and standards to see
  if there has been an improvement
• ensuring there are no more detailed areas of
  concern, such as progress among small groups of
  vulnerable pupils
• ensuring progress is equally positive across the
  range of the curriculum
• evaluating the schools response to the concerns
  shown in RAISE (linked to capacity issue)

• Methodology
• compare with SEF analysis which may also contain
  additional data and schools response
• classroom observation of current progress and
  recent progress in books
• discussion with headteacher and key managers
• observe internal assessment data, other external
  data, monitoring records etc presented by the
  school
• opinions of parents and pupils
• you will need clear evidence that the leaders
  have secured improvement

51
Challenging Targets

Judging Schools Use of Challenging Targets to
raise standards
52
Challenging Targets

Ofsteds new judgement is How effectively are
challenging targets being used to raise standards
for all learners. Which judgement is NO LONGER
being used?
The judgement no longer being used is How
effectively performance is monitored, evaluated
and improved to meet challenging targets.
53
Challenging Targets

Heres a simple summary to remind everyone of
targets used in schools.
Targets need to be set by 31 December each year
for performance five terms later (e.g. targets
set in autumn 2006 are for performance by pupils
in summer 2008). 
For the second Key Stage Level 4 or above in
National Curriculum tests in English Level 4 or
above in National Curriculum tests in mathematics
Level 5 or above in National Curriculum tests in
English and Level 5 or above in National
Curriculum tests in mathematics
54
Challenging Targets

For the third Key Stage Level 5 or above in
National Curriculum tests in English Level 5 or
above in National Curriculum tests in mathematics
Level 5 or above in National Curriculum tests in
science and Level 5 or above in National
Curriculum tests in ICT

• For Key Stage 4
• the percentage of the relevant group of pupils
  aged 15 to achieve by the end of the following
  school year, grades A-C in five or more subjects
  in GCSE examinations, equivalent vocational
  qualifications or a combination of both
• the average point score for the school to be
  achieved by that group of pupils by the end of
  the following school year in GCSE and vocational
  qualifications.

55
Challenging Targets

Specialist schools have specialist targets. There
are also attendance targets. These dont have the
same status and they dont have to be passed to
the Department in the same way, but they are
important especially so for schools where
attendance is a concern.
Targets also apply to special schools. For
special schools with secondary age pupils, Key
Stage 4 targets include all qualifications (eg
Entry Level, Level 1, Level 2 etc) approved for
use pre-16 under section 96 of the Learning and
Skills Act 2000.
There are no statutory targets for sixth forms,
by the way! Please discuss how this might affect
your inspection
56
Challenging Targets

I did a pilot inspection for this at Battle of
Hastings High School. The head, Harold Godwinson,
told me that the sixth form had no connection to
any judgement on targets. Do you agree?
Well, I think he was wrong. This judgement is NOT
just about statutory targets it is about using
any targets to raise standards at whatever level.
As an inspector, I would hope to see schools
setting themselves challenging but realistic
targets for post-16 performance.
57
Challenging Targets

and the answer is also.Yes!
At Drinsey Nook School, we have some Foundation
Stage children. Do we need targets for them?
Well the answer is ..No
Let me explain.. Although schools are not
required to set Early Years targets, the DCSF
does ask for LA targets and there are also
national Public Service Agreement (PSA) targets
such as 53 of children achieving 6 in all PSED
and CLL scales by 2008.
I think I need a bit of help with this one
58
Challenging Targets

I also need a little more help with this one.
What should I expect from our school?

• LA FS targets are set in
• Children achieving 6 of all PSED and CLL scales
  and 78 points overall
• Gap between the mean of the worst performing 20
  and the median of all children

There is a particular focus on narrowing the
disadvantage gap.
The Department expects that the 2007 Foundation
Stage Profile results will have been effectively
moderated by LAs and reflect a solid basis for
measuring performance changes in subsequent
years.
Although they are not statutory for schools, LAs
need schools to contribute to these targets.
59
Challenging Targets

Schools often have very good data for FS target
setting. If you were inspecting this good school,
what target might you be talking about?
60
Challenging Targets

The schools performance in CLL is rather uneven
so is an obvious area to focus on for
improvements.
61
Challenging Targets

LAs also have targets for the attainment of
looked-after children at the end of key stages 2
and 4.
Goshdoesnt he go on! What I NEED to know is,
what effect do these LA targets have when I am
inspecting Robin Hood Primary School?
62
Challenging Targets

The LA targets will not have a direct influence
on school inspection but they do contribute to an
understanding of the appropriate range of ground
to be covered when discussing Achievement and
Standards.
So, lets think about what this might mean in an
inspection!
Inspectors will need to focus on PROCESSES for
the school is setting itself targets for
of children in the school achieving 6 in all
PSED and CLL scales
of pupils gaining 78 points overall

• The gap between the mean of the worst performing
  20 and the median of all children

63
Challenging Targets

Our headteacher is worried about Floor Targets.
What are they? And does anyone know who is in
charge of them?
They are co-ordinated by the Neighbourhood
Renewal Unit of the DCLG!
Floor targets have been used over a number of
years to indicate the lowest standard considered
acceptable. Like statutory targets, they focus on
end of key stage percentages. This chart shows
the of schools where 65 of pupils reached
Level 4 in KS2.
What connection do you think the concept of
floor targets has with our inspection work?
64
Challenging Targets

Theres a clear link between the aim of floor
targets raising standards for those in
disadvantaged areas and the focus in our
targets judgement on raising standards for ALL
learners.
So discuss among yourselves should a school
with very low standards ALWAYS get a grade 4 for
this new judgement?
Not always, Dave. Are they USING targets to RAISE
standards things may be improving sharply from
a low starting point!
65
Challenging Targets

Lets not discuss WHO is special,
Dave. Specialist schools have to have targets
for attainment and participation in their
specialist subjects and should also have
community targets. What implications does this
have for our inspection?
Speaking as a rather special sort of person
myself, what extra targets do specialist schools
have? Where can I find them?
Well Davina, Im sure everyone understood this is
another opportunity for seeing HOW schools are
USING a range of targets to RAISE standards
generally. Poor use of specialist targets might
indicate a wider weakness in target setting as a
whole. And vice versa.
66
Challenging Targets

Well I now have an understanding of the range of
targets involved and that I should not focus just
on statutory targets. So now I need some help on
what to evaluate.
Lets make this CHALLENGING then, by missing out
some of the key words from the possible areas.
That way youll have to THINK about it!
67
Challenging Targets

Heres some suggestions for what Inspectors
should evaluate that were in the pilot guidance.
It WOULD help you to think about inspection
strategies, but I have cut out some words!
Q1 School trends in XXXXXXXXX for all groups of
learners in recent years in relation to the
XXXXXX that have been set
A1 School trends in standards for all groups of
learners in recent years in relation to the
targets that have been set
Q2 The procedures used to set targets for
learners in the future, so that all may XXXXXXX -
to the benefit of their future XXX.
A2 The procedures used to set targets for
learners in the future, so that all may achieve -
to the benefit of their future economic
well-being.
It is a bit mean not scaling the Xs to the number
of letters.
68
Challenging Targets

I hope these arent too easy!
Q3 The extent to which targets chosen reflect a
XXXX and XXXXX approach to school improvement,
given the XXX of the school.
A3 The extent to which targets chosen reflect a
realistic and challenging approach to school
improvement, given the context of the school.
Q4 An evaluation of whether the setting of XXXX
targets is being managed effectively to help
reduce the proportion of pupils with XXXXXX
attainment.
Q4 An evaluation of whether the setting of
attendance targets is being managed effectively
to help reduce the proportion of pupils with low
attainment.
69
Challenging Targets

Q5 The key question as to whether the targets
being set are contributing to the XXXX
improvement of XXXX.
A5 The key question as to whether the targets
being set are contributing to the consistent
improvement of outcomes.
Q6 Where a school is a XXX school, the use of
specialist targets to benefit the school and its
XXX.
A6 Where a school is a specialist school, the
use of specialist targets to benefit the school
and its community.
70
Challenging Targets

Im inspecting Friar Tuck High School and looking
at the first evaluation area what are the
trends in standards for all learners? Obviously I
will look at RAISEonline.
Why the sudden focus on STANDARDS? I thought we
were all more interested in PROGRESS these days?
We need to be aware that good progress still
leaves some groups with low standards and fewer
chances in life. This is the attainment gap or
disadvantage gap. Closing it is a national
priority from the Foundation Stage Profile
upwards! Also this judgement has a clear focus on
raising standards over time.
71
Challenging Targets

Standards are still very low, but there has been
SOME improvement. Do you think I am safe to rule
out Grade 1 or Grade 4?
APS has improved two years in a row, with a
significant improvement in one year.
However the improvement trend for GCSE higher
grades has NOT been sustained, 1 is little
better and APS has stalled compared to national.
72
Challenging Targets

I looked through the rest of RAISE and found some
interesting PROGRESS data like this. What has it
got to do with my targets judgement?
Well, Dave, it shows clearly that the
improvements you mentioned earlier have only
benefited SOME of the pupils.
This school appears to have focussed its
target-setting efforts at the top end and a large
proportion of pupils are not benefiting.
73
Challenging Targets

Recent performance for 5 has been 27, 33 and 32
in 2006. For 1 it has been 85 and 88. For APS
it has been 189, 222 and 231. What conversation
should I have with senior managers?
I suggest you ask them HOW they arrived at these
targets. Are they built up from actual pupils or
just strategic choices? They appear
challenging, but are they realistic? Targets that
are TOO challenging may not help the school to
improve. What information is there on the prior
attainment of these cohorts?
74
Challenging Targets

Effective targets are BOTTOM UP in other words
they start from pupil level and calculate what
this might mean for the school.
Effective procedures are based on data, often
using published packages that help schools
estimate what pupils might get they can then
add a degree of ASPIRATION on top!
Effective procedures involve staff and
sometimes pupils too in the setting of targets.
Effective procedures involve a degree of
challenge, but keep a focus on what is
achievable. They should be motivational, not
dispiriting!
75
Challenging Targets

Here I am at Robin Hood Primary, looking at
whether targets are challenging and realistic.
Ive got the SEF information and the 2006
RAISEonline attainment data. What would you be
saying or asking?
76
Challenging Targets

I noticed that Level 4 targets are level or below
what was attained last year. However, Level 5
targets look challenging the maths target is to
reach 52 although the school only got 33 last
year! Discuss what the inspection strategy might
be here.
I decided to look at prior attainment data, which
seems to show that the next year (target group
2007) did much better in KS1 than the group who
took SATs in 2006..what do you think?
77
Challenging Targets

I am thinking that Robin Hoods KS2 targets are
too modest at Level 4 and too ambitious at Level
5. Does this evidence help?
78
Challenging Targets

Does this help me with either the modesty of
Level 4 targets or the greater challenge of Level
5 ones?
I think this helps with the Level 5 issue on
the basis of KS1 results, the next cohort has a
greater proportion of very able pupils so Level 5
should rise!
79
Challenging Targets

I am inspecting a specialist school next week and
I have checked all the statutory targets. Do I
need to look at the specialist school ones and
what are they?
Norman, you are right that they are NOT statutory
but we are not just looking at statutory ones,
are we? Specialist school targets can be found in
the SEF but some schools for example special
specialist schools will use a different
approach.
80
Challenging Targets

• Target-setting is being introduced into
  RAISEonline.
• Note the four types of probability columns.
  Should all schools aim for the right hand column
  targets?

81
Challenging Targets

so aspirational targets may not always be
good practice.
Aspirational is certainly a buzz word but
should I be applauding the Chairs approach?
My school may be in a poor area, but we are
aspirational. I am therefore setting targets in
line with the top 25 of schools. We intend to
eliminate SIG- altogether!
We know her school is in a deprived area it
might be more realistic to start with a target
based on SIMILAR schools.
Standards in this school are known to be SIG-
in RAISE. To aim for the top 25 is possibly
unrealistic.
The RAISEonline top 25 is actually the average
for schools in the top quartile so it is really
closer to percentile rank 12.5 a tall order for
her staff to meet!
This approach to target-setting seems very top
down and directive the Chair needs to start
with the children!
82
Challenging Targets

If the Chair, Mrs Sheriff, had asked me I would
have shown her how to build up school targets
from pupil ones, using RAISEonlines new
features! What advice would you give me about
this two levels bit though?
There has been much debate over whether Two
Levels should be the standard target for
progress between key stages. Ofsted does not set
any fixed rules. Different pupils will progress
at different rates.
83
Challenging Targets

• There are other target-setting systems available,
  including Fischer Family Trust. Lets consider
  how FFT might work.

84
Challenging Targets

The data that FFT issue to schools to help them
set targets are called estimates. They estimate
the results that the pupils are likely to achieve
in future years, based on actual progress made by
similar pupils last year. FFT produces estimates
for each pupil and aggregated for the cohort. The
cohort estimates can be used to test the level of
challenge in a schools targets.
FFT give four kinds of estimates
85
Challenging Targets

If you are inspecting a schools use of targets,
you will need to

• Look at the B estimates which are indicators of
  average progress and the D estimates which are
  considered challenging as they are based on
  pupils making progress in line with the best
  25.

• Compare the schools targets with the D estimates
  initially and then with B estimates this will
  give you a view as to how aspirational or
  challenging the targets are

• Look at how targets compare with the schools
  average value added in the last three years how
  far are they aspiring to raise standards?.

• Investigate the extent to which targets seek to
  close any gaps, such as between genders and
  different groups of learners is the school
  thinking about this?

• Take into account any pupils included in targets
  but omitted from estimates, such as pupils
  without national assessment results at the
  previous key stage

86
Challenging Targets
Finding your way round FFT

The FFT estimates are found in two places in the
Analyses to Support Self-Evaluation booklet and
as separate sheets

Each gives similar, but slightly different,
information.
Estimates in the Analyses to Support
Self-Evaluation booklet Secondary P9 gives
KS2-3 L5 information P10-12 give K2-4 and KS3-4
information Primary P6 gives L4
information P7 gives L5 information. They give
type A (Prior Attainment), B (SE) and D (TQ)
estimates. They also give estimates based on the
schools average value added over the past three
years using the type A (PA) estimate. These are
shown in boxes. They are particularly useful for
schools that regularly exceed the type D (TQ)
estimates.
I hope this is easy
Now.lets look at some FFT targets!
87
Challenging Targets

Look at the extract below from p9 of the
secondary booklet (KS2-3 L5).
If the school has set a target for L5 in English
in 2008 of 83, this represents close to top
quartile progress. It is challenging but may be
hard to reach, particularly as the schools
three-year-average result is slightly below
expectation (78 compared to 79).
If the school has set a target in science of 75,
this does not seem challenging. It is in line
with average expectations, but lower than the
schools average three-year performance would
suggest (78).
83

Remember setting challenging targets is not
good in itself if they are unrealistic!
Boxes show PA (type A) estimates based on the
schools actual performance over the past three
years. This one shows us that in science the
school has performed on average 3 above PA
estimates over the past three years.
88
Challenging Targets

Estimates as separate sheets These are similar,
but give type A, B, C and D estimates. Below is
an extract from a KS1-2 sheet. Examples of sheets
are on the website http//www.fischertrust.org/per
formance.htm

• Compare the A and B targets you can see that
  contextualised targets are LOWER than targets in
  line with national expectations of progress

• Now compare the C and D targets you will see
  that to reach national targets, pupils will need
  to make non-contextualised progress at least as
  good as in the best 25 of schools in the
  country, which will be challenging for this
  school.

89
Challenging Targets

I set these targets for our school because they
are better than weve ever got before and provide
a consistent challenge for everyone!

• Pheasant Hill Secondary School has set Key Stage
  3 targets of
• 78 Level 5 in maths
• 78 Level 5 in science

As an inspector, consider whether Pheasant Hills
targets are challenging, realistic and likely to
lead to improvement.
90
Challenging Targets

78 Level 5 in maths would be a small improvement
on expectations (A) and close to a top 25
progress performance (D), but still below
national targets (C).
However 78 in science would be a big leap above
expectations (A) and put the school well inside
the best 25 of performers for progress (D). Is
it a realistic target?
Your science target is certainly challenging, but
is it realistic? How did you arrive at these
figures? Can we discuss PAST PERFORMANCE?