Schools with large numbers of lower ability pupils will be disproportionately penalised by Progress 8 – the government’s new “fairer” secondary accountability measure – according to new data analysis shared with TES.
The Department for Education had specifically designed the measure – which will be used to trigger forced academy conversions – to avoid giving schools with relatively low-ability intakes an in-built disadvantage.
But the results of a new study by Education Datalab director, Rebecca Allen, suggest that this goal has not been met. Dr Allen used summer 2015 GCSE results to work out Progress 8 scores for all state secondaries in England.
Her study found that secondaries with lower average prior attainment, according to pupils’ key stage 2 test results, were more likely to have low scores on Progress 8, which will be introduced for all secondaries in 2016-17.
“It has real consequences for the accountability threat to schools since those with lower ability intakes are far more likely to fall below the announced floor standard,” Dr Allen said.
Failing to reach the floor standard leaves schools subject to intervention and the possibility of forced academy conversion.
Dr Allen confirmed the picture by plotting the number of secondaries that would have fallen below the Progress 8 floor standard and finding a direct relationship with pupils’ prior primary attainment. The lower pupils’ average primary test scores were, the more likely a secondary was to miss the official DfE target.
In theory this relationship should not exist, as Progress 8 is designed to be fair to secondaries by measuring the progress pupils make from any given point, rather than overall achievement. So pupils’ lower prior attainment should not disadvantage a school.
Dr Allen said that she expected much of the relationship between prior attainment and Progress 8 to persist even after schools had adapted their curriculum to the new measure.
Duncan Baldwin, deputy director of policy at the Association of School and College Leaders (ASCL), told TES he had also found that secondaries where pupils’ average primary prior attainment was higher scored better on Progress 8. “I think what people will find difficult to understand is that Progress 8 is statistically unbiased on a pupil-by-pupil basis. But it does favour schools where attainment is high and we don’t know why,” he said.
Dr Allen posed two scenarios. In the first the relationship could be explained by schools serving “relatively affluent” areas having pupils with “a supportive home environment”. In this case, “Progress 8 would seem unfair to schools serving disadvantaged communities, unfit as an indicator for school performance for parents and a weak accountability measure,” Dr Allen said. “It is almost impossible for them to compensate for home environment. It is unfit as an indicator because parents might wrongly infer their child is likely to underperform. And it is a weak accountability measure because we hold some schools responsible for things outside their control, whilst others can coast without fear of intervention.”
An alternative explanation was that “there are more effective schools serving affluent communities than there are those serving deprived communities”. She said this could be because schools in disadvantaged areas had higher teacher turnover and recruitment problems.
The founder of the National Association of Secondary Moderns (NASM) Ian Widdows represents schools that have, by definition, lower-ability intakes. “Ofsted inspectors need to understand the context of these lower Progress 8 scores at secondary moderns. The danger is that people will assume schools with low-ability intakes are second best,” he said.
Others fear Progress 8 will not work because it is underpinned by primary assessment scores, which they see as unreliable (see box).
Mr Baldwin said the link between Progress 8 scores and pupil ability was a “system issue” but that should not detract from its benefits.
“No measure is perfect. It is a much fairer measure than what we had before. It’s a much more helpful measure than the 5 A* to C was for disadvantaged intake schools,” he said.
The DfE said: “Progress 8 will reward better teaching of all pupils, no matter their background, and make the system of measuring performance fairer by focusing on progress. Every increase in grade will attract additional points in performance tables.”
This is an article from the 4 March edition of TES. This week's TES magazine is available in all good newsagents. To download the digital edition, Android users can click here and iOS users can click here