Schools with lower performing intakes are “systematically penalised” by Progress 8 - despite the measure supposedly taking prior attainment into account - a new study has found.
Researchers at Cambridge Assessment found that schools’ Progress 8 scores “mainly reflected the student make-up of the school”, raising doubts about whether it is a fair way to hold schools to account.
A paper presented this week at the annual conference of the British Educational Research Association in Newcastle looks at Progress 8 scores for different types of pupils and schools.
The research found a “distinct positive relationship” between the mean prior attainment in a school and their Progress 8 score, with an increase of 1 in a school’s mean Key Stage 2 Sats leading to on average an increase of 0.48 in its Progress 8 score.
“Despite the measure taking account of prior attainment at the student level, Progress 8 systematically penalised schools with a lower performing intake,” the paper states, noting that “the size of the effect was substantial”.
This meant that the difference between a school with a prior attainment average of 4.25 at KS2 and a school with an average of 4.75 would equate to a Progress 8 difference of 0.24 - or one GCSE grade in every fourth qualification.
Progress 8 was introduced by the government because it was supposed to be a fairer way of holding schools to account than the old 5 A*-C measure.
The analysis also found other pupil-level differences in Progress 8 scores. For example, there was a “small, but statistically significant, negative relationship” between receiving free school meals and Progress 8, which indicated that “more deprived students had lower Progress 8 scores on average”.
There were also differences based on ethnicity, with white students having the lowest Progress 8 scores. Non-English speakers had a much higher mean Progress 8 score (0.42) than English speakers (-0.06).
These findings tally with claims reported by Tes earlier this year, that Progress 8 penalises schools in disadvantaged areas where there are few pupils who speak English as an additional language.
The paper notes that differences in schools’ Progress 8 scores “mainly reflected the student make-up of the school”.
“Schools with higher percentages of FSM, special educational needs or white students tended to have lower Progress 8 scores on average.”
The paper goes on: “These results suggest that some schools are penalised for factors which are beyond their control, and therefore, it is questionable that Progress 8 should be used as a measure for comparing schools, or for bringing them to the attention of the Ofsted (if they are below the floor standard).”
“It could be argued that a fairer way of judging schools would be to take account of some of these factors when calculating school performance measures.”
The Department for Education has been contacted for comment.
