Mounting pressure from schools is forcing the government to look again at its new headline “fairer” secondary accountability measure, Progress 8, TES can reveal.
A growing number of heads have raised fears about how much a school’s overall score on the crucial measure can be distorted by poor performance from a handful of pupils. And their concerns have been backed up by research.
Now multiple sources have said that the Department for Education is responding by considering changes to the rules around Progress 8 (P8), possibly to limit the impact that an individual pupil can have on a school’s score.
TES has seen a letter from a regional schools commissioner in which they give a concerned headteacher assurances that the department was looking very closely at the large impact that a very small number of pupils can have on a school’s P8 measure.
Meanwhile, the Association of School and College Leaders’ deputy policy director, Duncan Baldwin, revealed that the association was “deeply concerned about the issue”, but said it had elicited a positive reaction when it “challenged the DfE to respond”.
“I know they are looking at how to address it,” he told TES. “They do recognise that it is a serious problem.”
Ofsted also disclosed in a letter to inspectors this month that it, too, had concerns about how secondary school performance data “could have been affected by outliers” - “pupils with extremely high or low progress scores” (bit.ly/OfstedLett).
Now the DfE has told TES that it is “looking carefully at this issue [of whether a small number of pupils can distort a school’s Progress 8 score]”, following talks with school leaders.
The news came as it emerged that schools were also worried about how pupils absent from teaching and or exams because of serious illness can distort P8 scores.
The impact of sickness
Mr Baldwin said: “Lots of [ASCL] members have expressed concern about that. Those students who score 0 are deeply vulnerable students.
“Any school could have one or more students like this a year. You can get a sense of whether they are vulnerable or risky or not - but you can’t tell when a student is going to be ill.
“So we need to make sure that the effects the scores have are mitigated to some extent.”
We need to make sure the effects these scores have are mitigated to some extent
Progress 8 was designed by the government to be fairer by taking into account the achievements of pupils of all abilities, not just those who achieve five or more A*-C grades at GCSE - and is used for high-stakes secondary school floor targets.
But the measure, based on average progress across a school, means that pupils who take no exams, or hardly any, can do significantly more damage to the school’s overall score than under previous measures.
TES revealed in January how heads feared that some schools would try to “lose” their most vulnerable pupils because their GCSE results could be particularly damaging to P8 scores (“Exclusive: Schools may try to ‘lose’ vulnerable pupils because of Progress 8”, bit.ly/LosePupils).
The warning came as the first full set of official P8 scores was published by the DfE, and Education Datalab shared with TES the findings of a study showing that schools that were significantly affected by a few pupils with extremely low scores were overwhelmingly those that had more disadvantaged intakes.
Education Datalab researchers suggested that the problem could be solved by capping P8 to ensure that no pupil could fall below -2.5 or rise above +2.5 on the measure.
TES understands that the DfE has now asked the thinktank for its views on P8.
This week Rebecca Allen, director of Education Datalab, said she thought that there was “some prospect” of getting a cap.
She said it would be more difficult to establish new rules for “disapplying” - removing - unwell students from a school’s P8 score, which she said could result in new problems.
“It is not just that the incentives end up in a bad place around who provides an education for these students,” she said. “It is also how do we categorise illness?”
A DfE spokesman said: “Progress 8 is a fair judge of school performance. It assesses the value they add to a child’s education and holds them to account for the performance of all pupils, incentivising them to raise the attainment of every child including those from disadvantaged backgrounds.
“We are looking carefully at this issue [whether a small number of pupils can distort a Progress 8 score] and have been in discussion with ASCL and school leaders.”
