The cancellation of this year’s Sats tests could make the government’s main secondary school performance measure impossible to calculate, a leading statistician has pointed out.

Progress 8 is a value-added measure that compares the progress that pupils make between their key stage 2 tests and GCSE results. So the lockdown cancellation of this year’s Sats will make it impossible to calculate Progress 8 in the usual way in 2025.  

Dave Thomson, of FFT Education Datalab, writes in a blog today that “maybe things will never be the same again” and states: “One obvious answer is to do nothing. After all, there are well-known issues with Progress 8 and what can be inferred from it.”

Research: Northern schools ‘penalised by Progress 8 tables’

Education Datalab: Pupil background ‘levels out North-South divide’

Exclusive: Progress 8 ‘penalises schools in white working-class communities’

But then he offers possible solutions, including the use of data from CAT tests (cognitive abilities tests) at the start of Year 7 - although he says that scaling up to a full national collection of such data is “unlikely to be a good use of anyone’s time following the disruption”.

The impact of coronavirus on Progress 8

Another alternative, says Thomson, is to use of data from key KS1 Sats, but he says this is far less satisfactory than using KS2 data because:

• There is a nine-year gap between KS1 and KS4.
• There will be missing data for a small but not insubstantial number of pupils who arrived in England during KS2.
• Teacher assessment levels are not as granular as test results.
• There are comparability issues between the KS1 results of infant schools and all-through primary schools.

However, taking some of these factors into account, Thomson used 2018 KS4 data for students in state-funded mainstream schools and linked it to their KS1 data - to see if it could work as an alternative way of calculating Progress 8.

He says the correlation between scores was high, but that the results “masked quite large differences” for some schools, and that schools in London appeared particularly affected, with 10 per cent changing by more than 0.5 (equivalent to half a GCSE grade) and another third changing by more than 0.25.

He concludes: “What we have here is a way of calculating an alternative Key Stage 4 value-added measure that produces results that are more similar to Progress 8 than perhaps you might expect for many schools.

“But, nonetheless, it would probably still be considered unsuitable for school performance tables even if it produced results that more closely mirrored Progress 8. Just because a measure could be produced, it doesn’t mean it should.”

You can read the blog here.