Why predicting primary pupil progress is like forecasting the weather for next year
The weather app on my phone tells me that currently there is a thunderstorm in Tuvalu, that New Delhi remains under a hazy heatwave and it is partly cloudy in London – which I can verify by looking out of the window.
But the Met Office can also tell me that between six and seven hours from now, there is a 30 per cent chance of rain in London. To do this, it has a £97m supercomputer that undertakes more than 16,000 trillion calculations per second on hundreds of thousands of measurements of temperature, pressure and wind speed.
But even this computing leviathan loses precision as you ask it to predict further and further ahead. Forecasting weeks ahead, it will be pretty precise on the general picture, but not a specific area or hour.
Primary pupils are less chaotic than the weather, mostly. But not by much.
Look at the country as a whole – with around 600,000 children in each year group – and you can be pretty precise quite a long way ahead.
Previously, the starting point for the primary value-added measures was the levels children were given in reading, writing and maths at age 7. Four years later, where will those children be? They were expected to have made two levels of progress – which around 90 per cent managed.
Fine for the 600,000 year group as a whole, but taking an individual child’s ability to add two two-digit numbers at seven and using it to forecast whether they will be able to solve multi-step problems in May 2018 is not precise. In other words, you may be able to predict fairly accurately that it will be fairly warm, but not whether it will be 26C in Winchester.
And the new baseline test for primaries will stretch the forecast further by starting with maths, language and literacy at age 4.
It is, of course, worth trying to predict progress. We need to find out who is progressing and who is not – which can lead us to what impacts on progress and what does not. This week, the Missing Talent report from the Sutton Trust told us that bright children on free school meals are making less progress than similarly bright, but wealthier classmates during secondary school. An important finding.
Report author Dr Rebecca Allen, director of Education Data Lab, found that these “bright and poor” boys, who had been in the top ten per cent at primary, were more than twice as likely to fall out of the top 25 per cent at secondary than their non-FSM peers (36 per cent compared to 16 per cent). For “bright and poor” girls, the effect was greater, although fewer fell that far behind. (16 per cent compared to nine per cent).
But just look at those statistics again. Out of all of the students who arrive in secondary with having aced their primary education – one in six – will fail to reach their potential by quite a substantial margin.
The future grades of each child arriving in Year 7 this September are not set in stone. When children arrive, there will have been children just like them who have got the full range of GCSE grades. Like the probability of rain during your next holiday, you may want to know whether the likelihood of getting a C is 30 per cent or 70 per cent. Unlike rain, knowing that may help you change it.
In a secondary school, over five years, with large cohorts of 150 pupils plus, you can also get a fairly accurate picture of the year group’s progress overall. Those fuzzy data of children with good marks at 11 who don’t do so well at 16 – and the ones who exceed expectations – become smoother, but are not entirely ironed out.
But in primary, with cohorts of 60 pupils and progress measured over seven years, such accuracy is much harder.
Progress in primary will become much more important for schools from next year. Showing sufficient progress will mean a school is above the floor target – if they don’t achieve sufficient progress, then more than 85 per cent of pupils will need to reach the expected maths and English standards to avoid being labelled failures.
But predicting progress will always be a bit like sticking a weather probe in the air to see which way the wind will be blowing in 2016.
And before using progress to judge small numbers of pupils, over a long period of time, we should be careful not to leap to conclusions. Students not making progress could genuinely be the sign of a changing climate in the school or, like snow in June, just an unlikely – but not impossible – event.