Measuring attainment gaps ‘meaningless’, expert says

By Martin George on 10 September 2018

ResearchED told demographics mean schools serving 'incredibly diverse' communities struggle to close gap

Measuring attainment gaps between different groups of pupils is "usually a statistically meaningless exercise", a leading academic has said.

The gap between disadvantaged pupils and their peers at key stage 2 and key stage 4 is a key measure the DfE uses to judge its performance, while schools are given pupil premium funding to close the gap.

However, Becky Allen, a professor at the UCL Institute of Education, told Saturday’s ResearchED conference: “Don’t measure or monitor school attainment gaps. It’s usually a statistically meaningless exercise.

“It doesn’t help identify what works and the gaps matter little to the poor kids anyway. Absolute attainment matters.”

The researcher told the event that the size of the attainment gap between pupil premium and non-pupil premium children was largely a function of the demographic make-up of the latter, which can range from children of bus drivers to bankers.

She said schools that serve “incredibly diverse” communities are likely to struggle to close the attainment gap “not because of the things you do, but because of the diversity and the demographic make-up of the students that you have.

“This problem is one of the many reasons why no school can ever really claim that it has worked, closing the gap in their school.”

In her talk, Professor Allen cited the late health statistician Hans Rosling who she said “implores us not to overuse this type of group mean average analysis to make inferences about our world”.

She said his message was that the “gap stories that we like to tell are almost always a gross over-simplification of what is happening. They encourage us to stereotype groups of pupils who are not as dissimilar to others as the mean average would have us believe.

“And just because there are average group differences in things like attendance rates and behaviour incidents it does not make sense for us to then organise our attendance policies around things like pupil premium students.”