One of the most important questions asked about Ofsted is this: do its inspection gradings say more about the percentage of deprived pupils in a school than about the quality of education it provides?

(As one senior education adviser at the Department for Education (DfE) once said to me, in exasperated tones: “I don’t need Ofsted to tell me that schools are successful when their intake is largely the kids of professional, middle class parents”.)

The Education Policy Institute’s (EPI) report School Inspection in England: is There Room to Improve?, published this week, provides some answers to this question.

This report is the latest in a series of first-rate EPI reports which combine rigorous analysis of the national pupil database with careful commentary and challenging conclusions. 

EPI analysed the correlation between the grades given by Ofsted to schools whose performance, in terms of the academic progress of pupils, deteriorated substantially over time.

What the researchers found should give Ofsted, and the teachers and school leaders whose work it inspects, serious concern.

Value-added performance

The researchers conclude that school inspection outcomes appear to be disproportionately affected by the percentage of disadvantaged pupils in a school. Schools with more disadvantaged pupils were more likely to be judged by Ofsted as “requiring improvement”. The more advantaged the pupils, the more likely a school will gain a “good” or “outstanding” Ofsted rating.

The figures are stark. Secondary schools with no more than five per cent of pupils eligible for free school meals (FSM) are over three times as likely to be rated “outstanding” as schools where at least 23 per cent have FSM, only 14 per cent of which are given an “outstanding” grading.

This finding may not surprise many - schools with disadvantaged intakes have long complained that Ofsted inspectors fail to understand, or to evaluate, the immense amount of extra work they do to enable their pupils to learn and to make progress.

What is surprising, perhaps, is the EPI’s revelation that, based on value-added (VA) performance, the number of schools with advantaged pupil intakes given an “outstanding” Ofsted grading would be halved.

And, on the same VA criteria, the number of schools with disadvantaged pupils which would be rated “outstanding” would be doubled.

The EPI researchers conclude: “These findings raise questions about whether the inspection system is fully equitable to schools with challenging intakes. We have found that the least disadvantaged schools are most likely to be judged “good” or “outstanding”, and that notable proportions of “good” and “outstanding” schools are not down-graded, despite a substantial deterioration in their academic performance.

“We have also found that if schools were rated according to levels of pupil progress, we would expect many fewer “outstanding” schools with very low proportions of pupils eligible for free school meals, or low prior attainment when they joined the school.”

If this was not damaging enough to the view of an Ofsted inspection system free from pre-conceptions and bias, there is more.

Ofsted does not appear, the researchers conclude, to recognise, or its inspection judgements reflect, the lack of progress made by pupils in the schools where there has been a deterioration in VA progress scores.

Ofsted awarded higher inspection grades to 47 per cent of primary and 33 per cent of secondary schools, where there were large decreases in their VA progress measures between their previous and latest inspection.

This suggests, in the researchers’ words, that “Ofsted has not been as effective at consistent recognition of deteriorated academic performance as it has been at ensuring schools are inspected regularly.”

Just re-read the previous sentence: Ofsted is, according to the EPI researchers, better at inspecting schools to a regular timetable than it is at recognising when a school’s academic performance is deteriorating. 

Which leads me to ask a key question: Would it not be better if Ofsted did fewer inspections, but better?

Unreliable and invalid Ofsted judgements

Because too frequent inspections, allied with unreliable and invalid Ofsted inspection judgements, are two of the main reasons for the teacher exodus from the profession - a profession which, we should not forget, is being hollowed out (a previous EPI report revealed that more than half the teaching profession in England now has less than 10 years’ experience).

Schools with challenging intakes are, in a recruitment and retention crisis, finding it doubly difficult to attract teachers to work for them.

It is not hard to understand why these schools face this problem: it takes a brave teacher, or school leader, to risk their professional career on the judgements of an inspection agency which appears, on the basis of this evidence, to have difficulty in recognising the difference between a school’s pupil intake, and the quality of the education it provides.

These findings are, argue the report’s authors, highly significant both for Ofsted and for the ability of a self-improving school system to do what it says on the tin, i.e. improve.

The researchers argue: “The pattern we have described, whereby schools with the fewest disadvantaged pupils are rated more highly, and those with the most disadvantaged pupils are more likely to be judged “requires improvement” or “inadequate” matters for several reasons.

“It means that schools and school leaders with very few disadvantaged children are the most likely to become system leaders, providing training and advice to other schools, and that these system leaders are likely to be unevenly distributed geographically.

“It means that disadvantaged schools are more likely to struggle with recruitment and retention of high quality teachers and leaders. It encourages parents to believe that disadvantaged schools would offer a poor education to their child.”

The last point is really important.

For too long there has been an equation, powerful among parents and insufficiently challenged by those who should know better, that disadvantaged schools equate to poorly performing schools.

The EPI research challenges this forcefully, making the point that if schools with disadvantaged intakes were judged fairly, many would have much higher Ofsted grades, as the quality of their work in adding value to their pupils’ progress would be recognised.

As a result, their added difficulties in recruiting and retaining teachers and school leaders would be lessened, and parents’ perceptions of the quality of the education provided by these schools would be transformed.

All of which adds up to a powerful case for change on Ofsted’s part.

Perhaps this is something that Amanda Spielman will take up when she begins her term as Her Majesty’s chief inspector in the new year?

Dr Mary Bousted is general secretary of the ATL union. She tweets as @MaryBoustedATL

Want to keep up with the latest education news and opinion? Follow TES on Twitter and like TES on Facebook