Ofsted has failed to ensure that school inspection judgements are reliable, with some inspectors basing their decisions on a “narrow range of data”, one of the watchdog’s leading officials has admitted.

In a significant departure from Ofsted’s previously robust defence of its inspectors, national director for schools Sean Harford has publicly acknowledged for the first time that the watchdog does not “directly [ensure] that different inspectors in the [same] school on the same day would give the same judgement”.

The unprecedented statement, issued in response to a critical blog post by leading headteacher Tom Sherrington, also reveals that Ofsted is carrying out “reliability testing” in pilot inspections this term, in order to assess consistency between different teams of inspectors.

Mr Harford vows to take action if problems are uncovered. The “weakest” inspectors, he writes, “have been guilty of using the published data as a safety net for not making fully rounded, professional judgements”.

Criticism of the inspectorate has intensified in recent weeks. Ahead of May’s general election, both the Conservatives and Labour have mooted wide-ranging reforms. The Tories are reported to be considering a slimmed-down, data-driven inspection system, while shadow education secretary Tristram Hunt this week called for major reforms to move Ofsted away from a culture of “box-ticking and data dependence”.

Data compiled by the Watchsted website for TES in November revealed that nearly 50 of Ofsted’s most prolific lead inspectors had yet not judged a single school to be outstanding in 2014 (“Does your grade depend on the inspector you get?”, 21 November). School leaders claimed the figures confirmed long-standing complaints that a school’s grade largely depended on “luck of the draw” over which inspection team turned up.

In a robust response, chief inspector Sir Michael Wilshaw wrote in TES that accusations of inconsistency had become a “Pavlovian response to unfavourable inspection outcomes” and were used by schools “as a way of deflecting attention away from underperformance and failure” (“Schools, stop passing the buck to inspectors”, Comment, 28 November).

But school leaders described Mr Harford’s latest statement as a “definite shift” in tone. “I agree that Ofsted has not done enough in the past to test the reliability of inspection; we have concentrated on quality assurance,” the national director writes. “This provides assurance that the process is carried out consistently as we would wish, but not directly that different inspectors in the school on the same day would give the same judgement. I have built in reliability testing for the pilots for the new short inspections this term. If reliability is a problem, we will review the issues.”

Mr Harford says that some inspectors have focused too narrowly on data at the expense of the “bigger picture”, adding: “Published data should only ever be a `signpost’ for the schoolinspectors to consider what they may be telling us, not the predetermined `destination’. This is how we train inspectors and how the vast majority use data, but the weakest ones have been guilty of using the published data as a safety net for not making fully rounded, professional judgements.”

Mr Sherrington, headteacher of Highbury Grove School in North London, was “amazed” by Ofsted’s response to his blog. He said there were “plenty of stories of inspectors who rely on too-narrow data sets”, despite “massive margins of error”.

“We’re in a new era in education: everyone wants evidence and reliability,” he added. “For that to apply to the inspection process seems logical but hasn’t been tested. [Inspectors] can’t just continue to assert authority; they have to demonstrate reliability.”

Ofsted will bring all inspections in-house in September. It has also proposed the introduction of shorter but more frequent inspections for good schools and colleges.

Stephen Ball, principal of New Charter Academy in Greater Manchester, said: “It is astonishing and deeply troubling for an organisation that has one fundamental job - to judge the quality of education in schools - to admit it is sending inspectors into schools who cannot make a rounded, professional judgement.”

Russell Hobby, general secretary of the NAHT headteachers’ union, welcomed moves to increase reliability. “The stakes are so high for schools; we cannot afford to have inconsistency,” he said. “If inspectors’ judgements are predetermined by data, then we don’t need Ofsted sending people into schools. It becomes a pantomime to justify accountability. The current inspection regime is coming to the end of its lifespan.”

Jonathan Simons, head of education at the Policy Exchange thinktank, which published a highly critical report on Ofsted last spring, said Mr Harford’s comments marked a “definite shift in practice”.

“Heads and teachers have complained for a long time about inconsistency,” he said. “It’s really promising that Ofsted is now trialling reliability testing.”

The `absurd simplicity’ of data

Tom Sherrington is headteacher of Highbury Grove School in North London and a member of the Headteachers’ Roundtable thinktank. His school is expecting an inspection in the near future.

Mr Sherrington blogs at headguruteacher.com. In the post that prompted the frank admission of inconsistency from Sean Harford, Ofsted’s national director for schools, Mr Sherrington rejects the idea that ”schools can be judged in a meaningful way via inspections” and assigned an overall grade based on the “absurd simplicity of two or three data points”.

He calls for an official school response to inspectors’ findings to be included in the inspection report, as well as an initial meeting with inspectors to allow a headteacher to “establish the intellectual credibility of the team members”.