A finding that 10 per cent of Ofsted inspections are unreliable is no cause for alarm, according to a leading assessment expert.
Robert Coe, director for the Centre for Evaluation and Monitoring at Durham University, said: “On the face of it, I don’t think it sounds dreadful. In fact, it sounds quite good to me.
“The reality is that this is a subjective judgement process – you’re not going to have 100 per cent agreement. The reality is that we’re going to have to live with some misclassification or subjectivity.”
The new study, looking at the reliability of short inspections, is to be published tomorrow morning. A sample set of schools was given two inspections, conducted by different inspectors, in order to establish whether the watchdog’s judgments were objective and consistent.
The study, carried out by Ofsted itself, is expected to show that inspectors were in agreement over the rating that schools should be given in 90 per cent of the cases.
'It sounds very alarming'
The last piece of research into the reliability of Ofsted’s judgements was conducted by the inspectorate in 1998.
Professor Coe has previously drawn attention to the lack of research evidence behind Ofsted's inspection process. He said that there was no proof that its inspections and lesson observations led to valid judgements.
In light of this, he told TES that the new study was very welcome: “We’ve had 20 years of inspection since then, with no attempt to check whether judgements are reliable.”
But, he said, the fact that only nine out of 10 judgments were consistent was not a cause for concern. “If you’re a regular human being, you hear that 10 per cent are wrong and you get very alarmed,” he said.
“But, if you’re a statistician, you think: actually, they’re doing quite well, then. It’s in the nature of assessment that you get imprecision.
“There’s no system that anyone could invent, anywhere in the world, that’s going to reduce that to zero. Certainly no feasible or affordable system, anyway.”