It may be that primary registered inspectors are not “nervous” about submitting inspection bids, as has been suggested, but have scruples about being involved in such a doubtful exercise.

The problem lies in the nature of the judgments inspectors are asked to make. OFSTED requires judgments on standards of achievement, quality of learning and quality of teaching. The judgment on standards is in two parts: against norms or national expectations, and then against abilities. Each of these is expressed numerically on a 1-5 scale. It is easy to appreciate both the reasons for this conceptual framework and the desire for “hard” data, but I do not believe that those judgments can be made validly or reliably enough to sustain the weight put upon them.

Many inspectors start with quality of teaching, because they find this the most straightforward judgment to make. Yet in a slightly different context, in the assessment of teaching practice, many training institutions decline to grade students beyond a pass or fail. Their experience has shown that even with the involvement of tutors and class teachers over a five or six-week period, reliable judgments are hard to make.

Judgments about the quality of learning are often made next. Superficially, progress should be possible to identify and positive attitudes to see. In fact, the progress made by the whole class and whether it is sufficient is not easy to grasp. You can see if children are paying attention and applying themselves, but OFSTED’s requirement to grade learning independently of teaching presents a problem. Children’s attitudes are patently not independent of the quality of teaching.

It is, however, when we get to standards that difficulties really begin. It is possible to accept that a very experienced inspector, who has internalised the norms of national curriculum levels, had long experience as a subject specialist and has been in large numbers of classrooms, can make a sound judgment that most children are achieving at appropriate levels.

But even she would have problems with achievement against abilities. Is there such a thing as “ability” in that general sense? If there is, how do you show it with evidence? How much evidence do you need? Do you try instead to make some general judgment, based, say, on free school meals data or standardised scores?

Inherent in the Framework for Inspection is the fallacy of criteria. The presumption is that if the criteria can be specified clearly enough, the judgments will be valid and reliable. This is to ignore the nature of the judgment which can be made. In the time limits of a single lesson, there are so many variables and such selective evidence that judgments are partial, subjective and intuitive.

There is no doubt that most inspectors will have the experience to offer some useful comment on a lesson, to offer some insight about what was happening, what the outcomes were and how good it was. The most skilful and experienced will be able to offer some judgment as to how it compared with other lessons. As feedback to schools this is of value.

But the over-refined model in the Framework and the demand for 5 separate grades overstretches the limits of the technology even in the most experienced hands. Given the large numbers of inspectors, their diverse backgrounds, limited training and the absence of a collegial environment, such as exists within HMI, variation in interpreting and applying the criteria is unavoidable.

There is also a fallacy of enumeration. The pretence is that translating judgments into numbers doesn’t change them. The reality is that there is the evidence on the one hand and the number judged to correspond to it on the other; the number has a totally spurious objectivity and precision.

The reality is that the assessment of something as complex as classrooms is fraught with difficulties. Inspectors are invited to act as if it were simple and to operate the inspection model as if it and they were infallible. That is why it risks being unethical and dishonest.

Why has no-one pointed out that the emperor has only threadbare clothes? The answer is of course that the people who know want to earn a living; the living, it has been made clear, is operating the framework largely as a technician. Conformity is the order of the day. Schools may suspect but are in no position to put together an alternative picture. In any case inspection reports clearly work like horoscopes; even if you look at the wrong month it still tells you something that could be true.

Martin Shepherd is a team inspector