Truly, badly, probably

15th December 1995 at 00:00
About 2 per cent of schools are being classified as "failing" by inspectors from the Office for Standards in Education. There are good reasons to believe that they are mistaken in the majority of cases because, among other things, they are "neglecting the base rate". Even a very accurate test is likely to make many errors when detecting something with a very low probability.

Given a base rate of 2 per cent, the probability of error could be calculated if we also knew the reliability of the inspectors' judgments and used that to estimate their accuracy. No proper tests of reliability have ever been made but if we assume, generously and unrealistically, an accuracy of 95 per cent, the likelihood of false classification is 72 per cent. The method for calculating this was published by the Reverend Thomas Bayes in 1763, and there is nothing new or controversial in it.

The real base rate is probably less than the 2 per cent currently found, and the accuracy of OFSTED's procedures lower than I have assumed. So an even greater proportion of its classifications are likely to be wrong.

These calculations must be taken seriously. It is unfortunate that OFSTED does not appear to consider them, or the measures which might give its judgments greater credibility. Among other things, it should estimate the reliability of its methods and use independent assessments to improve the accuracy of its classifications. A second visit is sometimes made, but this can hardly be described as independent. OFSTED's work is amateurish and our education system deserves better.

IAN HOWARTH Emeritus professor Department of psychology Nottingham University

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number

Comments

The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now