Despite attempts over the past 20 years to make inspection judgements more consistent, it is obvious to headteachers who have been inspected more than once that lead inspectors can put a very personal stamp on final judgements.
Sometimes this comes out, as Ken Muir suggests, in the professional dialogue around the draft report (17 August). Perhaps the most worrying aspect of the HMI secondary school inspection process is not, however, inconsistency in the judgements of different inspectors, but the failure of HMIE to recognise that they use statistics of examination performance to make unreliable, potentially misleading, statements about school performance.
A notable example is the use of the term “similar schools”. Schools are said to perform better or less well than “schools serving young people with similar needs and backgrounds”. The impression is given that there is a definitive ranking across “those schools (up to 20) which are statistically close enough to the school in terms of the key characteristics of the school population”.
There are several reasons why these comparisons, based on a number of statistical elements, are only approximate: the most significant element is “free school meals registration” (FSM), a “proxy” indicator of socio- economic characteristics. However, even when the FSM is identical, this can conceal significant differences.
Two schools, for example, may have an identical FSM of 20 per cent. However, one is in a “desirable suburb”, attracting a high number of placing requests, with 50 per cent of its pupils coming from professional backgrounds, while the other draws from a very different catchment area, with placing requests out and with few parents in professional occupations.
These two schools are obviously dissimilar in the “key characteristics” of their pupil populations, yet might well appear in the same group of 20. Only 20 per cent of the pupil population is compared. Close enough? Clearly both the process of inspection and individual cases should be subject to appeal.
Once established, case judgements in appeal would be used to settle similar cases at an earlier stage in the complaints or appeals process. A technically-aware independent “appeals panel”, including, for example, an academic with a background in social statistics, along with headteachers and others drawn from the public service professions, would be perfectly capable of providing such a service. With a remit to comment on the process of inspection and its reliability, they would provide inspectors with much needed technical accountability.
Daniel Murphy, Stirling.
