The Observer’s statistics are intended to provide a different view of school performance from “raw” league tables, which have been widely condemned as misleading and harmful to the morale of schools in disadvantaged areas. The tables may, to some extent, provide a preview of the sort of analysis planned by the Scottish Office using a school characteristics index.

But, sadly, the results of the Observer analysis are as misleading and harmful as the tables its methodology seeks to replace. Indeed, this sort of pseudo value-added approach is more harmful because it gives a false sense of fairness in controlling for intake.

The Observer has obtained data from the Scottish Office school census on the percentage of pupils per school entitled to free meals and used this as a measure of the intake characteristics. The league table published is based on the percentage of pupils in secondary 4 who obtained five or more Standard grade awards at levels 1-2.

The data was analysed using a simple regression of the school-level results. The “value-added” indicators that are reported for schools are the raw residuals from the analysis. The paper’s table shows positive value-added scores for the majority of schools in Glasgow. It shows also that all Glasgow schools have high levels of entitlement to free school meals compared to the rest of the country. These results provide an interestingly different perspective on the performance of deprived Glasgow schools, and should provide a welcome boost to their morale.

In contrast, the figures for the east of Scotland show comparatively low levels of entitlement to free meals and negative value-added scores. In particular, many Aberdeen schools have large negative value-added scores, which appear to show that most of the city’s schools are doing a great deal worse than any others in Scotland. Research at the Centre for Educational Sociology shows that this is not the case.

The main difference between Aberdeen schools and others is the effect of a relatively buoyant local labour market on parental employment and income, and hence entitlement to free meals. Many parents and pupils don’t apply for free meals because this may incur a stigma, and ethnic minority families seem unwilling to apply.

A further disappointing feature of the Observer table is the rank ordering of schools within each authority. The schools at the top appear very similar to those shown by “raw” tables. But the placing of some schools near the bottom is puzzling and the effect demoralising. Differences between schools have been grossly exaggerated.

The main benefit of the “raw” tables is their simplicity, and there is widespread understanding of their inadequacy.

The purpose of a value-added analysis of school performance is to measure the progress pupils make as a result of their learning and experiences at school. To measure this we need adequate measures of children’s educational progress at the time they enter school. Such measures cannot be provided by average entitlement to free meals. This is a highly flawed proxy for pupil attainment, and distorts the association between poverty and ability to learn.

Research at the Centre for Educational Sociology in Edinburgh and the Institute of Education at London University has demonstrated that the analysis of value added of schools requires: * Pupil-level data, not average figures per school.

* Several measures of pupil intake, especially prior attainment, age and sex.

* School characteristics, including school size and socio-economic context.

* Varied measures of the outcomes of schooling.

* Statistical techniques that take account of differences in school size and other potential biases in the data.

* Techniques that examine differences in progress of different groups of pupils, such as those of high and low prior attainment.

Research shows that differences between schools in value added are very small indeed when measured using adequate data and statistical techniques. We have not yet been told by the Scottish Office how it intends to construct the school characteristics index for future analyses of value added by schools.

If the methods are restricted to school-average measures such as percentage entitlement to free meals and other catchment characteristics, rather than pupil-level analysis using measures of prior attainment, their results will be very similar to those demonstrated in the Observer league table.

I hope that the Scottish Office will look carefully at the distorted results published by the Observer and appreciate their inadequacy.

Linda Croxford is a senior research fellow at the Centre for Educational Sociology, Edinburgh University.