It took the OFSTED computer seven hours to crunch numbers for every school in England to produce an individual, detailed reports. But how useful is the information they contain? Mark Whitehead reports
Furry animals can help improve the health, doctors say, and that is exactly what the Pandas, sent to all 24,000 primary and secondary schools in England, are intended to do.
These Pandas are not lovable bears but performance and assessment reports - dozens of pages of statistics, tables and charts, individually tailored for each school, which are meant to help school managers improve their standards and results.
The reports, the latest development of long-standing policy of giving schools more information about their performance, represent the largest amount of such data ever disseminated. Each report, compiled by the Office for Standards in Education using data from the Department for Education and Employment and the Qualifications and Curriculum Authority, runs to several dozen pages of figures on everything from the number of children in overcrowded households to percentages of pupils gaining different grades at GCSE.
They cover exclusions, length of the teaching week, unit costs, pupil teacher ratios and numbers of support staff. In each category, the school's own figures are given with national averages to aid comparison. Benchmark figures also enable comparison with other schools that have pupils from similar social backgrounds.
OFSTED says the Panda is a "management tool" for headteachers that will give them, for the first time, the opportunity to see where their school's performance has fallen behind that of any comparable school.
The Panda could undoubtedly be an important step towards recognising the importance of "added value" in assessing schools' performance. Schools traditionally thought to be the best performers could be, as OFSTED has pointed out, underachieving when social and economic factors are taken into account. OFSTED says improvement is needed in nearly a third of the most prosperous primary schools. Conversely, some schools in deprived inner-city areas could be shown to be doing an excellent job.
Benchmark performance targets developed by the QCA and included in the Pandas show that a typical comprehensive school with less than 5 per cent of pupils on free meals gets 62 per cent of pupils through GCSE with five passes at grades A to C. The equivalent pass rate for a school with more than 35 per cent on free meals is 20 per cent, while grammar schools should score 96 per cent. The average for secondary modern schools is 27 per cent.
A Panda will not offer any obvious answers, but it could help point towards the right questions about which areas of performance need to be improved says Anne Waterhouse, an adviser in Lancashire and until recently a primary school head.
But it will only work, she says, if people understand how statistics can be interpreted and if they can overcome their initial suspicions of such an impersonal approach. Lancashire is planning to step up its training programme in basic statistics for teachers to help them make sense of such reports.
"As a primary school headteacher, the idea of turning children into numbers didn't appeal at all," Mrs Waterhouse says. "But you need to collect information and the best way to do that is with numbers. You have to know where you are before you can decide where you want to go, and that means gathering data. If you can sift out what is useful, it can be very valuable."
The amount of information gathered for the Pandas is enormous, taking the OFSTED computer seven hours just to sift through it and produce the data for each school, and there are bound to be dangers in attempting such a massive exercise.
The most obvious problem, critics point out, is that the information on schools' comparable local catchment areas is based on the 1991 population census and is therefore already seven years out of date. The next census will be conducted in 2001.
In any case, as a result of legislation several years ago creating open enrolment, school catchment areas technically no longer exist. So, inevitably perhaps, much of the hard information on which any analysis is based is flawed.
Neil Thornley, head of Fearns high school in Bacup, Lancashire, says his Panda has massively underestimated the number of children with special needs at his school and does not give helpful comparative data. He already uses the value-added information provided by Lancashire and the YELLIS analysis (Year 11 information system) compiled at Newcastle University.
"The information we already have is far more useful than this," he says. "The statistics they use are inadequate and there are no measures of validity or reliability. These figures have been plucked from the air."
* Panda stands for Performance and Assessment
* OFSTED compiles the reports using information from inspections, the DfEE and the QCA
* The reports include "benchmarking" data for schools to measure their performance against others with pupils from similar economic and social backgrounds
* Schools are meant to use the Pandas to draw up targets to improve their pupils' results
* The next set of Pandas is due out in the autumn