THE findings of an international survey that showed British 15-year-olds had leapt up the league of industrialised nations have been cast into doubt by an expert on comparative education.

Jubilant education ministers greeted the Programme of International Student Achievement (PISA) survey, published last autumn by the Organisation for Economic Co-operation and Development, as proof of the success of their strategies.

But Sig Prais, senior research fellow at the National Institute of Economic and Social Research, says the UK’s apparent surge up the maths league is due to flaws in the design and methodology of the survey rather than a sudden rise in standards.

He says the questions tested common sense rather than pupils’ knowledge, and that low participation rates by English schools and pupils mean low achievers were under-represented.

The PISA survey, carried out in spring 2000, placed British pupils fourth in science and eighth in literacy and maths out of 27 industrialised countries. This was a great deal better than the UK’s position in previous international surveys.

In maths, the PISA survey put British 15-year-olds on a par with the Swiss and ahead of France, Germany and Hungary.

But a survey conducted only a year previously on 14-year-olds (the same cohort) by the International Association for the Evaluation of Educational Achievement (IEATIMSS) put British pupils near the bottom of the European league.

Writing in the latest issue of the Oxford Review of Education, Professor Prais says previous international surveys of maths achievement were based on pupils’ mastery of the school syllabus. But the PISA survey aimed to test pupils on “everyday life” problems, which provides less guidance for policy on schooling.

”(The PISA tests) can be said to be nearer to tests of common sense (or of IQ),” he says. “No one has ever doubted the common sense of the British people, nor of British youngsters: the issue has been whether they are as well served as they might be by their schooling system.”

Professor Prais criticises the selected age of 15 for the survey. In Continental countries like Germany, he says, many pupils leave school and enter employment or vocational training at that age, thus making them hard to reach in a sample survey.

He points out that the response rate for English schools was particularly low (60 per cent compared with 95 per cent in France, Germany, Hungary and Switzerland), raising doubts about the inclusion of low-attaining schools.

And within participating schools in England, only four out of five pupils selected for the sample actually took part, the lowest pupil participation rate for any country - again raising concerns about under-representation of the low achievers.

“It is difficult to draw valid conclusions for Britain from this survey,” says Professor Prais.

Until the problems he lists are sorted out, he recommends the UK should not take part in any further rounds.

Oxford Review of Education, Vol.29, No.2, 2003 (www.tandf.co.uk)