International league boost 'due to flaws'

23rd May 2003 at 01:00
British 15-year-olds did astonishingly well in global maths tests last year. Or did they? Biddy Passmore on a top researcher's doubts

THE findings of an international survey that showed British 15-year-olds had leapt up the league of industrialised nations have been cast into doubt by an expert on comparative education.

Jubilant education ministers greeted the Programme of International Student Achievement (PISA) survey, published last autumn by the Organisation for Economic Co-operation and Development, as proof of the success of their strategies.

But Sig Prais, senior research fellow at the National Institute of Economic and Social Research, says the UK's apparent surge up the maths league is due to flaws in the design and methodology of the survey rather than a sudden rise in standards.

He says the questions tested common sense rather than pupils' knowledge, and that low participation rates by English schools and pupils mean low achievers were under-represented.

The PISA survey, carried out in spring 2000, placed British pupils fourth in science and eighth in literacy and maths out of 27 industrialised countries. This was a great deal better than the UK's position in previous international surveys.

In maths, the PISA survey put British 15-year-olds on a par with the Swiss and ahead of France, Germany and Hungary.

But a survey conducted only a year previously on 14-year-olds (the same cohort) by the International Association for the Evaluation of Educational Achievement (IEATIMSS) put British pupils near the bottom of the European league.

Writing in the latest issue of the Oxford Review of Education, Professor Prais says previous international surveys of maths achievement were based on pupils' mastery of the school syllabus. But the PISA survey aimed to test pupils on "everyday life" problems, which provides less guidance for policy on schooling.

"(The PISA tests) can be said to be nearer to tests of common sense (or of IQ)," he says. "No one has ever doubted the common sense of the British people, nor of British youngsters: the issue has been whether they are as well served as they might be by their schooling system."

Professor Prais criticises the selected age of 15 for the survey. In Continental countries like Germany, he says, many pupils leave school and enter employment or vocational training at that age, thus making them hard to reach in a sample survey.

He points out that the response rate for English schools was particularly low (60 per cent compared with 95 per cent in France, Germany, Hungary and Switzerland), raising doubts about the inclusion of low-attaining schools.

And within participating schools in England, only four out of five pupils selected for the sample actually took part, the lowest pupil participation rate for any country - again raising concerns about under-representation of the low achievers.

"It is difficult to draw valid conclusions for Britain from this survey," says Professor Prais.

Until the problems he lists are sorted out, he recommends the UK should not take part in any further rounds.

Oxford Review of Education, Vol.29, No.2, 2003 (

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar,, Virgin Wines and other partners
Order your low-cost subscription today