Ofqual says Pisa rankings should be viewed 'critically'

11th March 2011 at 00:00
Watchdog's view of league tables puts it on collision course with ministers

The exams watchdog has disputed the reliability of the international education league tables that ministers use to justify radical schools reform.

Rankings produced from surveys such as the Programme for International Student Assessment (Pisa) are "not that meaningful" and England's performance in them "is not that worrisome", according to research highlighted by Ofqual.

The watchdog's warning - part of a Government-commissioned study - that such surveys should be viewed "critically" will put it in direct conflict with Coalition ministers.

This difference of opinion comes after Isabel Nisbet, then Ofqual's chief executive, argued in December that the evidence the Government used to call for fewer exam re-sits was outdated.

Education secretary Michael Gove has repeatedly used England's fall from eighth to 28th in the Pisa survey for maths and drops in other subjects as a justification for change.

"It shows we are falling further and further behind other nations," he has said. "These are facts from which we cannot hide."

But this week's Ofqual report quotes recent research from King's College London, which says: "The differences between countries' performance are not that large and are usually statistically insignificant.

"The `horse race' approach to the rankings produced by international studies - looking to see which position England is placed in and whether or not it has moved up or down the league tables - is not that meaningful, partly because the absolute differences in scores between countries are not that great and partly because the constituent group of comparators changes from study to study and from year to year. Overall, and over time, England's performance is not that worrisome."

Mr Gove has told Parliament that the Pisa rankings are "ungainsayable (undeniable)". But Ofqual uses the King's College findings to argue "there are good reasons to view the findings of such international tests critically".

The watchdog said it "cannot be assumed that the findings provide a definitive answer about which learners are the highest performing".

It notes a number of "assumptions" made about studies such as Pisa - that they are "somehow an objective measure of what is best", that the pupils involved are "a balanced representation of all learners at that stage of education", and that pupils in each country are "equally motivated to perform well".

A Department for Education spokesman said: "We have based our reform programme on wide-ranging domestic and international evidence and practice."

He said the "bottom line" was "that by any measure" too many pupils still left school without the skills or knowledge they needed.

Ofqual's comments come in a progress report on its own work on comparisons between A-levels and international equivalents.

The watchdog has added Alberta in Canada, Denmark, Finland, Ireland and South Korea to its study, alongside Hong Kong, New Zealand, New South Wales in Australia, the Netherlands and some parts of the United States.

But comparisons with India and Singapore - called for by Mr Gove - are still not included.

See the report at http:tinyurl.com6y8lz9l.

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar, Buyagift.com, Virgin Wines and other partners
Order your low-cost subscription today