'See-sawing' rankings are shaky ground to build on

1st February 2013 at 00:00
Don't make education beholden to international tests, report warns

English education is in danger of being driven by its rankings in "see-sawing" international league tables that do not reflect the quality of schools, a major report has concluded.

The research, commissioned by social mobility charity the Sutton Trust, warns that ministers' repeated calls for better results in tests such as the Programme for International Student Assessment (Pisa) will result in "teaching to the test" rather than improvements to the system.

It also claims that politicians have "spun" England's test performance to justify their reforms, when schools play only a small part in the results' variations. International rankings are influenced by a range of factors, including cultural differences, the way the tests are conducted and who takes part, the report says. For example, the dominance of East Asian countries could be down to ambitious "tiger mothers" and a culture of hard work as much as schooling.

The Sutton Trust said that, to ensure England catches up with higher-performing nations, the government needs to focus on reducing social inequality rather than altering school structures.

The report's author, Professor Alan Smithers from the Centre for Education and Employment Research at the University of Buckingham, told TES: "The danger of placing too much importance on ranking is that it may start to define education, as it is already defined by key stage 2 tests and GCSEs."

He said that by focusing on international tests, which measure skills in maths, science and reading, the country risks adhering to a "very narrow definition of what education can be about".

The Sutton Trust report - titled Confusion in the Ranks - also highlights the dangers of placing too much emphasis on England's ranking in any one table at any one time. "Pisa appears to be saying all is not well. Timss (the Trends in International Maths and Science Study) seems to be showing us improving. The Pearson Global Index (of Cognitive Skills and Educational Attainment) has us among the best in the world. They cannot all be right," it says.

Education secretary Michael Gove has repeatedly used international rankings to justify his reforms. But Andreas Schleicher, deputy director for education at the Organisation for Economic Cooperation and Development (OECD), which administers Pisa, has said that the results give a "holistic" view of a child's abilities, with private tutoring and parenting as well as schooling influencing the outcome.

The Sutton Trust report argues that league tables exaggerate the importance of raw test scores: when a country is one or two points above or below another, it has probably got the same level of performance, it says.

There are also significant differences in the number of countries taking part in tests such as Pisa, Timss and Pirls (the Progress in International Reading Literacy Study), thus making the league tables incomparable (see panel, left).

""The messages are decidedly mixed," the report concludes. "The scores arise in many different ways, with perhaps the schools playing only a small part.

"The international comparisons are yielding masses of data, which could prove invaluable. But they need to be interpreted cautiously and sensitively."

Sir Peter Lampl, chair of the Sutton Trust and the Education Endowment Foundation, said that global education tables are "an increasingly important tool" in political debate but added that "see-sawing" rankings are "not always what they seem".

John Bangs, who is chair of the Education, Employment and Training Working Group for the Trade Union Advisory Committee to the OECD, said that many of the results in various tests do reflect what is going on in schools. For example, Pirls results over time have reflected "a coherent primary strategy in literacy and numeracy" since the late 1990s.

"We should not ignore international studies. The last thing we should do is treat the figures as too general to be relevant to individual countries," he said. But he agreed that ministers often exploit the figures for their own ends: "Michael Gove cherry-picks, and his reinterpretations of the evidence are unrecognisable to Pisa."


England's position in secondary maths:


Pisa 2009


Timss 2011

Secondary science:


Pisa 2009


Timss 2011

Primary reading:


Pirls 2011



Pearson Global Index 2012

Source: Confusion in the Ranks, the Sutton Trust


Pisa: Programme for International Student Assessment

Timss: Trends in International Mathematics and Science Study

Pirls: Progress in International Reading Literacy Study

Pearson Global Index of Cognitive Skills and Educational Attainment.

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar, Buyagift.com, Virgin Wines and other partners
Order your low-cost subscription today