'See-sawing' rankings are shaky ground to build on

1st February 2013 at 00:00
Don't make education beholden to international tests, report warns

English education is in danger of being driven by its rankings in "see-sawing" international league tables that do not reflect the quality of schools, a major report has concluded.

The research, commissioned by social mobility charity the Sutton Trust, warns that ministers' repeated calls for better results in tests such as the Programme for International Student Assessment (Pisa) will result in "teaching to the test" rather than improvements to the system.

It also claims that politicians have "spun" England's test performance to justify their reforms, when schools play only a small part in the results' variations. International rankings are influenced by a range of factors, including cultural differences, the way the tests are conducted and who takes part, the report says. For example, the dominance of East Asian countries could be down to ambitious "tiger mothers" and a culture of hard work as much as schooling.

The Sutton Trust said that, to ensure England catches up with higher-performing nations, the government needs to focus on reducing social inequality rather than altering school structures.

The report's author, Professor Alan Smithers from the Centre for Education and Employment Research at the University of Buckingham, told TES: "The danger of placing too much importance on ranking is that it may start to define education, as it is already defined by key stage 2 tests and GCSEs."

He said that by focusing on international tests, which measure skills in maths, science and reading, the country risks adhering to a "very narrow definition of what education can be about".

The Sutton Trust report - titled Confusion in the Ranks - also highlights the dangers of placing too much emphasis on England's ranking in any one table at any one time. "Pisa appears to be saying all is not well. Timss (the Trends in International Maths and Science Study) seems to be showing us improving. The Pearson Global Index (of Cognitive Skills and Educational Attainment) has us among the best in the world. They cannot all be right," it says.

Education secretary Michael Gove has repeatedly used international rankings to justify his reforms. But Andreas Schleicher, deputy director for education at the Organisation for Economic Cooperation and Development (OECD), which administers Pisa, has said that the results give a "holistic" view of a child's abilities, with private tutoring and parenting as well as schooling influencing the outcome.

The Sutton Trust report argues that league tables exaggerate the importance of raw test scores: when a country is one or two points above or below another, it has probably got the same level of performance, it says.

There are also significant differences in the number of countries taking part in tests such as Pisa, Timss and Pirls (the Progress in International Reading Literacy Study), thus making the league tables incomparable (see panel, left).

""The messages are decidedly mixed," the report concludes. "The scores arise in many different ways, with perhaps the schools playing only a small part.

"The international comparisons are yielding masses of data, which could prove invaluable. But they need to be interpreted cautiously and sensitively."

Sir Peter Lampl, chair of the Sutton Trust and the Education Endowment Foundation, said that global education tables are "an increasingly important tool" in political debate but added that "see-sawing" rankings are "not always what they seem".

John Bangs, who is chair of the Education, Employment and Training Working Group for the Trade Union Advisory Committee to the OECD, said that many of the results in various tests do reflect what is going on in schools. For example, Pirls results over time have reflected "a coherent primary strategy in literacy and numeracy" since the late 1990s.

"We should not ignore international studies. The last thing we should do is treat the figures as too general to be relevant to individual countries," he said. But he agreed that ministers often exploit the figures for their own ends: "Michael Gove cherry-picks, and his reinterpretations of the evidence are unrecognisable to Pisa."


England's position in secondary maths:


Pisa 2009


Timss 2011

Secondary science:


Pisa 2009


Timss 2011

Primary reading:


Pirls 2011



Pearson Global Index 2012

Source: Confusion in the Ranks, the Sutton Trust


Pisa: Programme for International Student Assessment

Timss: Trends in International Mathematics and Science Study

Pirls: Progress in International Reading Literacy Study

Pearson Global Index of Cognitive Skills and Educational Attainment.

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number


The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now