Pisa rankings are ‘as useful as student drinking games’

12th August 2016 at 01:00
Your country may top the table, but does a hangover come in the form of a creatively stifled workforce?

Countries should “ignore” the world’s most influential education rankings because they fail to measure what matters, an expert on the impact of globalisation on education has claimed.

The idea of nations competing to reach the top of the Programme for International Student Assessment (Pisa) league tables makes as much sense as university students competing to see who can drink the most beer, according to Professor Yong Zhao, from the University of Oregon in the US.

He told TES: “You’re maybe the best drinker but you’ve got to think, ‘Is it good for you and does it matter?’”

According to Professor Zhao, Pisa – which provides a snapshot comparison of how well 15-year-olds in different countries perform in reading, maths and science – homogenises education systems. While Asian countries tend to do well according to these measures, he insisted that a homogeneous workforce was not what was required for a successful future. Instead, he said, countries needed “creative, entrepreneurial talents, able to create value for others”.

Narrow spectrum of skills

Professor Zhao, who will deliver one of the keynote addresses at the Scottish Learning Festival next month, continued: “We should ignore Pisa entirely. I don’t think it is of any value. If you look at the so-called high-scoring areas, like Shanghai and all the East Asian countries, they are trying to get away from what has made them high on Pisa [rankings].”

The academic, who was educated in China, said that the country’s education system was an effective machine that could instil what the government wanted students to learn, but it did not nurture creativity. The result is that China has a population with similar skills on a narrow spectrum, he claimed.

Professor Zhao is not the first academic to criticise Pisa. TES has reported on experts who have “serious problems” with the rankings (bit.ly/PisaFlawed). Some criticise the statistical techniques used to compile the results, while others dismiss the whole idea of being able to accurately rank such diverse education systems.

The Organisation for Economic Co-operation and Development (OECD), which administers Pisa, has sought to broaden assessment for the next round of rankings, the results of which will be published in December. They will include the outcomes from a new “teamwork test” alongside scores in maths, reading and science.

Andreas Schleicher, director of education and skills at the OECD, said that the test, measuring “collaborative problem-solving”, could change the way that Pisa was viewed (see box, below).

But Professor Zhao insisted that success should depend not on good Pisa scores, but instead on emphasising what makes us human: the ability to design, tell stories, understand emotions, entertain and find meaning.

Young people all over the world were leaving university unable to find work, he added.

Kids should ‘invent their jobs’

Many education systems had been designed to produce employees for jobs that were now being done by machines or outsourced to Asia. What was needed in schools was an entrepreneurial mindset, so that children develop the ability to “invent their own jobs”, he argued. Such an education system would have no prescribed curriculum. Instead, the education would follow the child, tapping into what they were interested in or passionate about and encouraging them to identify problems that needed solving.

“Every country has been trying to improve its education system by better prescribing the curriculum and focusing on the basics such as literacy and numeracy, guided by studies like Pisa,” said Professor Zhao. “If you want to create something that will take you to the moon, you should stop tinkering with the horse and wagon; adding things to the traditional paradigm is no different to fattening a horse to go to the moon.”

Professor Zhao was himself a product of the kind of authoritarian education system that he is so critical of. He did not leave China for the US until he was 27, yet he is already the author of more than 100 articles and 20 books. Clearly his own creativity has not been stifled. But his response is: “Imagine if I did not go through that system.”

The OECD declined to comment.


‘Collaborative problem-solving’ test could transform rankings

When the next set of Pisa global rankings are published in December this year, there will be a whole new set of potentially game-changing data for governments, journalists and educationalists to pore over.

Participating pupils have, for the first time, taken a test in “collaborative problem solving”. This could lead to a big change in the way that the tests are viewed, according to Andreas Schleicher (pictured, inset), education director at the Organisation for Economic Co-operation and Development.

Under the new tests, 15 year olds who took part in Pisa last year were given computer-based tasks in which they worked through a “chat” function with computer-generated virtual collaborators to solve a problem. The measurement focuses primarily on the way a pupil engages with others, rather than solely on the correct solution.

“It’s the first time countries are going to have the evidence, and it’s going to be interesting and important,” Mr Schleicher told TES in December. “If we find some countries are very good in solving problems, but not good at collaborating, I think [governments] are going to pay attention because it is going to matter.”

He said that while the headline maths, reading and science scores would continue to be significant, employers were looking for a wider set of skills, including teamwork and communication.

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number


The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now