Sampling issues cast doubt on Canada's Pisa performance

Canada's apparently stellar performance on Pisa could be undermined by low participation and high rates of exclusion and absence, new analysis suggests.

canada day

A low participation rate in the Programme for International Student Assessment (Pisa) in Canada casts doubts on the country's strong performance, according to a leading academic.

In 2015, the Canadian sample was representative of only about half of the 15-year-old population in the country, compared to about 90 per cent in Japan and South Korea, according to figures cited by UCL Professor John Jerrim.

Analysis: Which pupils are the world’s biggest 'bullshitters'?

Research: London students do less well in Pisa rankings than GCSE

View: 'It is ridiculous that Pisa is now making teachers anxious'

The country is the only English-speaking western participant in Pisa able to compete on reading, science and mathematics scores with east Asian nations such as South Korea and Japan.

In 2015, Canada was ranked 10th in maths, seventh in science and third in reading. 

Its performance has been applauded and attributed to various factors, from high student motivation to the strong academic performance of immigrants, Jerrim explains.

But differences in the sample due to higher exclusions or absences can fundamentally affect the pillar of the Pisa exercise, which is to provide a representation of each country’s 15-year-old population, Professor Jerrim said. 

“I believe it could substantially undermine the Canadian results,” Professor Jerrim commented.

Professor Jerrim found three possible reasons why Canada’s participation rate was lower.

First of all, students with special educational needs were more likely to be excluded from taking the Pisa assessment: Professor Jerrim reported a 7.5% exclusion rate in Canada compared to 2.4% in Japan and less than 1% in South Korea.

Absence rates were also much higher than in Japan and South Korea: official figures show that one in five Canadian teenagers were counted as absent on the day of the Pisa test, compared less than 3% in the other two countries.

Canadian schools were also more likely to refuse taking part than schools in other countries. The province of Quebec, in particular, was prominent in the Canadian national report as less than half of its schools approached for Pisa agreed to take part.

“Together, this adds up to a sizeable problem, which I believe significantly undermines our confidence in the PISA 2015 data for Canada,” Professor Jerrim commented.

“I believe that there are particular problems in drawing comparisons to other “high-performing” countries – Japan and South Korea in our example – where a genuinely representative cross-section of children took part.”

Log in or register for FREE to continue reading.

It only takes a moment and you'll get access to more news, plus courses, jobs and teaching resources tailored to you

Latest stories

SEND support: How NQTs can support SEND pupils in mainstream schools

An NQT's guide to supporting SEND pupils

As an NQT, you’re responsible for the progress of all, including SEND pupils – here Lauran Hampshire-Dell offers advice

Lauran Hampshire-Dell 24 Jan 2020
teacher desk

How to create the perfect teacher desk

Does your desk look like a dream workspace or a dumping ground? Check out Grainne Hallahan’s tips to making the most of your table

Grainne Hallahan 24 Jan 2020
Singing in assembly, singing in schools

Why do we forget the power of singing in school?

It can give confidence to troubled pupils, it brings the whole school community together, it has little impact on teacher workload, and it’s cheap. What’s not to like about collective singing?

Jo Brighouse 24 Jan 2020