Depending on the paper or magazine you read, last week you might have heard that Scottish pupils “perform well” in a new international test of group problem-solving, that there is a yawning attainment gap between rich and poor, or that Scottish school children lag behind their English peers.
Here, we give a fuller picture and reveal the concerns about the test itself - devised by the Organisation for Economic Co-operation and Development (OECD) and the first large-scale test of its kind.
What did the test find?
Scottish 15-year-olds, who were tested on group problem-solving as part of the 2015 Programme for International Student Assessment (Pisa), did lag behind English pupils, but they performed better than average.
The 953 students in 109 schools who were assessed achieved a mean score of 513, which Scottish government analysts said put us on a par with the UK overall (519) and Northern Ireland (514), above Wales (496) and the OECD average (500), but below England (521). Topping the league was Singapore with 561. Bottom was Tunisia with 382.
And the attainment gap?
The Scottish government report on the results said the difference between the most disadvantaged and least disadvantaged students was equivalent to “nearly two and a half years’ education”. The two groups were apart by nearly 74 points, the report said, with “a year’s schooling…equivalent to about 30 points”.
However, it also said that “Scotland’s pupils were more likely to break away from the pattern of background affecting performance.” The amount of variation in score explained by social background was 5.6 in Scotland, compared with an OECD average of 7.9 points.
Anything else?
Girls in Scotland performed better than boys - which was the case “in every participating country and economy”. The gap between the sexes in Scotland was 33 points, which was similar to the OECD average of 29 points.
A smaller proportion of Scottish pupils performed really badly when compared with the OECD average - and a higher proportion performed really well.
Teenagers from immigrant backgrounds - where both parents were born outside the UK - performed just as well as non-immigrant students in Scotland.
How did other countries perform?
Australia, Canada, Germany and Finland performed better than Scotland, while France, Norway, Italy and Spain performed worse. Austria, the Netherlands, US and Denmark performed roughly the same as Scotland.
Why are the results important?
The OECD pointed out in its full report that “workplaces around the globe are demanding - and paying higher wages for - people with well-honed social skills”. But it also said that, on average across OECD countries, not even one in 10 students can handle problem-solving tasks that require them to “maintain awareness of group dynamics, take the initiative to overcome obstacles - and resolve disagreements and conflicts”.
Andreas Schleicher, the OECD’s director for education and skills, said that pupils “typically learn individually and at the end of the school year, we certify their individual achievements”, but schools “need to become better at preparing students to live and work in a world in which most people will need to collaborate”. It is, he continued, the “evolution of digital assessment technologies” that has allowed Pisa to carry out “the world’s first international assessment of collaborative problem-solving”. However, the use of technology to assess these skills is proving controversial.
Why is it controversial?
Students were asked to collaborate with computers - not humans. So the teenagers were asked to undertake tasks and the computer provided a number of simulated partners who took on different roles.
The fact that the tests did not involve human-to-human interaction has led some to question their validity - including the general secretary of the EIS teaching union, Larry Flanagan.
Mr Flanagan dismissed the idea that “human interaction could be replaced by a computer-based test”. He added: “Human relationships are the key to effective learning and teaching and these cannot and should not be replaced by technology, no matter how sophisticated it may be.”
What was the OECD’s response?
It addressed the issue when reporting the test results. It argued that because of the changing face of communication in the 21st century, the collaborative problem-solving assessment was “particularly pertinent”, given that “more and more collaboration takes place in virtual settings” and “people find themselves increasingly working with others located in different companies and organisations, and in other cities and countries”.
The OECD also pointed to research carried out by the University of Luxembourg, which found that students’ performance in a virtual, computer-based setting, was “a moderately good predictor of their performance in the face-to-face collaboration”.
@Emma_Seith
