How much heed should we pay the Programme for International Student Assessment (Pisa)? It’s a topic that has been explored in these pages many times, and rightly so, given that any international survey of pupil performance has huge implications for education systems around the world. It is even more pressing in Scotland, where performance has apparently dipped over time.

In today’s Tes Scotland, the outgoing chief executive of the Improvement Service, Colin Mair, becomes Pisa’s latest critic.

In the past, academics have criticised the “meaningless” rankings and some, including Yong Zhao, a prominent professor of education at the University of Kansas in the US, have argued that Pisa should be ignored entirely because it stifles creativity and creates homogenous education systems.

For his part, Mair argues that Pisa has “all the strengths and all the limitations of an international survey”, and that Scotland should have more faith in its own data, which shows that the education system in recent times has produced “the best-qualified generation of Scots, without a doubt”.

Pisa is not pegged to a specific curriculum; instead, it measures core international competencies in reading, maths and science. As such, Mair argues that our own exam system is the best measure of whether or not Scottish pupils are leaving school with the knowledge and skills we have collectively deemed important, not Pisa.

According to national data, the overall achievement of pupils at the time they leave school has increased by 15.1 per cent since 2011-12. For pupils in the most deprived areas, it has risen by 30.5 per cent, and for those living in the least deprived areas, it has increased by 9.6 per cent.

Mair points out that because Scotland takes part in Pisa as a region of the UK, the sample size is approximately 3,000 pupils - about 5 per cent of the whole pupil population at the required age and stage, and well below the minimum national requirement to participate in Pisa: 5,250 pupils in 150 schools.

He also points out that while all 3,000 students completed science questions in 2015, only 41 per cent of the sample sat reading and maths tests - about 1,265 pupils. This matters, he says, because when you are applying sophisticated statistical manipulation, the size of the sample is what gives you confidence that the results from the survey would be the same as those in the population.

However, it is probably unfair to say that those who judge the Scottish education system to be latterly underperforming are basing their view on Pisa findings alone. The SSLN - Scotland’s own survey of literacy and numeracy performance, scrapped last year - also showed a downward trend in pupil performance. And faith in the education system has been rocked by recent changes, including the poor implementation of the new exams, which have had to be reformed after the fact. The credibility of some of the new assessments - specifically National 4 - has been questioned by parents and teachers, too.

So, to say that the perception that all is not well in Scottish education is down to Pisa alone would be to oversimplify. But it is refreshing to see someone from outside of the teaching profession standing up for Scottish education, with passion, giving credit to teachers and pupils for their achievements, as well as urging Scotland to look at Pisa in the round and with a critical eye.

In short, Mair is making a plea for balance - for a narrative that acknowledges the system is performing well, but also needs to be improved. Such a measured and cool-headed assessment seems less likely to lead to the kind of impulsive, untested and ill-thought-through change that could do more harm than good to Scotland’s Pisa rankings - and to its pupils.

@Emma_Seith