Lack of global data may be damaging Scotland

25th September 2015 at 01:00
Quitting international surveys was a mistake, expert says

The Scottish government’s decision to withdraw from a major international maths and science survey may have been a grave error that paved the way for ineffectual teaching techniques to become commonplace, research has found.

The findings question the usefulness of many established classroom practices involving, for example, formative assessment, group work, active learning and technology.

Tom Macintyre, senior lecturer in mathematics education at the University of Edinburgh, analysed data for nearly 7,000 P5 and S2 pupils from the Trends in International Maths and Science Study (Timss) 2007 – the last year Scotland took part.

His findings cast doubt on the decision in 2010 by former education secretary Michael Russell to withdraw Scotland from Timss and the Progress in International Reading Literacy Study (Pirls), to save £800,000 and “reduce the burden on schools”.

Critics argued at the time that these surveys helped establish not only how well Scottish education was doing, but what worked in classrooms. Dr Macintyre’s research, presented at the annual conference of the British Educational Research Association (Bera) in Belfast last week, backs that view, indicating that some preoccupations of Scottish policymakers and school leaders may not have had the desired impact.

Going through the motions?

Scotland still takes part in one major international survey of educational standards, the Programme for International Student Assessment (Pisa), which deals with 15-year-olds. But Dr Macintyre finds it “disappointing” that the government believes a focus on older students is enough.

“It would be considerably richer to have the opportunity to explore robust, internationally gathered data on learners’ experiences and achievements while still in their primary and early secondary stages of school,” he writes in his research.

His analysis of Timss data looks at the use of quizzes or tests to give pupils feedback on how to improve their learning – a process central to formative assessment and the Assessment is for Learning (AifL) approach that is widely used in Scotland.

“This raises the question as to whether teachers are just ticking a box in terms of feedback: are they just going through the motions in setting quizzes, rather than using them more carefully to inform what happens next in lessons?” he said.

Dr Macintyre has also linked heavy use of computers with lower scores in maths tests. As a result, he said, equipment such as tablets risked becoming “gimmicks” because they often appeared in classrooms “without a clear rationale as to how the equipment is to be deployed, and used most effectively”.

Timss data also suggests that active learning, group work, relating learning to everyday life and heavy use of calculators may not be having the desired impact.

Ken Cunningham, former general secretary of School Leaders Scotland, backed the original decision to withdraw and still believes it was the right move. “Timss was desirable in some minds but by no means essential,” he said, adding that any benefits had to be balanced against cost and workload.

He also said there was evidence that a poor Timss ranking led to “knee-jerk reactions of the highest order” from politicians.

‘Ironic’ situation

EIS general secretary Larry Flanagan said the union was “unconcerned about the withdrawal from Timss, or Pirls, as these are effectively standardised testing approaches which lead towards the type of league-table approach we see with Pisa”.

It was “somewhat ironic”, he added, that the Scottish government had decided to introduce standardised testing in schools while avoiding such international programmes.

Sue Ellis, professor of education at the University of Strathclyde, was a critic of the Timss exit. This week she told TESS: “In my view, Timss provided more useful data than the Scottish Survey of Literacy and Numeracy (SSLN), because it gave international comparisons, it generated information that was more focused and more clearly linked to research underpinning developmental trajectories, and it gave us information about science. We undoubtedly lost a potentially useful data source.”

There was a need for data on young children, she said, and it was a shame there had been little analysis on the impact of AifL on attainment in Scotland, as it had been a large financial investment.

A spokeswoman for the Scottish government said: “Since 2000 we have participated in the largest international survey, Pisa, run by the OECD [Organisation for Economic Cooperation and Development]. This provides a more effective indicator of how the Scottish education system is performing relative to other countries. Scotland withdrew from Timss and Pirls to minimise the assessment burden. Unlike Pisa, not every OECD country participates in Timss and Pirls.

“We are also looking forward to the findings of the OECD review of Scottish education, which took place earlier this year. This will provide us with a detailed and independent view.”

‘It’s not necessary to test every pupil’

Tom Macintyre, a senior lecturer in mathematics education at the University of Edinburgh, questions the logic of a new national testing system – controversially announced by first minister Nicola Sturgeon last month – while withdrawing from the Trends in International Maths and Science Study (Timss).

The National Improvement Framework has “a lot of good things”, he says, but “the extra layer of reintroducing national, standardised tests – I just can’t see the rationale”.

It is not necessary to test every pupil to gauge overall progress, he says; a survey that samples smaller numbers, such as Timss or the Scottish Survey of Literacy and Numeracy, can do that at a lower cost with fewer demands on teachers.

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number


The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now