Cautionary tale of Timss, Pisa and Sats

12th December 2008 at 00:00
Professor Alan Smithers Director of the Centre for Education and Employment Research at the University of Buckingham

At last there may be some good news for the Government on the educational comparisons front.

But there is need for caution. The only other time when we showed up well in maths, in Pisa 2000 - much celebrated by ministers - it turned out to be an artefact due to the over-representation of high-performing schools among those that responded.

The latest Timss maths results, however, may be more soundly based. They may reflect the working through of the Government's numeracy strategy. The 14-year-olds taking Timss in 2007 were the 10-year-olds in Timss 2003, when they also showed a big improvement.

But there was no sign of this in last year's Pisa results. Timss and Pisa test different things. Timss is more curriculum-based, focusing on mathematical understanding, while Pisa's mathematical literacy is about maths in everyday situations.

Some countries do notably better in one or the other. Hungary stars in Timss, but lags behind in Pisa. For Australia, it is the reverse. It is possible that Timss better reflects the aims of the numeracy strategy.

The tests are certainly more similar to our Sats, having been put together by the same people.

The country coverage of Pisa and Timss is also very different. Only 11 of the 30 Organisation for Economic Co-operation and Development countries appear in Timss, with big hitters such as Finland, Flemish Belgium and Canada absent, boosting our relative position.

But England is well ahead of Sweden - often held up as an example - and Scotland.

It will take Pisa 2009 and Timss 2011 to show whether this week's results are another blip or the beginning of a welcome trend.


Timss was developed by the International Association for the Evaluation of Educational Achievement (IEA) and is run by academics in Boston, Massachusetts.

The tests have taken place every four years since 1995.

This time, 425,000 pupils were tested in 59 countries, four Canadian provinces, two American states and the city of Dubai.

The science study tests primary children in biology, chemistry and physics, and secondary pupils in those subjects plus earth science.

In maths, they are tested in algebra, number, geometry and data analysis.

Expert observers say that while the rival Pisa study tests pupils' ability to apply their knowledge, Timss is a more direct test of pupils' mastery of curriculum content.


English pupils appear to do less homework than their counterparts almost anywhere else, the Timss study found. Pupils were asked how much homework they did in the subject of the test they took. English children were in the bottom five countries in all four questionnaires. But it is not clear that this affected their achievement. In the primary questionnaires, those spending less time on homework tended to do better in the tests. In secondary, the reverse was true in maths, while in science there was no clear correlation. The Timss report said that brighter children might complete homework quicker, while those who were struggling might be given more work to help them catch up.

Timss appears to provide little evidence to support the view that cutting class sizes will raise test performance. The country with the best results in primary science, Singapore, also has the largest classes: 95 per cent of pupils are in classes of more than 32.

Pupils were asked how many books there are in their homes. Generally, the more books, the better the pupils' scores in all tests.

English teachers generally rate their schools as having a good environment for learning, the study found. One of the Timss indicators is "perceptions of school climate", in which heads and teachers are asked to rate their schools on eight factors, including teachers' job satisfaction, parents' involvement and pupil motivation.

England finished towards the top of the tables in both teachers' and heads' views.

Timss also found that teaching is no longer an ageing profession - at least in primary schools. Some 31 per cent of English primary teachers in the tested schools were aged 29 or under, as opposed to only 16 per cent aged 50 or over. In secondaries, there was a flat age profile, with roughly the same number of teachers near the beginning as near the end of their careers.

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar,, Virgin Wines and other partners
Order your low-cost subscription today