If we are honest, most of the shiny initiatives we start in the frantic hamster-wheel of school improvement don’t make much difference. Hopefully, they are a little bit better than what came before. But, despite our best intentions, quite often they are not. In fact, only about 20 per cent of the trials that the Education Endowment Foundation (EEF) commissions have a positive outcome in favour of the idea.

Despite this, a lot can be learned from every evaluation. They can point to what not to do (at least in the way that was tested), as much as what we should do.

Take the EEF Promoting Alternative Thinking Strategies (Paths) study, for example. This approach, developed in the US, aims at promoting the social and emotional learning of primary-aged children. It has been evaluated in many studies, with some suggesting a positive effect on academic attainment.

In 2011, a group of evaluators at the University of Manchester designed a randomised controlled trial to study the impact of the Paths programme on children’s social and emotional development. The original trial was funded by the National Institute of Child Health and the EEF joined it to add another measure: the impact on academic outcomes in reading and writing.

Sense of achievement

The study was conducted over two years in 45 schools in Greater Manchester with children in Years 5 and 6. The outcomes suggested that the programme significantly improved children’s psychological wellbeing and their health-related quality of life, but had no positive, detectable effect on the academic outcomes of the children - and, in some cases, had a negative impact.

As an approach to developing psychological wellbeing, Paths is worth considering. Nevertheless, if you are looking to improve outcomes in reading and writing in Years 5 and 6, it probably should not be top of the list of options to try.

Yet, beyond this headline finding, the data collected revealed some further interesting insights regarding the way Paths was being delivered. The Paths intervention is a “manualised” intervention with highly structured materials and scripts provided by the developers. Teachers who are using it are encouraged to follow those plans closely, based on the belief that close adherence to plans will provide the maximum effects.

Flipping the script

However, the University of Manchester evaluation team found that, in this trial, this did not hold true. The teachers who had the greatest impact were the ones who showed “moderate” fidelity to the materials but had a deep and secure understanding of the key principles of the intervention - the quality of their work and their responsiveness to the children was just as important.

In other words, rather than slavishly following all the scripts, they were faithful to the underlying principles of the programme, and adapted and presented the materials accordingly to respond to the children in front of them.

It is easy to forget that, although the script, workbook and guides for practice can be an important part of a successful programme, they are simply the resources we draw upon to inform and support. Another essential aspect is the teacher and how they draw upon and mediate these materials to develop their classroom practice.

Teachers - and teaching - matter. It might be worth putting less time into the resources and more time into supporting teachers to develop their classroom skills. Teachers: they are absolutely worth it.

Megan Dixon is director of literacy at the Aspire Educational Trust

This article originally appeared in the 8 March 2019 issue under the headline “How we can build on our mistakes”