.Sally Thomas, a professor of education at the University of Bristol, agrees that replication is vital.
“If you have 10 studies pointing in one direction, that’s obviously stronger than one study showing something,” she said. “Like in legal cases, where you’re building evidence to make a convincing argument, the more evidence you have, the more convincing the outcome will be.”
The problem with judging an intervention based on a single study is further highlighted by Bette Chambers, director of the Institute for Effective Education at the University of York.
One might, for example, be looking at a reading intervention intended to be implemented by trained teachers for three hours a week. However, the schools involved in the initial pilot study might be able to spare only one hour a week, or a shortage of teaching staff might lead them to delegate the initiative to teaching assistants.
“So, when you come up with results, you don’t know if the programme didn’t work or if it wasn’t implemented well,” Professor Chambers said.
By conducting multiple studies, however, academics can determine how effective a particular intervention is on average.
According to Kevan Collins, chief executive of the Education Endowment Foundation (EEF), a charity that funds educational research, repetition is vital even when an initial study has been conducted properly.
Dr Collins cited a new writing intervention that had proven very effective in 20 initial schools. “But then you have to ask the question: is it just the people who led it?” he said. “The authors are really inspiring people - lots of education is built on gurus being inspiring.”
`Ethical obligation’
The next step, therefore, would be to ask teachers to implement the programme - this time in many more schools. “Only then can you say to a teacher in, say, Newcastle: `This is really worth your time and money’,” Dr Collins argued. “You’ve got to test things in a range of settings and contexts. It’s an ethical obligation in my view.”
He pointed out that other industries were governed by stringent rules. “If you want to create a new lipstick to go on the high street, by regulation you have to go through a whole lot of hurdles to prove it’s not going to do any harm,” he said.
By contrast, he added, programmes that claimed to be based on neuroscientific fact could be sold to schools without any evidence-based studies to back them up.
Inevitably, this comes down to money. “If there isn’t replication in education research, it’s because it’s not funded,” Professor Thomas said.
Professor Chambers echoed this view. “For a long time, education has been perceived as a craft instead of a science,” she said. “And, because everybody’s been to school, everybody claims to know what’s best for schools.”
Nonetheless, Dr Collins argued, organisations such as the EEF and Professor Chambers’ institute were increasingly emphasising that changes in education policy must be based on rigorous research. “There’s a degree of science and understanding in how these things work,” he said. “It needs to be applied in education as much as in anything else. More so, in fact.”
Call for open access to academic papers
All academic research should be free for teachers to access, according to a higher education thinktank.
The Higher Education Policy Institute (Hepi) launched a report this week arguing that a new national licence should make the latest academic journals available to anyone who wants to read them (bit.lyHepiAccess).
“The UK is a world leader in academic research,” says Nick Hillman, director of Hepi. “Yet access to the latest work is severely restricted. For those outside universities, including public sector workers, it is not easy to find out what is happening at the forefront of knowledge.”
Some research journals offer “open access”, allowing non-academics to read articles for free. Most, however, have a hefty subscription fee, which is usually paid by universities on behalf of academics.
The national licence would allow any UK resident to access the latest academic journals for free. Hepi says this would benefit teachers, teacher-trainers and policymakers.
