Beware the one-armed bandits of research
“Next time you have a vacancy, employ a one-armed researcher.” Sir Ashley Bramall, leader of the now-defunct Inner London Education Authority (Ilea), couldn’t have been clearer in his instructions.
It was the mid-1970s, and among my duties as the Ilea’s deputy education officer was looking after its research and statistics branch. “Whenever I ask what the research says,” Sir Ashley complained with a smile, “your reply is always, ‘Well, on the one hand this, and on the other hand that’.”
Of course, I sympathised. I have thought of it often since, especially when politicians promise evidence-based policymaking, so as to appear to get away from the anecdotal evidence and prejudices of their political advisers. They then seem to choose the research evidence that suits their opinions.
But even when politicians really try, it’s complicated. All of them should have a copy of Nancy Cartwright and Jeremy Hardie’s Evidence-Based Policy: a practical guide to doing it better. It would introduce them to the hazards of “horizontal” factors – all contexts are subtly different – and “vertical” challenges: the difficulty of analysing how an apparently successful policy or practice should be implemented when repeated. What works in one place will often fail in another.
Michael Gove, who seems to be doing better at the Ministry of Justice than he did in education (do I hear you say, “Not too difficult”?), was a keen advocate of research-based evidence. Not for himself, of course, but for teachers and schools. So he turned to Ben Goldacre, who has done so much to improve medical use of research and is an advocate of randomised controlled trials (RCTs). Ben’s influence at the Education Endowment Foundation is clear.
But education is not medicine. In education, much can depend on getting the right person in the right place, and doing the right things at the right time. Sometimes, even if you have all of that, it still doesn’t work. So I am not holding my breath for RCTs in schools.
On the other hand, there are some really helpful meta-analyses of what works in schools. Some are based on evidence plus Ofsted corroboration – for example, peer-tutoring, mentoring and counselling. A few schools make research a key ingredient of their improvement strategy. Each year, some award bursaries to pairs of teachers to carry out action research. Others have a visiting researcher from a university. Many primaries have replaced projects with research topics for their half-term curricular themes.
Huntington School in York has a research lead, Alex Quigley, whose task includes checking any decisions made by the senior leadership team against the evidence. His checklist includes “having a good understanding of the support factors that might make an intervention work in your context” and “evaluating any project/decision and not hesitating to tinker with support factors and stop if it isn’t working”. One check struck a nerve: “Don’t cherry-pick the evidence to prove you are right.”
I’m all in favour of schools taking research seriously, because they are too sensible to consult one-armed researchers. Who, in any case, are all employed by politicians.
Sir Tim Brighouse is a former schools commissioner for London