Research about education is a bit like the news in newspapers. Genuinely informative, objective, useful accounts occasionally rise above reams of prose far better suited for wrapping fish and chips.
The research landscape is a complex mixture containing everything from high-quality, peer-reviewed, sincere, investigative effort by experienced academics at one end, to crass marketing and self-publicity at the other.
But more significantly, in between sits a horde of interested organisations eager to influence schools and teachers. Businesses, charities, NGOs, think-tanks, government agencies all energetically publish glossy, convincing looking material they call research but that never attempts academic standards. Research that sets out on a cause is just lobbying.
This is especially questionable when so many of the organisations doing this kind of lobbying trace much of their funding back to central government.
If you’re set up, like Teach First were for example, to put high-quality graduates into struggling schools, spending money carrying out research that finds pupils in deprived areas are five times more likely to be served by schools rated less than "good" might reasonably be regarded as self-justification.
If teachers are to stand any chance at all of making use of the good stuff then it helps to know how little credibility the rest has, even amongst those keenest to generate it.
My work demands I read this stuff. And I mean read it because, I absolutely assure you, no one else bothers.
Some will flick through the executive summary or the conclusions, marketing departments will cherr pick findings regardless of context, variability or irregularity in the findings, and headline claims will be widely published without anyone, and I mean anyone at all, bothering to get to grips with the entire thing.
Any substantial piece of research is likely to take you a full working day, or more, to read properly.
In the world these influential organisations occupy, it is extraordinarily rare to meet anyone, whatever their role, who will bother to read a serious, comprehensive piece of research from page one to the end.
And even the academic top end has its problems, as is easy to see from recent arguments about the methodology and value of the Education Endowment Fund’s toolkit.
I recently attended an event about the success (or not) of the London Challenge, where two experienced academic researchers presented their findings, followed by a presentation from the former civil servant who had directed the programme. I’ve never seen a graph fight before and, though hugely entertaining, it was no use whatsoever to real teachers.
A project I’m working on at the moment meant I had to read some research published by a leading university, which had been commissioned by the Department for International Development (DFID), who have a set of ethical principles and guidelines they expect anyone using research evidence to follow.
When I read this paper in full, which was admittedly rather good, it concluded by recommending less than a handful of specific international educational projects it had examined. One of those projects had not only been run by the university, but was actually directed by the lead researcher who had published the paper.
I guess they hadn’t read the DFID ethical guidelines, which state, "research and evaluation should usually be independent of those implementing an intervention or programme under study", or else they applied a bit of special weighting to the word usually.
In the world of educational research, “weighting”, for those of you who don’t know, is a bit like “irregularity” in a budget.
So what can you do to negotiate this rocky landscape? These are my top tips based on years of having to read this stuff.
1. Before reading something in full, always take a good look online at the individual author’s background. Their Twitter feed or LinkedIn profiles are especially helpful.
If, as I recently discovered, you find that the huge and impressively long list of research papers an academic has published in peer-reviewed, credible journals, has almost all been commissioned by one business, then you may wish to adjust the confidence you place in their findings accordingly. Quite often that extra little effort will render reading the stuff pointless anyway and save you precious time.
2. If the research stems from an organisation, take a thorough look online at their vision and funding, as well as at the individuals leading them.
Quite often that will immediately explain why they have produced the headline findings that sent you there in the first place. Sometimes it will expose a political connection you weren’t aware of, but most often it will simply expose what they are saying as lobbying or marketing.
3. Listen to what other teachers you respect are saying about research or read what they are writing and discussing.
The key here is that word respect. It’s not much use sharing views with a self-selecting clique of mates. You only have to read the comments on Facebook some teachers make to realise that. But it is valuable sharing information with other professionals whose work and skill you admire.
There are some gems out there amidst all that dross that can help you do a better job, but gems are valuable for a reason.
Joe Nutt is an educational consultant and author
To read more columns by Joe, view his back catalogue