Education research is being hindered by the “dangerous interdependence” between bodies such as the Education Endowment Foundation and the research companies they use, according to a leading academic.

Durham University’s Professor Stephen Gorard claims that, despite a move to ensure that education reform is more evidence-based, the bodies that oversee evaluations “have picked up too many of the bad aspects of the kind of research they were intended to replace.”

In a new book, The Trials of Evidence-Based Education, he finds that part of the problem is down to an inability or unwillingness among traditional researchers in university departments to carry out evaluations on behalf of organisations such as the EEF, and its US counterpart the Institute of Education Sciences (IES).

In March, Professor Gert Biesta, director of research at Brunel University’s education department, slammed the EEF’s toolkit, which aims to help schools use research evidence to help them spend pupil-premium money, as “extremely misleading and utterly unhelpful”. He said the logic behind its scoring system may work with “pig farming, but not with the complex endeavour of education”.

The book, co-authored by Gorard, states: “The moves towards evidence-based education are welcome and well-intended, and many improvements have been made as a result.

“However, progress has been sketchy for a number of reasons, including continued resistance from university academics, which means that the funding is disproportionately going to bespoke companies and organisations with an unhealthy reliance on repeat ‘business’.”

This, it warns, “can lead to dangerous interdependence” between organisations like the EEF and the external research companies they use.

The book explains: “IES (and EEF in the UK) need the capacity that these organisations offer in order to conduct evaluations, and the organisations need the external funding maintained in order to pay the salary of the staff employed to do the evaluations.

“This might make the organisations more likely to provide what they feel the funder wants (for example, all EEF press releases so far at time of writing have emphasised positive results), and for the funder to condone that docility, so inhibiting the innovation and dissension that blue-chip funding for universities permits and can even encourage.”

However, the EEF denies the claims. Last month, it published results showing that a literacy programme led by teaching assistants that aimed to help pupils who struggle with reading had no impact.

The Trials of Evidence-Based Education criticises the EEF for spending equal amounts of money on ideas that are quickly proved to have no merit, and on large trials of fully-developed projects.

It also criticises the way that many randomised control trials - overseen by the EEF and others - have been “often poorly reported”, omitting key information such as the amount of missing data.

Key issues ‘not properly tested’

Some of the big issues in education - such as the pitfalls and value of setting by ability within schools, and selecting children by ability at a young age - are yet to be fully evaluated, according to the book.

It says: “There are a large number of serious issues that have not been properly evaluated by randomised control trials or similar. Instead, we have a plethora of very small trials about some rather trivial issues, while the big issues continue to be debated fruitlessly over decades because they are not being properly tested.”

Gorard, along with co-authors Beng Huat See and Nadia Siddiqui, has been involved with the move towards evidence-based education research. They suggest several ways in which evaluations could be strengthened, and urge that these improvements are put in place.

The book concludes: “There is an urgent need for decent research in education, and for its users and funders to demand a much higher quality of evaluation. The results of research can have a profound impact on children’s future careers, earnings and happiness.

“Poor quality studies are not just a waste of money. Many people, including teachers, parents and children invest a lot of their time participating in these studies. Policymakers make decisions based on their misleading findings. And young people can be harmed by poor work.”

The EEF was founded in 2011 by social mobility charity The Sutton Trust, with a £125 million grant from the Department for Education. It aims to improve education outcomes, particularly in challenging schools.

Triin Edovald, head of evaluation at the EEF, issued a strong rebuttal to the claims in the book. She said: “Randomised controlled trials were relatively new to British education before the EEF and we’re proud of the role we’ve played - working with schools, academics and not-for-profits - in catapulting the use of evidence up the agenda.

“The EEF funds projects delivered by a wide range of organisations (including commitments of almost £20 million to university-led projects) and uses a wide range of independent teams to evaluate these projects. In fact, of the 137 trials either completed or underway, 80 were evaluated by universities - including 12 by Prof Gorard - and 57 to other research organisations.”

She accepted that there was always “a balance to be struck” between academic rigour and ensuring that findings were read and used by school leaders, teachers and policymakers, adding: “Too much academic research in the past has lingered on library shelves, quoted only by other academics and having little classroom impact.”

EEF press releases “highlight the results we think are most likely to be of interest to schools”. But, she said, they “always include details of positive, negative and null results”. All EEF reports go through a rigorous peer-review process and the results of every EEF evaluation is reported in full on its website, she added.

Want to keep up with the latest education news and opinion? Follow Tes on Twitter and like Tes on Facebook