A government-backed research body is reviewing the “validity and security” of all the individual studies that contribute to its highly influential summaries of evidence for teachers.

The decision by the Education Endowment Foundation (EEF) comes amid concerns that education research is facing an “existential crisis” that has “terrifying” implications.

The field of psychology has already been shaken by the replication crisis, which arose when researchers found they could not replicate the findings of key pieces of research that had influence policy and spending decisions.

Quick read: Education research ‘must get its house in order’

Yidan Prize: Meet the $4m academic who revolutionised education research

Profile: Meet the educational research chief in awe of schools

Larry Hedges, who last year won the $4 million Yidan Prize for educational research, told Tes: “It’s pretty clear that at some point this replication crisis is going to be observed in education if we don’t head it off.”

He added: “It is terrifying, and it should terrify us. When I talk about this very issue to education researchers, what I have said is that this is an existential crisis for us.”

Jeremy Hodgen, professor of mathematics education at the UCL Institute of Education, told Tes he “agrees there is a replication crisis”.

He went further, warning that many education research papers do not publish enough detail about their research to allow others to attempt to replicate their findings.

He added: “Sometimes it is difficult to figure out exactly what is going on, and that then makes replication quite difficult, let alone a direct replication. I think there are some real problems, actually.”

The EEF publishes a teaching and learning toolkit, which is designed to provide “accessible summaries of educational research to guide teachers and senior leaders on how to use your resources to improve learning outcomes”.

It covers areas including feedback, homework, learning styles, performance pay, phonics and setting or streaming.

Now, its chief executive Sir Kevan Collins has told Tes that although he views the problem of replication as a challenge rather than a crisis, the EEF is revisiting, one-by-one, the thousands of pieces of research that feed into those overall summaries.

He said: “We are unzipping, if you like, all the individual studies in the meta-analysis, and re-coding them individually, one-by-one, so that we can try and check the validity and security of those individual studies.

“That’s a huge task, but it’s all in the quest of trying to make sure that the information is reliable.”

However, he said the EEF already has “world leading” safeguards in place to address the challenge of replication.

He said these include all its studies being delivered by people with no vested interest in the outcome; every result being published; studies being carried out in large numbers of schools, and the toolkit being based on aggregates of studies.

He added that while there are “risks around replication”, he was convinced that using current evidence “outweighs the risk of just going by gut or by prejudice, which is the alternative”.

Mary Bousted, joint general secretary of the NEU teaching union, described the EEF’s toolkit as a “really important resource for anybody who’s interested in improving their practice as a leader or a teacher”.

She told Tes that the EEF’s decision to review the studies was “very necessary”, and added: “For any research organisation that creates or collates research as well as commissions research, it’s good practice to revisit your findings to see whether they are secure or need to be looked at again.”

Geoff Barton, general secretary of the Association of School and College Leaders, said: “It is vital that the research provided to schools is reliable and this is a prudent move by the EEF to double check the validity of studies.

“It does highlight the point that while it is important to utilise research in our decision-making we should be careful not to over-rely on a single study.”

You can read more about the crisis facing education research in this week’s Tes magazine, available in all good newsagents. To download the digital edition, Android users can click here and iOS users can click here