Why superficial compliance with research is dangerous

Our education system is more research-rich than ever – but evidence lives or dies in the detail, says Becky Francis

Our education system is more research-informed that ever - but superficial compliance is a danger, warns Becky Francis, of the Education Endowment Foundation

The first time you describe your new job at an event is a big moment. What is it that defines your role and organisation?

I’m two months into my new role, as chief executive of the Education Endowment Foundation (EEF) and I am honing my answer.

I’ve decided to describe it as a job of two gaps. The EEF supports schools, colleges and early years settings to close the achievement gap between socially disadvantaged students and their peers. 

We do this by trying to close the evidence gap, ensuring that teachers and school and college leaders have access to the best available evidence on the approaches that will have the greatest impact, in terms of ensuring good practice and equalising opportunities. 

Social mobility

The EEF’s combination of mission and method has proven to be extremely powerful. When it was created, my new colleagues were anxious about whether any schools would want to participate in the projects it funds. 

But their fears were misplaced. Nine years in, the EEF has just published the results from its 100th randomised control trial (RCT).

Together, more than half of England’s schools – 1.5 million pupils – have now taken part in an EEF project. This is a programme of research that is unparalleled anywhere in the world.

For some people, these headlines are enough. But many want to know more. What works? Does all the effort put in to finding out make a difference? 

“What works?” is a simple question, which deserves a complex reply. Context is important and, especially in the case of RCTs, the consistency and quality of implementation also affect the conclusions we can draw. 

Steve Higgins, lead author of the Toolkit, says that we know what worked, but emphasises that there are no guarantees, only better or worse bets.

But without losing this nuance, there is a lot to talk about. Thanks to the collaborative efforts of teachers and researchers, we are learning a huge amount, and quickly.

Is education research making a difference? 

We have learned, for example, that early intervention trumps catch-up, that curriculum change requires a great deal of time and support – attending both to the what and the how – and that a successful pupil-premium strategy starts with great teaching, rather than a long list of interventions.

But, of course, the sharpest query is whether, together, this evidence is making a difference.

I believe that it is, and that we can see this by examining both policy and practice.

At policy level, the pupil premium has been a powerful tool for social justice. As my predecessor, Sir Kevan Collins, has noted, the premium was a victim of its own success, in that the speed with which it was introduced prevented a robust evaluation.

But there is clear evidence that it changed behaviour, and since its introduction the gap has narrowed – by around 9 per cent – in both primary and secondary schools.

We can’t be complacent, as the slight reopening of the gap in the past two years shows, or underplay the challenge that remains. But the sustained efforts of teachers and school and college leaders, against a backdrop of significant funding pressure and wider austerity, must be recognised. 

The biggest threat to education

We also see the impact of evidence on classroom practice. However alluring, the promise of evidence isn’t that we will discover “cures” to low literacy or numeracy.

Instead, our practice will improve incrementally, as insights and improvements accumulate. We see this in the way schools have changed the way they deploy and train teaching assistants, and in the growing interest in metacognition and how we learn.

The truth about the hardest question though – whether research evidence will really help – is that it is down to the profession, and to school and college leaders.

Our system is more evidence-rich than a decade ago. But leaders face a clear choice about the ways in which they use it.

Ironically, as the language of evidence proliferates, there is a risk that it loses its impact. Surface-level compliance is the biggest threat to any change in education.

For evidence, which lives or dies in the detail, that worries me a great deal. We know the stories of triple-marking and the mini-plenary.

Instead, evidence must help to democratise education. The studies the EEF publishes are owned by the profession. Our Toolkit represents a knowledge base built by literally millions of teachers and students around the world. 

Evidence does not provide easy solutions, but evidence-informed improvement is a process that has integrity and holds greater promise than any alternative.

And as we move forward, big questions remain for the sector about our approach to evidence. Will we use evidence for accountability or impact? Who owns evidence?

Our answers will determine whether the evidence’s promise is fulfilled, in our collective endeavours to improve our profession and to support all young people to achieve, irrespective of background.

Professor Becky Francis is chief executive of the Education Endowment Foundation

Professor Francis will be at Association of School and College Leaders' Annual Conference on Saturday 14 March, joining a panel discussion with Natalie Perera and Jonathan Simons. More information here

Register to continue reading for free

It only takes a moment and you'll get access to more news, plus courses, jobs and teaching resources tailored to you

Latest stories

Mark Emmerson, CEO of City of London Academies Trust

DfE behaviour expert ran 'unlawful' exclusions

Exclusive: Mark Emmerson was head of a academy where pupils were voluntarily 'withdrawn' by parents – without these absences being recorded as fixed-term exclusions

Amy Gibbons 28 Feb 2020