The idea that teaching should be a research-based profession is attractive for a number of reasons. Unfortunately, it is never going to happen.

That may seem like a rather bleak, sweeping statement. Certainly, turning teaching into a research-led profession would improve student achievement, because all teachers would be using the most effective methods. It would also raise the status of the profession, thus helping to encourage the “brightest and best” to become teachers, which many believe would lead to further gains in student achievement.

But for teaching to become a research-based profession, staff would need to be able to find and access credible research studies that show which courses of action will lead to better outcomes for their particular students. Furthermore, for teaching to be truly research-led, this would have to be the case for most of the decisions made by teachers. We need to accept that this isn’t likely to be possible.

The long view

I’m definitely not saying that research has no role to play in education. In recent years, researchers have made extraordinary progress in determining the best ways of synthesising research from different sources. Although individual studies can produce conflicting results, researchers can make incredibly useful discoveries by analysing different studies in a systematic way. In medicine, systematic reviews of research have saved tens of thousands of lives: researchers have found that sudden infant death is much less likely if babies sleep on their backs, and that some antiarrhythmic drugs actually increase the risk of sudden death in patients who have suffered heart attacks.

Much of the success in the medical field has been down to the Cochrane Collaboration, a not-for-profit organisation that promotes and gives access to systematic reviews of medical research. In the past decade or so, researchers have tried to mimic this work in the social sciences via the Campbell Collaboration.

Similarly, the Education Endowment Foundation (EEF) has used a systematic review process to produce a Teaching and Learning Toolkit, which helps schools to identify the changes that are likely to be the most effective. These reviews are enormously useful in helping schools to direct their efforts. Some strategies that look like they really should make a difference, such as matching teaching to students’ preferred learning styles, turn out to have no benefit. And others that appear slightly old-fashioned, such as regular testing, turn out to be highly beneficial.

But is this as helpful as it seems? Two issues with systematic reviews mean their usefulness in education is limited.

1 Research can’t tell you what could be

One problem with systematic reviews is that they only tell you what was, not what may be. For example, the EEF’s Teaching and Learning Toolkit indicates that grouping students by ability has a net negative effect on achievement. This is probably a fair summary of the research that has been undertaken so far. Grouping students by ability produces gains for high achievers at the expense of losses for low achievers, and since the gains tend to be smaller than the losses, the net effect is to slightly lower the average achievement and to increase the range.

But it is important to bear in mind that this may be caused by the way in which students are grouped, rather than anything inherent in the idea of ability grouping. In the past, when students were grouped by ability, it was typical for the best teachers to be assigned to the highest achieving students. More recently, in England, secondary schools have assigned the best teachers to students on the C-D borderline at GCSE, since that is likely to have the greatest impact on the school’s position in the league tables.

However, as the work of Simon Burgess and his colleagues at the University of Bristol has shown, the most effective teachers have the biggest impact with low achievers. In other words, it could be that grouping students by ability and then assigning the best teachers to low achievers would actually increase average student achievement. The losses among high achievers from being taught by average rather than excellent teachers would be smaller than the gains made by low achievers from being taught by the best. The point is that a different way of grouping students by ability may actually have a net positive impact on achievement, but the research has not yet been done.

Similar arguments apply in the case of class sizes. We know from extensive studies conducted all over the world that reducing class sizes generally leads to only small increases in student achievement. But teachers in most of these studies were given little support to take advantage of the teaching strategies that are possible with a smaller class. If educators teach a class of 15 in the same way that they teach a class of 30, it isn’t surprising that student achievement doesn’t improve much. As Peter Blatchford, my colleague at the UCL Institute of Education, has shown, smaller class sizes allow teachers to work in alternative ways - ways that are likely to be much more effective - but this happens only when a reduction in class size is accompanied by professional development.

Research can tell you only what was, not what could be. We take the findings as truth but the reality is much more complex.

2 Research is rarely clear enough to guide action

The second problem with applying research findings in practice is that although a strategy might improve learning on average, the effects can vary quite a bit in practice.

For example, the EEF’s Teaching and Learning Toolkit indicates that providing feedback is one of the most effective ways to improve student achievement. In their review of almost a century of research on feedback, Avraham Kluger and Angelo DeNisi find that it does indeed produce substantial improvements in learning on average (bit.lyKlugerDeNisi). However, they also find that in 38 per cent of cases, feedback actually leads to worse performance - one of the most counter-intuitive results in all psychology.

So, telling teachers to give more feedback may improve learning, but unless we understand what kinds of feedback are effective, there is a real danger that teachers will give more of the type that actually lowers achievement.

In response to challenges such as this, researchers usually say that more research is needed (they would, wouldn’t they?). And in the case of feedback, they have undertaken more research: they have tried to find out whether feedback is better given orally or in writing; whether it should be immediate or delayed, specific or general. But, as Kluger and DeNisi point out, these features of feedback are almost irrelevant. The only thing that matters about feedback is what the recipient does with it. And that, in turn, depends on the relationship between student and teacher.

Every teacher knows that the feedback that can make one student try harder can make another student, with a similar level of achievement, want to give up. To maximise the effect of feedback, teachers need to know their students. They need to know when to push and when to back off. And students need to trust their teachers. If a student feels a teacher doesn’t have their best interests at heart - or doesn’t know what they’re talking about - they are unlikely to invest effort in taking the feedback on board.

The problem is that hardly any of the thousands of studies of feedback conducted in the past hundred years have taken any account of the relationship between the donor and the recipient of feedback. Hence, whatever can be learned from these studies, it is highly unlikely that it will apply to schools, where teachers have continuing relationships with students.

Feedback is one area of many where this issue of practical application undermines the findings of research. There are no findings that will work for every teacher in every situation.

Underpinning all these issues is the fact that we have got our central question wrong. Politicians and educators want to find “what works”. But the simple truth is that, in education, everything works somewhere and nothing works everywhere.

A much better question is: under what circumstances does this work? Perhaps one day we will have a way of describing school settings in such detail that we can say, for example, that if a certain set of conditions are met then grouping students by ability is the best thing to do, but if these conditions are not met then it is better to teach in mixed-ability groups. Sounds great, right?

But think about the three schools nearest to you, as well as your own. Try to define them and to come up with the conditions you would stipulate. You will quickly find that the complexity of schools, and the fact that they are full of unpredictable humans, means this utopian vision of context-tailored research is a mirage.

This does not mean research is irrelevant, but it does mean we have to look at other ways of improving the profession. The best way to become a more effective teaching body is to build and trust the “practical wisdom” of teachers. Of course, this expertise is likely to be improved by knowledge of any relevant research that has been carried out, but we can’t rely on that research alone.

Hence, one of the characteristics of expertise is knowing when to follow the rules and when to break them - when to ignore the research. Experts are able to recognise deep and meaningful patterns in what they see and to quickly identify appropriate courses of action. But they are rarely able to articulate how they do this. As academic Michael Polanyi said, “We can know more than we can tell.”

Teachers are individuals, too

So how do we build expertise in teachers? Research on expertise in general indicates that it is the result of at least 10 years of deliberate practice - practice that is neither motivating nor enjoyable, but is instrumental in improving performance.

Unfortunately, the research on teaching shows that although teachers improve rapidly in their first two years, all teachers slow and many stop improving after three years (see panel, page 21). In other words, many, if not most, of our teachers could be even better than they are now.

We need to arrest that slowdown in improvement by facilitating access to effective professional development. Teachers should seek to improve their practice through careful reflection. They need to explore different ways of meeting the needs of their students, and try to evaluate (to the extent that this is possible) whether new approaches are more successful than previous ones. This should not come from an insinuation that the teacher is “not good enough”, rather that they have the potential to be even better.

We also need to accept that teachers improve in different ways. Some teachers may wish to share their work with colleagues at conferences or professional development sessions, for example. But although all teachers should be encouraged to do this (and I hope that most would want to), some may feel their time is better spent in other ways. Some may wish to prepare their findings for publication; others may wish to pursue degrees and publish their work in research journals. These are all worthwhile activities with equal footing.

What we can’t do is insist that teachers follow one path: the only “non-negotiable” aspect of every teacher’s contract should be a requirement to continue to improve their practice.

The desire for teaching to become a research-based profession is a noble one, but ultimately it will not achieve the result that its advocates want. Instead we need to focus on teacher expertise, of which research is one of many parts. We need to support current teachers to improve their practice and inform their judgement. Then we need to trust that judgement and encourage people with a real passion for working with young people to become teachers - the ones who will push themselves to get better when no one else is pushing them.

There is no limit to what our nation’s teachers can achieve, but relying on research alone will not help them to reach their potential. The best thing we can do is to support them to be the best they can be in the right way for them as individuals.

Dylan Wiliam is emeritus professor of educational assessment at the UCL Institute of Education and the author of publications including Inside the Black Box and Assessment For Learning

Do teachers stop improving after three years?

The research behind Dylan Wiliam’s comment that “although teachers improve rapidly in their first two years, all teachers slow and many stop improving after three years” (see page 22) is published in Teachers, Schools, and Academic Achievement by Steven G Rivkin, Eric A Hanushek and John F Kain (see bit.lyTeacherImprovement).

It’s quite a claim, so how do teachers and educationalists respond to it?

David Weston, chief executive of the Teacher Development Trust, says: “Wiliam quotes research that analyses a very specific set of tests in Texas, showing that teacher experience past the first few years does not correlate with improved scores. On the other hand, Andrew Leigh’s research finds a positive correlation between teacher experience and test outcomes in Australia, for up to 20 years’ experience. M A Kraft and J P Papay find that the positive impact of teacher experience on test scores in a district of North Carolina slows - not plateaus - after three years, but is strongly affected thereafter by the quality of professional development support and school environment.”

Sinead Gaffney, a primary school teacher based in Sheffield, responds: “I qualified in 2000. By 2003, I was on top of the things required to look and sound like a real teacher. It was only then that I began to have the breathing space to reflect on my practice and make improvements. We have to improve just to stand still in this job.”

Meanwhile, David Hall, associate headteacher of Bay House School in Hampshire, says: “Teachers early in their careers are often more closely supported through mentoring and are likely to look to their peers for advice and guidance. It is not surprising that they show improvement. However, I would challenge the idea that other teachers stop improving. I have seen long-serving staff re-energised by high-quality professional learning and by pursuing postgraduate qualifications. Teachers at all stages can thrive in a culture that encourages them to learn.”

And Hayley Ryan, head of teaching standards and innovation at a Hampshire FE college, says: “This claim goes against everything I have experienced teaching to be. The end of a teacher’s formal training is not the end of their learning process. Teachers constantly seek to improve their impact on learner progress, experimenting with new resources, ideologies and practice.”