If, as research suggests, learning is invisible at the point of teaching, then Dylan Wiliam has a problem. Or rather, his big idea, which has infiltrated nearly every school in the UK, has a problem. Formative assessment seemingly cannot work.

According to Wiliam, emeritus professor of educational assessment at the University of London’s Institute of Education, we should “use evidence about learning to adapt teaching and learning to meet student needs”. This depends on a belief that you can assess what pupils have learned in an individual lesson and adjust future teaching based on this assessment.

However, there’s no meaningful way to assess what pupils have learned during the lesson in which they are supposed to be learning it.

As academics such as Robert Bjork and Nicholas Soderstrom (from the department of psychology at the University of California, Los Angeles) have demonstrated, you can see only performance, not learning.

If we measure performance, we can infer what might have been learned. But such inferences are highly problematic for two reasons. First, performance in the classroom is to a large extent dependent on the cues and stimuli provided by the teacher. Second, it is a very poor indicator of how well pupils might retain or transfer new concepts.

So where does this leave formative assessment? Let’s take a look at the “five key strategies” Wiliam presents as the bedrock of embedded formative assessment, as well as what changes might be needed.

Learning intentions and success criteria

First comes “clarifying, sharing and understanding learning intentions and success criteria”. The basic premise of letting kids know what they’re supposed to be learning is fine. But I do have a bone to pick with success criteria.

Wiliam recommends sharing mark schemes with pupils so they will know whether they have successfully achieved the learning intention. “Student-friendly” mark schemes are, he contends, “useful, as students are introduced to a discipline”. But many rubrics are inherently meaningless and rarely provide anything of value to students. This is especially pronounced in subjects such as English: what’s the difference between “confident” and “sophisticated” answers, really? The distinction seems to be arbitrary and exam boards are forced to provide exemplar work for teachers to understand the process by which they arrive at their decisions.

Instead of wasting students’ time with vague, largely unhelpful success criteria, why not spend time deconstructing exemplars and modelling the expert processes we would use to complete a task?

Eliciting evidence of learners’ achievement

This is all about how we use questions. Wiliam argues that only two valid reasons for questioning exist: “to cause thinking and to provide information for the teacher about what to do next”. I’m happy with the first but I’m not sold on the second.

In his groundbreaking work on classroom learning, the late New Zealand academic Graham Nuthall asserted that if we asked questions - no matter how well-designed they were - and used the responses to decide whether we should “move on”, we were in serious danger of overlooking the fact that “as learning occurs, so does forgetting”. According to Nuthall, “learning takes time and is not encapsulated in the visible here-and-now of classroom activities”.

If pupils don’t know the answers to our questions, we can remedy this. If misconceptions are revealed, we have an opportunity to explore them. But if they answer our questions correctly, it means very little. Just because they know it now, it doesn’t mean they’ll know it next lesson or in an exam. A correct answer is, perhaps, the least useful response we can hope for.

All the same, questions might be important for stimulating discussion. “Engaging in classroom discussion really does make you smarter,” says Wiliam. If this is right, then it’s certainly incumbent on us to ensure that all pupils participate in these discussions. But creating the conditions for effective classroom discussion has very little to do with finding out what pupils know and everything to do with provoking thought. Far from hoping for neat answers, perhaps the ideal end to a lesson is for pupils to be actively struggling with difficult concepts.

Nuthall’s research suggests that pupils are unlikely to transfer concepts from working to long-term memory until they have encountered them on at least three occasions. So who cares what they know at the end of a lesson? Let’s assume they are likely to forget it.

Feedback that moves learning forward

Who could argue against providing feedback that moves learning forward? Few would dispute that feedback is powerful but it can actually impair performance. The key to “moving learning forward” is for feedback to provoke thinking, Wiliam says, and that’s a lot harder to accomplish than we might imagine. Feedback that provokes thinking needs to offer hints and suggestions rather than complete solutions. It’s much better to ask “How could you.?” than to say “You should.”

Students who instruct one another

Wiliam argues that “cooperative learning” is “one of the greatest success stories of educational research”. He advocates a variety of techniques for encouraging cooperative learning which put the child at the centre of the classroom and move the teacher to the side. But there is always an opportunity cost: if we ask pupils to spend a lesson working things out in groups, that is a lesson that we cannot spend teaching them.

Cooperative learning certainly has a place but it should come only after careful explanation and modelling. Peer-assessment is often presented as a useful strategy for improving students’ attainment. But in light of Nuthall’s discovery that most of the feedback pupils get on their work is wrong, it is clear that this is extremely unreliable.

Students as owners of their own learning

It’s a truism that no one can teach you anything; they can only help you to learn. And yet we insist on holding teachers and schools to account but not students. It is no surprise that teachers often seem to work harder than the students they are supposed to be teaching.

Students can certainly develop sufficient insight into their own learning to improve it, but simply expecting them to assess their work using “student-friendly” mark schemes is unlikely to lead to success. Better to develop their error-checking skills: proofreading is very useful for making students engage metacognitively with their work, as well as ensuring that teachers target the feedback where students are making mistakes.

Many of the strategies discussed by Wiliam and enshrined in formative assessment have worth, and if considered carefully enough they will have a positive impact on pupils’ progress. I am a big fan of Wiliam’s and have been influenced enormously by his body of work. But I’m afraid I am forced to conclude that the “big idea” that you can assess learning and respond usefully is wrong.

David Didau is an education writer and trainer and can be found on Twitter @learningspy

What else?

This Teachers TV video reveals how to use pupil-to-pupil dialogue to boost learning.

These target sheets based on the traffic light system will help your students with self-assessment.

References

Wiliam, D (2011) Embedded Formative Assessment (Solution Tree).

Nuthall, G (2007) The Hidden Lives of Learners (NZCER).