Suddenly, we are all talking about teacher judgements again. Prompted by the Covid-19 pandemic and the fallout of the GCSE and A-level exams “solution”, we are rehashing the debate about whether the “human condition” means we are full of biases or whether a contextual judgement by a teacher is more valid and reliable than other measures.
Is there any evidence out there about which argument may have more ballast to it than the other? A recent review of studies looking at the accuracy of teacher judgements (Urhahne and Wijnia, 2020) attempts to give an overview of this complicated topic.
The authors distinguish three kinds of teacher judgement accuracy: task-related, person-related and person-specific.
- Task-related judgement is about the ability to assess the difficulty of tasks based on the class’s performance level.
- Person-related judgement accuracy is about the teacher’s knowledge of different student characteristics, such as academic achievement, academic self-concept, work and social behaviours, and motivation.
- Person-specific judgement accuracy refers to the ability to identify and classify personal characteristics or behaviours of individual students.
The authors reviewed 385 articles, books and book chapters, and decided a distinction had to be made between relative and absolute accuracy. Relative accuracy refers to the correlation between teacher judgements of student achievement and actual achievement. Here, teachers tend to get the level a student is working roughly correct.
Absolute accuracy is the difference between the predicted and the actual level of student achievement. Here, things don’t go so
well. Teachers tend to overestimate student achievement on a standardised test.
So teacher judgement leads to grade inflation? Not necessarily: the authors call this discrepancy a “mild overestimation” and stress that it is not uniformly negative. Instead, the teacher “belief” serves as a kind of encouragement, eventually leading to higher standards of performance.
But there are complications. For example, teachers show only moderate accuracy in ranking the relative difficulties of tasks. And for less achievement-oriented outcomes, teacher judgement showed little agreement with students’ cognitive functioning, emotions and self-assessed study and social skills. Both could have a negative impact, particularly for lower-attaining students.
We also discover some clues about what influences teacher accuracy. Experience, for example, is not, or is only weakly, associated with judgement accuracy - familiarity with the assessment task is more important.
Meanwhile, the included studies suggest that student gender is not, or is only weakly, associated with judgement accuracy, but student gender indirectly affects teacher judgements of students’ academic skill through their perceptions of students’ behaviour - for example, for boys in their tidiness or their actual behaviour.
So, what are we to make of all this? The authors conclude that teachers should become more aware of stereotypes and try to rely on valid indicators to gauge student characteristics. They also add that teachers need to find a balance between “correct” and “fair” judgements.
Accurate teacher judgement is difficult, then. One cannot help but think, though, that it would be a lot easier if the judgements teachers were asked to make most often were not so high-stakes.
Christian Bokhove is associate professor in maths education at the University of Southampton
This article originally appeared in the 8 January 2021 issue under the headline “Teachers’ judgement under the microscope”
