Teacher assessment can be used to produce objective and reliable results that can be used in a high-stakes context, a new study shows.

More than 1,600 teachers from 199 schools took part in the study, judging the writing portfolios of 8,512 Year 6 pupils.

No marking criterion was used at any stage. Instead, teachers compared two pieces of work each time, and decided which one was better. In that way, each school determined the internal order of its own pupils’ writing ability.

Primarily, teachers judged work of pupils from their own schools. But they also shared in the judgement of other schools’ work, in order to create a ranked order of all the pupils’ writing.

This resulted in a national scale of writing, with more than 8,500 pieces of work. Using statistical predictions, this work was then graded to the standards of “greater depth”, “expected standard” and “working towards”.

Improving

The study showed that teachers demonstrated a high degree of consistency in their marking: the reliability of the judgements was in excess of 0.84 out of 1.0.

But the study also highlighted the ways in which schools differ in their interpretation of key stage 2 writing standards.

The written report of the pilot study, which was conducted by No More Marking, an online engine to help teachers with comparative-judgement assessments, states: “Highly performing schools are relatively harsh in their judgement of poorly performing pupils within their schools.

“Poorly performing schools, however, are relatively conservative in awarding high achievers within their schools.”

The writing portfolio that was judged to be the best was the work of a pupil at Springwell Junior School, in the west London borough of Hounslow.

Dominic Hughes, Springwell deputy head, said: “The whole process has given us confidence that our writing is improving.”

Tim Oates, director of research for the Cambridge Assessment exam board, said: “There are logistical challenges for using comparative judgement for very large-scale assessment, but these are being tackled by the development of scanning technology and on-screen rating by assessors.

“It has not yet been used in large-scale high-stakes assessment, but the underlying theory is sound, and existing experience very positive. Trialing it for large-scale high-stakes assessment looks like a very sensible option.”

Want to keep up with the latest education news and opinion? Follow Tes on Twitter and like Tes on Facebook