One in eight would-be Sats moderators failed to prove they could accurately judge key stage 2 writing this year, Tes can reveal.

The figures come despite a major overhaul of the training process in a bid to resolve the “chaos” of inconsistent moderating.

Moderators check that teachers’ assessments of Year 6 pupils’ writing are consistent across the country.

They do this by visiting primary schools and checking that teachers’ assessments match a set of national criteria.

After a report from Ofqual found moderation in 2017 was “more inconsistent” than it could have been, The Standards and Testing Agency said it would improve the system in 2018, including by providing more “authentic” samples of pupils’ work in the standardisation test taken by moderators.

The STA also said it would support local authorities to share good practice and improve helpline and email response times.

The standardisation test involves looking at three collections of pupils’ work and assessing whether children are working toward the expected standard, working at the expected standard or working at greater depth.

Freedom of Information responses reveal the proportion of failed “pool” moderators - normally teachers working in the same local authority in which they are based - is stuck at 12 per cent: the same as last year.

While 2,288 pool moderators have been approved, there were 308 who failed the standardisation exercise.

However, the test has become tougher. This year, moderators needed to identify all three collections correctly either on the first or second attempt in order to moderate.

Last year, pool moderators who identified two out of three collections could moderate, after they had extra training.

For 2017, just 33 per cent of the pool moderators correctly identified all three collections, while 88 per cent did so this year.

James Bowen, director of NAHT Edge, said the figures highlighted that writing assessments were “full of problems”.

He said: “Moderation is better now than it was a couple of years ago, but clearly we need to think about whether this approach to moderation is producing reliable enough results that schools are held to account against.”

“The moderators themselves are teachers and we shouldn’t be too critical of them - this shows they are working in a very difficult system rather than anything about their own skills. It is an incredibly difficult job.”

The STA trains the “lead moderators” - a small number of people from each local authority - who then train the “pool moderators”.

The FOI shows that lead moderators have seen an improvement since last year, with 89.4 per cent identifying all three collections correctly on their first attempt.

A further 33 lead moderators identified all three collections correctly in a second exercise, bringing the total pass rate up to 98.6 per cent - an improvement on the 90 per cent pass rate for lead moderators last year.

Simon Kidwell, head of Hartford Manor primary and nursery in Cheshire, said: “It is still not an accurate way of judging schools and the key thing is this [the writing assessment results] is still linked to high-stakes accountability through floor standards, coasting standards and progress measures.”

A Department for Education spokesperson said: “In order to ensure that the teacher assessment system is valid and robust we have made improvements to the key stage 2 English writing moderation standardisation training this year, including holding a number of national key stage 2 local authority training events.

“We are already seeing improvements - the percentage of moderators who successfully completed key stage 2 standardisation has increased in 2018.”

This is an edited version of an article appearing in the 1 June edition of Tes. Subscribers can read the full article here. This week’s Tes magazine is available in all good newsagents. To download the digital edition, Android users can click here and iOS users can click here