It is the time of year when the great semicolon hunt begins in Year 6 classes around the country. Each year, thousands of teachers are asked to present their pupils’ work to local authority moderators. The moderators then check whether these stories, non-chronological reports and pieces of persuasive writing support the teacher’s judgement that the child is indeed working at greater depth in writing - something that the correct use of a semicolon can help decide.

But who checks the moderators?

Following concern over the level of variation between authorities in 2016, the Standards and Testing Agency (STA) introduced a national test for moderators last year. The “standardisation” test involves looking at three collections of pupils’ work and then assessing whether children are working towards the expected standard, working at the expected standard or working at greater depth.

This year, almost one-in-eight “pool” moderators - who tend to be teachers moderating within their own local authority - failed that test, according to freedom of information responses obtained by Tes. This is the same proportion as in 2017, despite a raft of changes being introduced to improve consistency.

The STA’s plans to improve moderation in 2018 included the provision of more “authentic” samples of children’s work in the standardisation test, support for local authorities to share good practice, and faster helpline and email response times.

So why haven’t we seen a reduction in the failure rate? James Bowen, director of middle leaders’ union NAHT Edge, is reluctant to blame the moderators. He says: “The moderators themselves are teachers and we should not be too critical of them. This shows they are working in a very difficult system, rather than [saying] anything about their own skills. It is an incredibly difficult job.”

Perhaps the problems lie with the assessments rather than the skills of moderators. This is Bowen’s view. “The way we are assessing writing at the moment is full of problems,” he says. “We have to start questioning whether, given the amount of work and workload involved in all this moderation, it is the right system to be using.”

Flaws in the system

There has been some recognition from officials that the moderation system is imperfect. Since the new Sats were introduced in 2016, unions have been concerned that some local authorities interpret the national framework - which sets out how teachers should assess pupils’ work - more strictly than others. In response, the government and Ofsted have agreed that the writing data would not be used alone to prompt intervention in a school.

But the key stage 2 writing assessments involve a lot of time and effort from schools, as well as from the STA.

The STA trains the “lead moderators” - a small number of people from each local authority - who then train the pool moderators.

After the training, both lead and pool moderators take the standardisation test to check their judgements match those of the STA. Local authorities sent 359 lead moderators to do the national training run by the STA in November and December, according to an FoI response from the STA.

The response also reveals that 321, or 89 per cent, of lead moderators categorised all three collections of work correctly on the first attempt. A further 33 lead moderators identified all three collections correctly in a second exercise, bringing the total pass rate up to 98.6 per cent - an improvement on the 90 per cent pass rate last year.

The same improvements to the pass rate have not been seen among pool moderators. Training of pool moderators took place at the beginning of the year. The Department for Education has confirmed that 2,288 have since been approved, while 308 have failed the standardisation exercise - a failure rate of 12 per cent, which is the same as last year.

However, the test has changed slightly in order to make it tougher. This year, moderators needed to identify all three collections correctly, either on the first or second attempt, in order to moderate. Last year, pool moderators who identified two out of three collections could moderate after they had extra training. While just a third of pool moderators correctly identified all three collections last year, 88 per cent did so this year.

The statistics show how difficult it is to achieve a consensus on the quality of pupils’ work. And heads point out that despite this difficulty, writing judgements still form part of a high-stakes system. “The moderators are better because they are doing it for the second or third year, but there is still a gap between moderated and unmoderated schools,” says Simon Kidwell, head of Hartford Manor primary and nursery in Cheshire.

“It is still not an accurate way of judging schools and the key thing is this [the writing assessment results] is still linked to high-stakes accountability through floor standards, coasting standards and progress measures.”

A DfE spokesperson says: “In order to ensure that the teacher-assessment system is valid and robust, we have made improvements to the KS2 English writing moderation standardisation training this year, including holding a number of national training events. We are already seeing improvements - the percentage of [all] moderators who successfully completed KS2 standardisation has increased in 2018.”