Confusion over how teachers should assess pupils’ writing as part of the key stage 2 Sats brought chaos to primary schools last summer.

There was so much variation between standards across the country that both Ofsted and the Department for Education eventually pledged not to use the results to judge schools.

Faced with the threat of a Sats boycott, the government also vowed to avoid a repeat this year by ensuring “a more consistent, reliable approach” to moderating teacher assessments of writing.

But new data uncovered by a Tes investigation suggests the government may have failed. The data reveals that two-thirds of the moderators trained for this summer incorrectly assessed pupils’ work when tested earlier this year. They were required to judge it against three possible government standards.

Moderators have described the system as “crude”, “ridiculous” and a “farce”.

Teachers fear the problems will mean more chaos. They point out that if moderators who have been given extra training and guidance are unsure where standards are, then the teachers who will be the final assessors of most pupils’ work do not stand a chance.

And heads are concerned that teacher assessments of writing will still be published and used to judge their schools, despite continuing confusion over where standards should lie.

“Schools have every right to be worried,” says Mary Bousted, general secretary of the ATL teaching union.

“We know moderation of writing was a complete nightmare last year. The moderation arrangements were chaotic. It’s still not sorted. If [the Standards and Testing Agency] can’t guarantee consistency, why are we spending these millions of pounds and having all this stress, for writing scores that in all probability won’t be accurate?”

Tes used freedom of information (FOI) requests to obtain figures showing how the moderators - trained to check teacher assessments of pupils’ writing for local authorities - performed when they assessed three specimen portfolios of pupils’ work earlier this year.

‘A new problem’

The STA revealed that only a third of 2,547 team moderators correctly assessed all three portfolios during its mandatory tests. Fifty-five per cent correctly identified two out of three portfolios and were judged to need further training, and 12 per cent were officially judged to have failed, having got just one correct or none at all.

Michael Tidd, deputy head of Edgewood Primary in Hucknall, Nottinghamshire, says: “Most of these experienced moderators should be able to get this and if they can’t do it, what hope is there for the schools who have not done the training?

“It was chaos last year and the theory was that this training would solve that problem - it’s just created different chaos. It’s a new problem.

“I think it is absolutely right that the tests should make sure there is a minimum standard and some people will fail to meet that. But for substantial numbers to fail is concerning.

“My real concern is that they are going to moderate only a quarter of schools. So the other 75 per cent of us are still guessing and if trained moderators are still finding it really challenging, what does it mean for the rest of school population?

“How can you use these high-stakes results for high-stakes decisions when it’s such a difficult thing to do?”

FOI responses from 101 local authorities also revealed large variations in the proportions of moderators managing to correctly assess all three portfolios of pupils’ work - ranging from 6 per cent in Sheffield to 100 per cent in 13 other authorities.

There were a total of 21 authorities where fewer than a fifth of moderators got the three assessments right, including five authorities where the proportion was less than 10 per cent.

The DfE has tried to reassure schools by stating in its current consultation on primary assessment that “no single piece of data will determine any decision on intervention”.

‘Huge pressure’

Meanwhile, Ofsted has said that it will treat data from this year’s writing assessments with caution (see bit.ly/OfstedCaution).

But schools are concerned that results from the writing assessments will be recorded as public records of their performance, despite the ongoing confusion over where standards should be set.

“Writing data is still published,” says Brian Walton, headteacher of Brookside Academy in Street, Somerset.

“The pressure is huge. [The scores] are on Raise online, on my website, everywhere. Anybody who comes in, this is the first thing they see. I have to explain the situation and look like I’m making excuses. I’m not, that is the reality.

“It doesn’t matter whether they say they’re using it. If a figure is published in any way, it matters hugely.”

Tidd has similar fears and points out that teacher assessments of writing will be used to calculate progress measures which regional schools commissioners will use to judge whether schools are coasting.

“The published data, it’s all very well saying the DfE won’t use it, but parents will look at it,” he adds.

“If the DfE is saying they know there are problems, why are we publishing it at all?

“I have a lot of time for moderators in local authorities. I feel for them because they have been put in a terrible situation, and they are the ones who are going to have to face schools and try to get this right.”

The moderation system is designed to ensure that the standards used in teacher assessments are the same in schools throughout England.

The local authority moderators go into schools and ask to look at work from a sample of children, to check that teachers’ assessments of writing are correct.

Following last summer’s problems, mandatory national training and testing for moderators were introduced for the first time this year.

But Russell Hobby, general secretary of the NAHT heads’ union, says: “Unfortunately, at the end of the day, no amount of training can wholly correct the fundamental flaws in the current system of assessment.”

The previously unpublished moderator test results indicate extensive disagreements about how to interpret the assessment criteria.

In Sheffield, only two of the 35 moderators who were trained got all three portfolios in the test correct, according to STA standards, and in Hampshire the figure was seven out of 42.

The DfE says on moderation: “Local authorities have given written assurance to the STA that they have provided additional training to individual moderators who did not correctly identify one of the KS2 standardisation collections [portfolios].

“LAs have also been required to implement quality assurance procedures to ensure that all moderators are able to make consistently accurate judgements.”

Simon Kidwell, head of Hartford Manor Primary in Northwich, Cheshire, sympathises with moderators who failed the test.

“They are not poor moderators,” he says. “I have a lot of respect for them but I do know how difficult they find it.

“How experienced are the people at the STA who are making these judgements, compared with the moderators in local authorities?

“How can they say their judgement is better than people who are doing it for schools as teachers and have been actually moderating for many years? You can’t. It is a human judgement. The STA itself is under huge work pressures.

“I feel sorry for them as well, because they are trying to interpret an almost unworkable framework. I feel for all of the moderators and the STA, but the impact falls on schools.”

@teshelen