Research throws doubt on English tests' reliability, reports Warwick Mansell
The pass mark for a crucial set of national curriculum tests, which apparently showed the Government transforming primary pupils' English skills, was set at least four marks too low, research for the test regulator has shown.
In findings which cast a shadow over the reliability of the testing regime, an independent study for the Qualifications and Curriculum Authority, which sets the tests, has revealed that pupils' achievements in English were overestimated.
Pass marks for the key stage 2 tests in 1999 and 2000 should have been set four or five marks higher than they actually were to maintain standards with previous years, the research says.
If they had been, the proportion of youngsters achieving the benchmark level 4 would have been cut by 5 to 10 percentage points.
And Labour would not have been able to claim such startling improvements in primary English at the last general election. Professor Peter Tymms, an expert on testing from Durham university, said other research evidence suggested that the English test results had overestimated improvements in schools.
He said: "This adds to growing evidence that suggests that standards over time are not reliably measured by the system."
The findings are contained in a pound;300,000 study commissioned by the QCA in 1999 which, as The TES revealed last week, has remained unpublished for nearly two years.
In 1999 and 2000 there were dramatic improvements in English tests for 11-year-olds. In 1998, 64 per cent of youngsters reached the expected level 4. In 1999, the figure rose to 71 per cent and in 2000, 75 per cent.
But the study by Alf Massey, of the University of Cambridge Local Examinations Syndicate, suggests those figures were too high.
Mr Massey led a team of researchers which asked more than 11,000 youngsters in Northern Ireland to take versions of key stage tests from 1996, 1999, 2000 and 2001. KS2 English covered the years 1996, 1999 and 2000 only.
The TES understands the research found that in English KS2, the later tests were easier than the 1996 version.
If the score needed for level 4 had been set four or five marks higher, results would have climbed from 63 per cent in 1997 to only 70 per cent, at best, in 2000.
The QCA cut the number of marks needed for pupils to reach level 4 from 51 out of 100 in 1998 to 48 in 1999. David Blunkett, the former education secretary, was forced to launch an inquiry after newspaper claims that ministers had lowered this pass mark.
The inquiry cleared the Government and Mr Massey's research, on which he would not comment, comes to the same conclusion.
However, it does suggest that flaws in the QCA's level-setting procedures, in particular the failure to take a long-term view of the effect of changes to the tests, allowed a downward drift in standards over a number of years.
Official maths test results improved from 62 per cent achieving level 4 in 1997 to 72 per cent in 2000, and the research is understood to find no fault with that.
A QCA spokesman said: "QCA has maintained the standard of the national curriculum tests over time. A child reaching level 4 in 1999 underwent as rigorous an assessment as in any previous or subsequent years."