Last week’s claims by a right-of centre think-tank that the average 11-year-old is nearly two years behind in mathematics have come under fire from academics and local education authorities.

They say the Social Market Foundation’s high-profile analysis of the 1995 national curriuculum test scores used flawed data and statistically incoherent methods.

Even Government officials seem to believe that last year’s junior school results are unreliable. LEA leaders were denied access to the national figures when they attempted their own analysis earlier this year. “We asked for these data in April and were told that the aggregation to local authority level was not reliable,” said John Fowler, of the Association of Metropolitan Authorities. “Therefore the department would not release the data.”

Dr John Marks, a right-wing commentator and author of the pamphlet, was allowed to see the figures, however. He says he was not warned about their reliability.

Accepting the Government view that the average year 6 pupil should reach level 4, he concluded that British 11-year-olds are 18 months behind in English and nearly two years behind in maths. He also found wide disparities within LEAs. He said the best schools were on average four years ahead of the worst in English, and five-and-a-half years ahead in maths.

But academics have cast doubt on his methods as well as on the unreliability of the data - even though they share his concern about underperforming schools.

Professor Ted Wragg from Exeter University’s School of Education said: “You need a more fine-grained analysis to reach his conclusions. If a student brought that analysis to me, I’d say: ‘Go away and read a book on statistics. ’” Professor David Reynolds from Newcastle University also expressed concern: “John Marks’ conclusions can’t be obtained from the data. The data are flawed because they don’t take into account social background.”

The data appear to be weak in a number of ways. The School Curriculum and Assessment Authority has accepted that last year’s test results depressed the results in English and maths. The collection method also involved the double counting of some low achievers. SCAA says that this year’s figures will provide a better basis for national comparisons.

Another factor depressing the results is the “categoric” nature of the national curriculum scoring system. “Pupils cannot achieve part scores, ” said Professor Wragg. “You cannot get level 3.5. If every pupil in a class of 11-year-olds completed all the criteria for level 3, and half the criteria for level 4 the average score would be 3. John Marks would conclude that they were two years behind the expected level, but they wouldn’t be.”

In technical terms, says Professor Wragg, the report treats national curriculum levels as a “ratio scale”: each point is equidistant from those on either side. But there is no basis for assuming that each national level accounts for exactly two years of development.

There are also complaints about the basic accuracy of the DFEE’s information. Somerset was astonished to see one of its schools scoring an average of zero (in fact, the results were lost). LEAs have been angered by the SMF’s “league table” of authorities, which ranks them on the basis of very small differences in average performance.

Dr Marks accepted that Government figures are not sophisticated. “I argued very strongly that we should have a more discriminating measurement system. Two years is too big a gap between each level. But I lost that argument. I am very much in favour of properly age-standardised tests.”