What is so special about specialist schools? “The individual ethos and specialism of a school is vital ... to raise standards across the board,” according to school standards minister David Miliband. “The longer a school has been specialist, the better its performance,” insists Sir Cyril Taylor, chairman of the Specialist Schools Trust, who last week predicted that 93 per cent of secondaries would gain specialist status by 2006.

So how do specialist schools outclass their non-specialist neighbours? Is it the extra money? (A 1,500-pupil secondary that goes specialist will see its funding boosted by pound;888,000 during the first four years). Is it the target-setting culture? (Specialists must agree and meet targets with the trust). Is it the parent-attracting individual identity, or the curricular sparkle of specialism itself?

Some X factor makes specialist schools do better, according to data regularly produced by the trust and widely-quoted by ministers and in the media. But does it? What if, as academics and MPs across all political parties have suggested, factor X is a fiction?

Raw GCSE results from specialist schools are certainly higher on average than in non-specialist schools. But this could simply be due to differing intakes. Academic studies have reached conflicting conclusions in balancing non-specialist and specialist intake and performance.

While research for the trust suggests specialists add more value than non-specialists, the work of Sandie and Ian Schagen of the National Foundation for Educational Research suggests their performance is only slightly above the norm in technology and language colleges, and not at all in other specialists.

What no one has done is to look at whether any non-specialist schools are doing just as well as specialist schools which have the same kind of intakes. If in fact they are doing just as well, without the money, the targets, the curriculum or the name, then that suggests that there is no magic X factor, and that being specialist does not in itself improve the performance of a school at all.

The TES decided to put it to the test. We randomly selected 24 specialist schools, ranging over arts, media, sports and languages colleges. All had between one and eight years of specialist status. Eight were “high achievers”, with around 75 per cent of their 16-year-olds gaining five or more GCSEs at grades A*-C in 2002. Eight were “middle achievers”, and eight were “low achievers”, with 50 per cent and 25 per cent of students reaching that GCSE benchmark respectively.

To characterise their intake, with the help of research from the Liberal Democrat education team, we took the percentage of their pupils who qualified for free school meals (FSM), and we then tried to match their intake and their results with non-specialist schools in the same or similar geographical areas (see graph).

It was surprisingly easy. Of our 24 specialist schools, 23 could be matched.

All are doing no better than another school with the same intake, in the same or similar area, which does not have the benefits of specialist status.

For example, 49 per cent of pupils at Standish community high school, a languages college since 1996, gained at least five A*-Cs in 2002. (9.1 per cent of them were entitled to free schools meals in 2001.) Also in Wigan, non-specialist Up Holland high school got 49 per cent of pupils through the five A*-C boundary, with 14.8 per cent of them entitled to free school meals.

Our schools covered the South-east, South-west, East Anglia, London, the Midlands, Yorkshire, the North-east and the North-west. Only one, Ninestiles school, a technology college in Birmingham, could not be matched for intake and results with another non-specialist secondary school.

Of course, this does not necessarily mean that specialist schools are not good schools. Figures published by the trust this year show that some of them are very good indeed. However, they also show that some of them are doing rather badly.

The pupils in 199 specialist schools got significantly worse GCSE results than their key stage 2 test scores would predict, according to the trust.

(These were calculated by Professor David Jesson of York university using a value-added formula based on standard levels of improvement between performance at KS2 and GCSE.) When pupils at a school get exactly the GCSE results their KS2 scores would have predicted, that school has a value-added score of zero.

One hundred and ninety-nine schools is a lot - almost a third of the 656 pre-2002 specialists whose results Professor Jesson analysed. Factor X appears not to be working for these schools whose value-added scores range from minus three to minus twenty-three.

Several of The TES’s 24-school sample fall into this under-performing category. Intake high school in Leeds, for example, is an arts college which selects 10 per cent of its pupils. Just over a quarter of them (27.1 per cent) had free school meals in 2001 and 24 per cent gained at least five A*-Cs in 2002. At non-specialist West Leeds high school, 27.8 per cent of pupils have free school meals and 26 per cent gained five or more A*-Cs.

Intake’s value-added score, as calculated by the trust, is minus eight.

For our test, we deliberately chose schools at random. Had we picked the bottom 24 schools on the trust’s value-added table, masses of non-specialists would have outperformed them.

So where does this leave the repeated claims that specialist schools do better? On the individual level, they do not. As a group, according to the trust, in figures published most recently this April, on average, specialist schools add more value than non-specialist schools.

Specialist schools have an average value-added score of plus three, according to Professor Jesson’s calculations.

Critics have attacked these figures on several grounds. For example, the trust claims only to include “non-selective” specialist schools in its comparison. This is true, in the sense that the specialist grammar schools have been removed. However, the 40-plus specialist schools which select 10 per cent of their pupils have not been removed.

The extent and effect of selection is hard to pin down in specialist schools. Our unmatchable school, Ninestiles in Birmingham, selects 10 per cent of its pupils on technological aptitude, using literacy, numeracy and spatial reasoning tests. All applicants sit these tests. Ninestiles’ first criteria for admission is sibling connection. This means up to 20 per cent of pupils in any year may come from selected families, more if there are twins.

Overall, Professor Jesson says, the achievement of 11-year-olds on entry varies only slightly between non-specialist and specialist schools - the 1997 specialist schools’ intake had 25.9 KS2 points, compared with non-specialist school pupils’ 25.6 points. He argues that this strengthens the case that specialist schools achieve better results for similar students overall.

He also points to another performance indicator - Office for Standards in Education benchmark evaluations of school performance. These show specialist schools clustered much more heavily in the top AA* category - 36 per cent of specialist schools, compared with 23 per cent of non-specialists - while far fewer specialist schools appear in the lowest-performing Ofsted EE* category: 14 per cent of specialist schools, compared with 27 per cent of non-specialists.

“I’m not saying the EE* scores are great, but there are far fewer of them among specialist schools,” he says. “There has been a whole shift upwards of the system on those benchmarks. A lot more specialist schools are moving forward than other schools.”

Whatever the statistical arguments, there is no doubt that specialist schools feel successful. They attribute their success to the money: “We have a dance teacher and another music teacher which we’ve never had before”, said Lee Harris, head of Lostock (arts) college, Trafford. They attribute it to a sense of shared purpose: “You can say ‘That’s what happens in the PE faculty, there’s no reason it shouldn’t happen anywhere else’”, said Peter Blenkinsop, head of Whitefield (sports) school, Barnet.

And they attribute it to the motivated staff, pupils and community support that the name and status bring with them. “It means when I have a history teacher apply for a job, they say things like ‘Can I start a water polo club?’” said Bernard Clarke, head of King Alfred’s (sports) school, Wantage.

Without specialist status, all say, they would achieve less. And that raises a big question. Why do they need pound;400 million to be so successful, when some other schools with the same intakes achieve the same results without it?

“Educational outcomes and value added by specialist schools, 2002”, analysis by Professor David Jesson in association with Sir Cyril Taylor