Critics say the Qualifications and Curriculum Authority is displaying secretiveness by failing to release full findings. Warwick Mansell reports
In January last year, the results of the biggest-ever experimental study of pupils' results in national curriculum tests were sent to England's exams regulator. Nearly two years on, these findings, which are likely to prove embarrassing to the Government and could undermine one of Labour's biggest success stories in education, have still not been released.
Yet this is just one of a number of instances in the past year in which the Qualifications and Curriculum Authority appears to have been reluctant to publish information which ministers would rather not hear.
To critics, they show how the regulator remains a secretive, politicised organisation, unable to act independently of the Government despite last year's A-level regrading outcry which focused attention on its relationship with ministers as never before.
In 1999, the QCA commissioned Alf Massey, of the University of Cambridge Local Examinations Syndicate, to lead a team of researchers investigating the achievements of seven, 11 and 14- year-olds in national tests.
In a three-year study which cost pound;300,000, the team aimed to compare the difficulty levels of the tests of 1996 with those of 1999, 2000 and 2001. The improvements in those years had been startling. In 1996, 58 per cent of 11-year-olds reached the expected levels in English, and 54 per cent in maths. By 2001, the figures were 75 and 71 per cent respectively.
The increases were seen as proof of the success of the Government's literacy and numeracy strategies for primary schools.
But questions remained. Were teachers simply getting better at teaching to the test? And how could we be sure that the tests were not getting easier?
It was in attempting to answer this latter question that the QCA commissioned the UCLES research. One group of pupils would sit a test from 1996. Another group, assessed as being of the same ability level, would take a test from another year, and the results would be compared.
More than 11,000 pupils took the tests which were set in 1996, and then those which were set in 1999, 2000 and 2001.
The study covered tests taken at all three key stages, in English, maths and science. But because of the widespread use of past papers in English schools, pupils from Northern Ireland, who study a similar curriculum but take different national tests, were chosen for the experiment.
Separately, the study considered the results of independent tests taken by pupils in six English local authorities, and interviews with pupils, asking whether they thought the tests have changed.
So what were the results? We still do not know. To the frustration of some within the research community, the findings have not been released, even to those closely involved in the study's compilation. Mr Massey would not share them with The TES.
However, it seems likely that they will not be unalloyed good news. He did say: "The research does demonstrate that there were genuine improvements in performance in schools across this period.
"The big question that I am not answering for you at the moment, is whether or not the improvements that we were able to detect were as big as, or smaller than, the changes in results in the national test levels.
"The research most certainly answers that question, and the media will find it interesting."
This is not an isolated case. This spring, the QCA commissioned a survey by the opinion pollsters MORI into the public's opinion of A-levels in the wake of the regrading controversy. Parents, teachers and students were asked whether they thought the standing of the qualification had been damaged. Again, the full results have not been released. A public relations firm acting for the regulator did release some selected, positive, highlights in May, but The TES has been unable to see the full findings.
In fact, much research that the QCA commissions is published in summary form only. And the lack of transparency extends further: for example, no public minutes are published of the QCA's board meetings though the QCA is reconsidering this. By comparison, those for the Teacher Training Agency are.
The QCA argues that research that is not published is for "internal purposes only". It says that some of the work which it publishes is potentially embarrassing to the Government, and points out that its chief executive, Ken Boston, strikes no-one as someone afraid to speak his mind.
Mr Massey said the regulator was "brave" to commission his research.
The QCA's defenders emphasise the atmosphere in which it works: any report which questions the quality of a qualification or whether standards have been maintained will be seized upon by a largely hostile media.
Some argue that it is not always in the public interest for information to be released. Does the public benefit if every year there is a questioning of the validity of standards and the examining process itself?
But is this really a good enough defence for a regulator? The contrast with the Office for Standards in Education, the schools inspectorate which makes great play of its ability to speak out independently and has greater institutional freedom from the DfES, is stark.
Ofsted, unlike the QCA, is classed as a non-ministerial government department, giving it greater freedom than the QCA, a quango or "non-departmental public body". Claims that the QCA's lack of transparency is the result of ministers' control of the regulator's output are hardly new. Nick Tate, the quango's first chief executive, admitted that, when he challenged ministers' primary reforms, he was told that he had "blotted his copybook".
The fall-out from last year's A-level furore is proof that the arguments are still relevant: the relationship between the QCA and ministers was at the heart of that controversy.
Sir William Stubbs, the former QCA chairman, accused Estelle Morris of interfering in the official inquiry into the last-minute grade changes. He was sacked.
Dr Boston later said he had no evidence of ministerial interference in the QCA. But last October, he told the Commons education select committee that he favoured "greater distance" for the QCA from ministers, and said the Ofsted model was a "very good one".
He added: "It is clear that the independence of the organisation is not transparently there, it is not unambiguously accepted, and it needs to be, in a far stronger and clearer way."
So what has changed since then? In his official inquiry into the A-level debacle, former chief inspector Mike Tomlinson rejected big changes in the QCA's status, arguing in part that this risked destabilising the exams system.
But, after highlighting concerns about the independence of the QCA's advice to ministers, he proposed some reforms.
Traditionally, QCA advice has been published only when the Government has decided how to act on the issue. Mr Tomlinson said this meant "that QCA's advice may be informally constrained in order to produce an outcome that is acceptable to ministers".
He recommended that all government remits to the QCA for advice, and the subsequent advice that the QCA sent back, should be published immediately.
Back in March, ministers and the QCA appeared ready to accept this recommendation. The QCA press office told The TES that the regulator was about to publish a "schedule of advice", setting out what would be recommended and when.
Six months on, the schedule has not appeared. The Government rejected the recommendation, saying that the QCA could publish advice to ministers when it made it, but only if it chose to.
Otherwise, the arrangement would remain as before: that advice to ministers would be published "once ministers have considered their response".
John Dunford, general secretary of the Secondary Heads Association, criticised the move. He said: "Ministers would benefit from a wider public debate before they take a decision on an issue."
Dr Boston has clearly not received the "greater distance" from ministers he asked for. But these issues are unlikely to go away with ministers apparently reluctant to let the QCA speak out frequently.
A QCA spokesman said: "Ken Boston is committed to making the organisation a more open one. It's very much the intention to move in that direction.
There is more that can be done, but the commitment to openness is there."