It is supposed to be the overarching policy for improving education in Wales, with schools, local authorities and the Assembly government working together for the common good of pupils.

But three years after it was launched, and in the term in which it was due to be rolled out, there is significant doubt over the future of the school effectiveness framework (SEF).

TES Cymru has spoken to dozens of leading academics, civil servants, politicians and union leaders who share similar concerns - namely, that SEF has lost its sense of direction and that its aims are no longer clear.

Heads are unsure how SEF will apply to their schools and classroom teachers are unclear about their roles in the framework, despite the government’s best efforts.

The recent emphasis on developing professional learning communities (PLCs), designed to promote individual and collective staff development, has only added to the confusion. Some educationalists have branded them a “gimmick and a distraction” from the original purpose of SEF, which was to share good practice.

Education minister Leighton Andrews hoped to inject some clarity by adding three new priorities to SEF - improving literacy and numeracy levels, and reducing the impact of poverty on attainment.

But a group of influential academics revealed their worries to TES Cymru.

“We have real concerns about where this is going,” they said in a joint statement. “We don’t have a feel for any sense of direction, and it’s not clear if anybody really knows who is actually leading and driving this. We have been on it for three years; we have got to implement something.”

The Assembly government has long been accused of using SEF as a “sticking plaster” for all that is wrong in Welsh education.

At their annual conference in April, members of teaching union NASUWT Cymru expressed concerns that the framework could be used as a cover to disguise “woeful under-funding” of education and teacher job cuts.

In May, Michael Fullan, a leading authority on school improvement and an adviser to the Canadian government, told TES Cymru that SEF could fail to work properly because of unrealistic expectations of what it can achieve.

Teaching unions have complained that there is confusion among their members, some of who claim the importance of the SEF has not been adequately communicated.

Tory shadow education minister Paul Davies told TES Cymru that he was concerned at the lack of understanding about SEF.

“I get a mixed reaction from teachers, but there are certainly concerns,” he said. “It’s important the government gets its act together and ensures that the information is available and it is clear to everyone.”

But Gareth Jones, secretary of heads’ union ASCL Cymru, urged patience and said it would take time for SEF to reach all levels of education.

He said the minister “deserved praise” for introducing the new priorities. “We don’t want nationally driven programmes for numeracy and literacy like in England,” he said. “The minister has put SEF on the right path by saying these are the priorities; now how do you want to tackle them on a local basis? That’s a far better approach.”

Mr Andrews claims there has been a “great buy-in” to SEF and its new dimension gives the policy a more precise focus.

“I think there’s clear understanding,” he said. “What SEF is about is creating high-performing schools based on strong leadership, developed through professional learning communities.”

Whether the critics are justified remains to be seen. But the fact they exist at all at this late stage, and from so many different corners of the education sector, will ring alarm bells in Cardiff Bay.

By dismissing the concerns as “complete and utter tosh” and accusing those making them of “whingeing”, the education minister risks causing further anger.

One academic said: “This abrasive, aggressive language is unfortunate and will only alienate people he needs to be keeping on side during this crucial time.”

SCHOOL IMPROVEMENT: TEACHING IN THE SPOTLIGHT

SEF was created in response to Wales’ poor performance in the 2006 Pisa international student assessment, in which it performed worse than the other UK nations.

It is the Assembly government’s key reform policy, aiming to increase pupil achievement and wellbeing by improving the quality of teaching and leadership in schools and local authorities.

It is underpinned by five core themes - equality, bilingualism, support, systems thinking, and a high-performance culture.

Based on a partnership between the Assembly government, local authorities and schools, SEF aims to reduce variations in attainment between, and even within, schools.

The government has agreed a national model for school improvement with all 22 local authorities, and delivered leadership training to them. It has also provided training on professional learning communities to 850 schools and 12 local authorities.

Training for a further 650 schools and five authorities is currently taking place, and remaining schools and local authorities will receive training by March 2011.

COMMENTARY: APPLY OUR BRAINS TO `WHAT WORKS’

David Reynolds

We are now skilling our teachers as many others did in the 1990s and 2000s - as England did with its somewhat different numeracy and literacy strategies.

Doing things after everyone else is not recommended, but there are advantages. The field that provided the intellectual foundations for the English strategies - school effectiveness and school improvement - was in its infancy a decade ago.

Teacher effectiveness did not exist in UK research and was among the “known unknowns”. And as for cognitive neuroscience, we didn’t know we didn’t know.

So far, SEF has concentrated on systemic and leadership change to deliver improvement: this is useful. Setting up professional learning communities to promote sharing is useful, too: teachers need teachers. Work has been done to develop core data sets, stronger self-evaluation and to align SEF with the new inspection framework.

However, processes must have content to fill them. Without evidence about what “works”, teachers may reinvent the wheel or may collaborate to go in the wrong directions, which wastes precious time. Without content, SEF can resemble one cynic’s definition of California - “there is no `there’ there”!

So SEF needs to ensure that teachers are effective and have the best tools and support to improve practice and outcomes.

What kind of content could be brought in?

- The new science of the brain - how it can be developed, fed, rested and trained. l The new procedures involved in making each school’s best teaching practice its standard practice, or identifying `within-school variation’.

- The evidence on high-reliability schools that has come from our world- class Neath Port Talbot.

- The new evidence on what schools in socially disadvantaged areas need to do.

This material can be brought to our teachers through inexpensive electronic means. Critics may say it is “prescription”; it is actually the opposite. It is giving our teachers “what works” to give them the opportunity to become even better. It is respecting our teachers’ rights to have the very best knowledge that the world can offer.

David Reynolds is professor of education at Plymouth University and lives in South Wales.

• Original headline: Just how effective is the school effectiveness framework?