“When you go down south, you realise just how different we are.”

That sentiment is common among Scottish teachers who venture into the English education system, but on this occasion the comparison was being drawn with somewhere less familiar - Wales.

School Leaders Scotland general secretary Ken Cunningham has made a number of visits to Wales on business, which left him “pretty shocked” by what he found.

Post-devolution - the Welsh Assembly was established in 1998, the same year as the Scottish Parliament - Wales saw Scotland, with its distinctive education system, as somewhere to emulate. But change was always more fragile in a country where educational approaches have historically been more tightly wedded to England’s.

From the abolition of secondary league tables and the scrapping of Sats tests to curricular revamping and the promise that free schools and academies would have no place in Wales, the country began divorcing its school system from that of England.

Then Wales got a huge shock from its poor performance in the 2009 Pisa international comparisons of 15-year-olds’ educational performance - and started falling back into old habits.

Crude league tables, observed Mr Cunningham, have been making a comeback and pitting schools against each other, regardless of social context.

The “language of threat” is increasingly common; schools are “compelled” to change. Mr Cunningham has a sense of inspectors leaving behind the idea of working in partnership with schools that had been fostered by a Scot, Bill Maxwell, who now heads up Education Scotland but was chief inspector of Estyn, the Welsh inspectorate, from 2007 to 2010.

Mr Cunningham has been struck, too, by the Welsh insistence that “qualifications are what matter”, at a time when Scotland is trying to show that qualifications can follow the lead of the curriculum, rather than the curriculum jumping to exams’ tune. Pisa-style testing has also become more prominent in schools in the run-up to the 2012 round of international results.

Important changes, meanwhile, are being made with negligible consultation; Mr Cunningham remarked on the centralising move to create four regional consortia to operate above the heads of the 22 local authorities.

Much of the angst within Welsh education has stemmed from a spending gap between it and the English education system, which has grown since devolution. In 1999-2000, spend per pupil was pound;58 higher in England than in Wales; by 2009-10 it was pound;604 higher.

It seems to matter little that many experts on such matters say it is more important how money is spent in schools, rather than how much they have; the perception is that England is striding ahead, and some believe that has had a damaging effect on the Welsh psyche.

“It’s the psychological situation of having your big brother living next door and trying to compete, which I think did Wales huge damage in the short term,” says David Reynolds, professor of educational effectiveness at the University of Southampton and senior policy adviser to the Welsh government.

“For much of the 2000s, Wales defined what it should do in terms of not doing what big brother does, and that was a huge mistake.”

At Christmas, headteacher David Swallow retired from Barry Comprehensive in the Vale of Glamorgan after almost 14 years at the helm.

Mr Swallow’s CV is impressive: he was awarded an OBE for services to education in 2003 and under his leadership the boys’ school, one of only two all-boys state schools in Wales, was named the most improved school for three successive years.

In 1999, only 24 per cent of pupils gained five GCSEs at grades A*-C; last year the figure was 70 per cent. Yet this remarkable success has been achieved despite low funding, even by Welsh standards. Vale of Glamorgan Council has the lowest budgeted expenditure per pupil in Wales at pound;4,901, some pound;1,538 less than the highest-spending, Ceredigion.

When Mr Swallow moved to Barry in 1998 from running a boys’ school in Manchester, he was surprised at the financial situation.

“Even in 1998, before devolution, England had so many more initiatives that enabled certain grants to get to schools,” he says. “The difference in funding between Wales and England was a shock then and it’s even more significant now. Funding on its own doesn’t necessarily make all the difference, and it’s not an excuse for poor results, but it is certainly part of the vehicle that will help you improve.”

Michael Davidson, the Scot who heads the OECD division responsible for Pisa, cautions against reading too much into spend per pupil, pointing out that the lauded Finnish education system is not a big spender in international terms.

“A certain level of funding is necessary for key resources needed for an effective school,” he says. “But throwing money at education is a policy that isn’t necessarily going to raise standards - it is much more about how the money is used.”

More pertinent, believes Mr Davidson, are perceptions in Welsh schools about their levels of educational resource. This is far lower than Scottish headteachers report, although still hovering around the OECD average.

He believes that may explain big differences between the two Celtic countries’ performance. Wales does considerably worse than Scotland across the board, at both the higher and lower ends of achievement: 23 per cent of pupils in Wales are below level 2, the second lowest on the Pisa scale, compared with 16 per cent in Scotland, and Wales has around half the rate of pupils at the second-highest grade, level 5, that Scotland does.

Although David Swallow is positive about many of the Welsh government’s curriculum developments, such as the skills-led Welsh Baccalaureate - a more advanced model than the newer Scottish qualification - he envies his English colleagues.

With a strong record in the performing arts, he could see Barry Comprehensive becoming a successful specialist school or academy, if Wales offered such incentives. Instead, the government has promised to ban their introduction altogether.

Some educationalists, including senior government figures, thought it a rash decision to scrap secondary school league tables in 2001, because effective comparisons between schools were no longer possible.

Last year, the move was called into question by a controversial report from the University of Bristol, which claimed it had had a “sizeable” impact on results, equal to a drop of almost two GCSE grades per pupil per year.

Between 2002 and 2008, the deeply unpopular Sats tests for 11- and 14- year-olds were phased out and replaced by teacher-led classroom assessments.

Again, that decision was called into question by a report last year, this time from Estyn, which found the teacher assessments of 11-year-olds that replaced Sats were often inaccurate and unreliable. Inspectors highlighted huge inconsistencies between individual primary schools and expressed concern over the lack of a national system to verify results.

But now, national testing of maths and reading and school ranking are both back on the cards, following the 2009 Pisa results. In their wake, education and skills minister Leighton Andrews announced a programme of radical reforms to improve standards, after Wales performed significantly worse than each of the other UK nations in all three Pisa measures (reading, maths and science) and worse than in the previous tests of 2006.

Andreas Schleicher, former head of the OECD’s indicators and analysis division, played down claims that abandoning Sats had an impact on Wales’ Pisa performance, but did question whether Wales went too far by ditching benchmarks that could help gauge pupils’ progress.

“The extent to which there are good practices and systems in place to take the temperature and monitor the quality of an education system seems to be less well developed in Wales than in Scotland,” says his successor, Michael Davidson.

Some 39 per cent of pupils in Wales are in schools where they have never had standardised tests of performance, compared with 16 per cent in Scotland. Transparency about standards need not equate to league tables, stresses Mr Davidson.

Neil Foden, headteacher of Ysgol Friars comprehensive in Bangor, says that, despite the gloom and doom, Wales has much to be proud of.

“Until relatively recently, Wales was better than England at listening to its teachers,” he says. “It was also less inclined to do some of the socially-divisive things that England has done, such as introducing academies and free schools.

“Scrapping league tables was good because they just encouraged schools to focus on improving their position in the tables, and not improving their offering to pupils.”

He also points to the success of the foundation phase policy for 3- to 7- year-olds. Inspired by Nordic models, the formal curriculum is set aside in the early years in favour of a play-led approach.

Teachers in Wales are enthusiastic about the greater freedoms the policy gives them. They see the English equivalent, the foundation stage, as too prescriptive and target-led.

The foundation phase in Wales is similar to Curriculum for Excellence in offering a broad, flexible framework to meet the needs of individual children and allow seamless progression through early education, says Children in Scotland’s chief executive, Bronwen Cohen.

The key difference - and failing - of both, in Dr Cohen’s eyes, is that they do not start from birth, unlike the Nordic countries.

Brian Lightman was head of St Cyres comprehensive in Penarth before becoming general secretary of the Association of School and College Leaders, so has experience of education in both England and Wales.

“In England in the past year we have had a massive change,” he says, “Every single education policy is being completely altered. It’s similar to the kind of change that Wales started in the early 2000s, but the difference is that the Welsh government was setting a vision. This (action in England) is stripping away all kinds of regulation and structures and leaving everything to individual schools to do.

“That has positives and negatives. At the moment, schools are in a complete state of flux and there’s a serious risk of fragmentation of the system. But more positively, it has strengthened school leaders’ attitudes to collaboration - more are working together and sharing services.”

Professor Reynolds says: “England has moved from prescription to more freedom, while Wales has gone the other way. You cannot have freedom if you don’t know how to exercise it. England has been resource- and capacity-building, but Wales did not do that; it gave teachers freedoms inappropriately. Autonomy works, but you can only do it when you give people the capacity to do it.”

Neither country has got it 100 per cent right, according to Dylan Wiliam, emeritus professor of educational assessment at the Institute of Education, University of London.

“I’m a firm believer that high-stakes accountability testing can improve pupil achievement, but it has to be done much smarter than it is in England,” he says.

“There’s probably not enough pressure on teachers and pupils to do well in Wales, and too much in England,” he says. “Wales in particular has suffered by taking the pressure off teachers and schools. On the other hand, whether you would want the kind of pressure you have in England is another question. Pupils in England aren’t happy.”

David Bright, headteacher of Llantarnam comprehensive in Cwmbran, took up his first headship in Wales in 1999 after several years of teaching in England, and shows a disdain for the influence of trade unions that would be unusual in Scotland.

“We were looking at what the school down the road was doing instead of the best-performing schools in the UK or Europe,” he says.

“Increasingly, we were running our schools for the benefit of the people who worked in them instead of the people who attended them. The unions had a lot more influence.”

This chimes with the views of the Westminster Tories on what schools would be like if the NUT were in charge.

Mr Bright concedes that union influence has had many positive benefits, with teachers feeling more respected and trusted to do their jobs. This came at a price, he says: the “dilution” of performance management and school inspection, both of which are now being strengthened again.

The “decade-long retreat” from high-stakes testing and accountability was coupled with a slew of well-meaning initiatives and a plethora of grants.

“There were too many initiatives and for years we struggled with priorities,” he says. “Nobody ever measured the outcomes of all these schemes or the impact of the grants, so we don’t know whether they even made any difference.”

It would be wrong to view Wales only as a cautionary tale, as there are also striking trends from which Scotland might learn.

The time Wales devotes to science, as established by Pisa, is remarkably high. Welsh pupils spend 279 minutes a week on science, compared with 240 in Scotland; the OECD average is 202 minutes. This appears to be a factor in Wales scoring higher for science (496) than for maths (472) and reading (476) in the Pisa results.

A perennial Scottish problem, the educational ball and chain of social deprivation, does not appear to affect Wales.

In Scotland’s comprehensive system there is relatively little difference between schools, observes Mr Davidson, “and yet pupils from poorer backgrounds don’t perform so well”. But the picture in Wales, which itself has a fairly homogeneous schools system, is markedly different.

“It’s not the case that in Wales your social background is such a predictor of your success as in Scotland - something else must be going on,” says Mr Davidson.

“It shows that in Wales you can succeed against different, difficult social backgrounds.”

ANYTHING YOU CAN DO: NORTHERN IRELAND’S SYSTEM

Northern Ireland, although it follows the English curriculum and qualifications system and still has grammar schools and 11-plus selection, no longer feels the need to look across the Irish Sea for inspiration.

Former secondary headteacher Professor Sir Robert Salisbury, who has been leading a literacy and numeracy review in Northern Ireland for the past four years, says: “Increasingly, there is an all-Ireland look at what works in education, because the Republic does very well in all sorts of areas.

“Educationalists here do look at what’s happening in Scotland, and they do look outwards to see what is happening in other countries, but they find it increasingly difficult to compare with what is happening in England.”

On paper, Northern Ireland’s GCSE results are usually higher than in England and Wales. Last year 74.8 per cent of entrants gained A*-C grades, compared with 69.8 per cent in England and 66.5 per cent in Wales.

But Professor Salisbury says that the figures merely support an “enduring myth”, as it is actually the country’s selective grammar school system that accounts for most of the top grades.

“The reality is that the top 5 per cent of pupils do well, but there is a very long tail of under-achievement, and at the bottom end, the results are poor,” he says.

“There’s almost been a sweeping under the carpet of poor achievement and accountability.”

LAGGING BEHIND THE NEIGHBOURS

FREE SCHOOL MEAL ENTITLEMENT, 2009*

Scottish secondary pupils - 12.3%

Welsh primary pupils - 17.1%

Welsh secondary pupils - 14.8%

English primary pupils - 16%

English secondary pupils - 13.4%

Scottish primary pupils - 16.7%

*2009 was the last year for comparable figures.

Original headline: Wales dared to be different, but is it now paying the price?