My gran was a champion cake-maker. She put her success down to using eggs laid by happy hens. Her neighbour also won prizes but used a different recipe. While they disagreed on how to get the perfect Victoria sponge, they did agree on two basic points. First, if you want to make a good cake, you need a range of ingredients. Second, the way the ingredients are put together matters as much as the quality of the ingredients themselves.

Just as with my gran’s cakes, programmes matter in teaching reading but so does how you do them and how you mix them together. Clackmannanshire’s synthetic phonics programme was presented as a radical change in how reading is taught, and one that has produced stunning gains for children.

Jack McConnell, the First Minister, urged other authorities to adopt the programme and announced that Learning and Teaching Scotland is to appoint a development officer to promote synthetic phonics in schools.

Scotland is unused to this level of political and media debate about the teaching of reading. National policy promotes local decision-making and Scottish teachers tend to make highly pragmatic choices. Change has been evolutionary rather than revolutionary, driven by measured professional discussion rather than politicians and media headlines.

Let us be clear about what the children in Clackmannanshire were good at, and what this might mean. They were very good at decoding individual words - on average, three years ahead on standardised tests. The trouble is that sounding out individual words, no matter how long and complicated they are, is not a particularly useful life skill on its own. What employers, universities and colleges want is people who can read for meaning; people who are able to visualise as they read, infer meaning, recognise key ideas, new knowledge and contradictions, and think about how their reading applies to real life. Parents in particular know that comprehension is vital for reading to be useful in and out of school.

While the comprehension gains of the Clackmannanshire children were much smaller - three and a half months - they offer a more solid basis for claiming success and a much better yardstick for measuring future policy development. I understand why comprehension received no media attention or debate, competing against headlines screaming a three-year gain in the ability to sound out words. My fear is that the three-year gain in word recognition takes root in the collective “psyche” and teachers are bounced into making it their goal.

Other aspects of the Clackmannanshire results were disappointing and received no press attention. Despite comprehension and word reading gains, the pupils did not become more engaged in reading and, irritatingly, boys were less engaged than girls. Disappointing results need not imply that the research was unsuccessful; a careful analysis of why engagement didn’t rise, despite the other reading gains, would help take the national debate forward.

It is particularly important that we analyse the issues about reading engagement, because the amount children read is the biggest influence on attainment after social class. Avid readers develop a better vocabulary, a wider general knowledge and better verbal reasoning skills. This literally makes them smarter. They “punch above their weight” across the whole curriculum, outperforming peers with the same basic cognitive ability but who read less widely.

Let us also be clear about what schools and the authority actually did to get these results. The first message is that they did not just do phonics.

Nor did the authority just give the schools a programme and tell them to get on with it. My conversations with teachers in Clackmannanshire indicate that the children actually had quite a varied programme. Specific phonic instruction was accompanied by story-building activities, listening to stories, talking, reasoning, word and sound-play, comprehension, writing and reading, together and individually. Like gran’s prize-winning cake, there was a nice mix of ingredients. The experiment was part of the authority’s early intervention initiative and the pilot schools also introduced nursery nurses in primary 1, story bags, home-link teachers, homework clubs and nurture groups. The staff development programme for teachers was systematic, coherent and delivered with voice and verve.

Clackmannanshire’s project planning and management was super. Initially, the initiative focused on just a few schools and the new approaches to reading and writing were explained first to the headteachers and senior management teams. This ensured a high and informed commitment from these crucial managers. The authority’s development officer was highly knowledgeable about the teaching of reading. She worked closely with the university team, who had a clear view of what the project was about.

The staff development for teachers provided specific content knowledge about phonics and teaching literacy, as well as practical advice about using the resources, making learning purposeful, motivating children and the importance of noticing and building on success.

The timing was good. Staff development was provided at the point of need.

P1 teachers went first. Then P2 teachers were trained as the children moved into their class, and so on up the school. A rolling programme was provided to “catch” teachers transferred to a new stage halfway though the year. If no courses were running in the first four to six weeks of taking over a class, the new teacher was sent to observe how that stage was taught by experienced colleagues in other schools.

An independent, external evaluation of the whole Clackmannanshire initiative might have explained this wider context. It might have reminded everyone that the phonics part of the original experiment was conceived as a randomised controlled trail to compare different methods of teaching phonics. It wasn’t designed, as it is now being portrayed in the media, to show the effectiveness of phonics instruction over other methods of teaching.

Experienced teachers and education researchers know the folly of focusing on single programmes and hoping they will act as magical cure-alls.

Research on rolling out education reform indicates that the context of implementation has a crucial influence on the success or failure of any single programme of instruction. Success isn’t just down to the internal design features and content of the programme, but how it dovetails with other programmes and whether key staff “buy in” to its core ideas and methodologies. You need all the ingredients to work together.

Sue Ellis is a senior lecturer at Strathclyde University.