An expensive drill-and-skill program or a tool that can significantly boost the way children learn? Integrated learning systems (ILS) have attracted a fair degree of controversy in the educational world, and the signs are that the arguments will run and run.

An ILS is a computer-based learning system designed to help children develop specific skills, often in maths and English. Computer-based systems are not new, but what makes an ILS different is the way it works. An ILS presents users with a question, records the response, provides feedback and automatically selects the next task based on the user’s answer. It is the automatic nature of ILS that differentiates them from other computer-based learning systems.

That said, the term “integrated” can be misleading, as some schools use ILS in isolation. What is more, the term “ILS” covers a broad range of systems, in the same way that the term “car” describes many different types of vehicles.

An ILS may be a closed system (teachers having little or no input into the content or presentation of the materials). Or it can be open, providing some degree of teacher input, and allowing customisation for individual users. There are various ways or models of using an integrated learning system, such as a 5050 model, with half the class on the ILS and the rest working away from the computer.

The number of schools using ILS products today probably runs into several thousand, although few manufacturers offer firm numbers. (RM says 1,450 schools are using its ILS products, with primaries accounting for 300-400 users.) Although the price of implementing ILS varies greatly from school to school, a basic investment of around pound;20,000 is not uncommon. This means that for some secondary schools, investing in ILS can mean using the entire ICT budget, and for primary schools their entire curriculum budget too.

There are other costs to consider, such as training, managing and maintenance, upgrades and the disruption caused by changes to timetabling or the block-booking of a computer room. But if an ILS delivers what it promises, this investment may be a price worth paying.

The problem for schools is that the jury is still out over the effectiveness of ILS. The choice is made more difficult by the growing number of educational vendors offering ILS products including RM, Longman, Tag developments and SIR. The new generation of ILS products arrived in the UK in the early Nineties and such was the impact of their arrival that the DFEE commissioned a series of research projects, managed by the British Educational Communications and Technology Agency (Becta - formerly NCET). Between 1993 and 1997, Becta managed three major projects, and a final evaluation report was published last year 1998.

But anyone hoping to find the definitive judgement on integrated learning systems in this report would have been disappointed. The report started off in an upbeat fashion: “There is considerable evidence that pupils do learn from ILS. The main issue is not if pupils learn, but what and how they learn.” But the report added that integrated learning systems “have a long way to go before they can receive unequivocal endorsementI The research does not allow clear-cut advice for schools.” Becta says those schools considering purchasing an ILS should look at its value for money compared with other ICT and non-ICT alternatives.

Gerry Daish, sales director of Longman Logotron which markets the Tomorrow’s Promise ILS - a Britishversion of the product sold by Jostens in the US - is refreshingly candid about the state of ILS research as he sees it: “There are no reliable, independently-produced studies to show that ILS gives X amount of gain. The evidence is mainly anecdotal.”

Ruth Lawrence, RM’s senior marketing executive for learning systems, disagrees. “There are many reports and studies on ILS from Becta, LEAs and schools.” RMis so confident about SuccessMaker that it offers to refund money to schools not satisfied that their students have made learning gains in literacy and numeracy.

A number of companies have sponsored research into ILS. Longman funded a project run in Hartlepool earlier this year and RM sponsored a two-year evaluation, run by Lancaster University, involving Year 1 and Year 2 pupils. RM found that in the first 12 months, the Year 1 trial pupils gained 40 per cent more on average in number skills and abilities than the control group, and the figure for Year 2 was 34 per cent more in number concepts. The results for the latter group also appeared to show many gains. Three schools using RM Maths were compared with two control schools, with the study focusing on national test results. At the end of Year 2, 92 per cent of the trial group achieved Level 2 and above, compared with 81 per cent of the control group, and more trial group students (69 per cent) achieved Level 2B or better than the control group (63 per cent).

The report concluded that RM Maths Learning System had a significant and positive impact on their national test results. But when it came to assessing the impact of ILS on key stage 3 tests and GCSE , Becta’s final evaluation found that ILS could have a negative impact!

Professor David Wood, of the ESRC Centre for Research and Development in Training - or Credit - at Nottingham University, was the author of Becta’s final ILS evaluation. Asked what impact integrated learning systems can make on learning, he replies: “If you look at motivation for learning, all the signs are positive and that they are long-lasting. The impact on schools is more subtle. If I was in a school and setting up ICT with teachers who do not know much about it, ILS could be attractive because they can be a good way of bringing ICT experience into a school. But if my teachers were skilled in ICT, I would be very doubtful about using ILS.”

When it comes to ILS and learning outcomes, David Wood adds: “If I wanted to put on a positive gloss, (these systems) are very good for basic numeracy skills, but I would be cautious about using them in English. There’s no evidence that they make a difference here.”

Another issue is whether children can transfer or develop the skills they acquire by using an ILS, says David Wood. “We would expect children to build on their skills in key stage 3, when more open-ended problem-solving is used. But that does not happen.” He speculates that this might be because teachers have not yet learned how to integrate skills acquired via ILS with other problem-solving skills. However, another reason could be that today’s ILS products are not well designed to help teachers, because they don’t give them the right feedback to help build on the skills.

Angela McFarlane, director of the Centre for Research into Educational ICT at Homerton College and the author of a new ILS guide, published by Becta, is wary about the claims that ILS can increase pupil motivation. “Enjoying an activity does not mean that you are learning from it. I’m not suggesting that learning should not be fun, but simply that an ILS that makes children more positive about a particular task is not a sufficient enough reason for investing it.”

Ruth Lawrence admits that in the early days there was a lot of hype about the potential to improve pupils’ learning. “The first thing to be clear about is that a learning system is not going to do the job in itself. It’s a tool that can have highly variable results based on the way it is implementedI An ILS is not a panacea.”

She says that successful implementation means having the senior management of a school behind ILS, offering timetabling support, having clear objectives, ensuring that the implementation model is agreed by the people using it, making sure that others in the school are aware of what the software is and, finally, having an initial and a continuous training programme.

Tony Wheeler, of Tag Developments, which markets the Academy of Reading ILS, adds: “We see ILS as a useful part of a balanced ICT programme. The problem is that some systems have been given to technicians rather than to people interested in learning and children. This has disenfranchised some teachers. But an ILS used as part of a larger learning framework can be very valuable.”

Last month, Becta released yet another ILS publication, ILS - A Guide to Good Practice, written by Angela McFarlane, which offers schools advice. Peter Avis, co-ordinator of Becta’s chief executive’s policy unit, said. “Through our earlier reports, we got a good idea of how schools were using ILS. There was a feeling that quite a lot of schools were not getting the best from their ILS investment. This publication is not an endorsement of ILS, and our advice would be to read all the evidence and make up your mind. ILS is not a short-term investment.”

So, with all the conflicting opinions, how can schools make up their minds about ILS? Gerry Daish says: “We invite schools to contact our reference schools and speak to someone who is using ILS with their children.” Cynics might be wary about contacting a school recommended by an ILS vendor but, as Ruth Lawrence points out, “It’s not in anybody’s interests to have schools that are unhappy about using ILS. We have actually discouraged some schools from purchasing an ILS because it clearly would not have worked with them.”

Manor College of Technology in Hartlepool, which caters for 11 to 16-year-olds, has invested pound;73,000 in ILS, using Longman’s Tomorrow’s Promise literacy program and ScanTEK 2000, a design and technology program from LJ Technical Systems.

“We wanted a system in place that would allow us to concentrate on specific problems. We used ILS with all ability levels. It encourages children to take the next step and gives teachers flexibility and the ability to work with small groups. When you see the power of these products, you realise its potential,” says Martin Robson, the college’s director of technology.

At Henry Mellish School in Bulwell, Nottingham, pupils have been using RM’s SuccessMaker for several years. The program is used with about 120 Year 7 pupils, who each spend 15 minutes on SuccessMaker four times a week for almost three full terms.

June Gourlay, the school’s educational and behavioural difficulties manager, says: “We believe 100 per cent in SuccessMaker. Not only has it helped to raise the reading ages of our pupils, but they have transferred their skills and motivation into the classroom.”

Teacher involvement is also important. “An ILS has to be properly managed - you can’t just plug in a pupil and say, ‘See you later’. The progress pupils make has to be reinforced by the teacher.” Good support from pupils and parents is also vital for success, she adds. “The progress we’ve achieved is not solely the result of the ILS. It works in combination with other support programmes. But the ILS has made a significant impact.”

Bournville School for 11 to 18-year-olds school, in Birmingham, is using Tag Developments’ Academy of Reading, and has seen some impressive learning gains - one pupil’s reading age leapt by almost five years. But Julian Sybbald, its head of special needs, says such results should be treated with caution. “The ILS was used in conjunction with a number of activities designed to help improve reading skills, such as small-group work, reading clubs and home reading schemes.”

Even so, Julian Sybbald is enthusiastic about ILS, provided they are used correctly. “Don’t let ILS be seen as separate to what they’re doing, or pupils will find it hard to transfer their skills. Don’t leave children alone because they can become demotivated. And close monitoring is important. An ILS is certainly not a time-saver - you need to put in a lot of hard effort to make it work.”

The latest ILS products have moved on from their first-generation versions. Many now include multimedia content and provide more feedback to teachers. Systems such as Academy of Reading allow staff to add their own content, and others such as Tomorrow’s Promise are modular, allowing teachers to mix and match learning activities.

“Teachers are looking for flexibility and a modular system offers that,” says Daish. “The early ILS products were too prescriptive.”

Ruth Lawrence defends systems such as SuccessMaker, which take a more closed approach. “If you are a teacher who is prepared to put in the extra time, then that’s fine. But I would add the health warning that you need to be clear that the developer of these modules has had some consistency. If modules have not been developed from a single philosophy, they could overlap or even repeat what has already been done.”

ILS technology is improving, although most systems are still restricted to multiple-choice questions or clozed text exercises. But perhaps the biggest challenge is to design integrated learning systems that not only show the correct answer but can explain the reasoning behind it.

“My own feeling is that it’s not that easy,” says David Wood. “Helping children see the relationship between solving different problems is a discursive activity, and good teachers are making these connections on the fly. You don’t have this degree of flexibility with a machine. There’s still some way to go in the design of these systems.”

So should schools invest in ILS? “I’d love to be able to say that this technology is great and that you’ll value it, and that kids’ exam results will improve, but I can’t,” says David Wood. “Schools should ask the question of where they are with ICT. Do we have a clear sense of direction and are we making the most effective use of ICT?” June Gourlay says schools should talk to teachers using ILS. “Come and see people like me! We have used ILS for several years and have the results. I feel I have the evidence to prove the value of ILS.”