Literacy and numeracy standards are being highlighted as never before, so anything that looks as though it is going to be a trouble-free, fast track to success gets attention. An integrated learning system can sometimes seem like such a panacea.

The third and final report from the British Educational and Communications Technology Agency (BECTA) on a pilot scheme in British schools is particularly important. But anyone looking for clear messages will be disappointed - this is a complex subject.

An ILS is a computer-based system that manages the delivery of curriculum materials to pupils so that they are presented with individual programmes of work on screen. It will diagnose difficulties and analyse progress, presenting reports for teachers.

What schools require from these BECTA reports is simple. They want to know what an integrated learning system is, what it does, what is available. They want to know which systems are successful, how much they cost and where they can be obtained.

But the report does not answer all these questions. Maybe they will be answered in some support documents for headteachers, expected in six to nine months from now.

BECTA’s chief executive, Owen Lynch, says the new systems are tools that must be used in a sophisticated manner if they are to blend effectively with an individual’s study. He feels that, at their present evolutionary stage, the systems give a tantalising glimpse of the future.

Owen Lynch sees today’s products as forerunners of systems where the main power lies not in the delivery of information but in the quality of their analysis of students’ learning.

The use of ILS goes to the heart of what learning is about, though there is no doubt they are controversial. Some see them as mechanistic, while others argue that they should not be included under the banner of information and communication technology.

Nevertheless, past reports have found significant learning gains in certain aspects of maths. Those gains are not exactly replicated in this BECTA report (in one case, the systems are said to have caused a fall in standards). And all three reports have been unable to identify significant learning gains in literacy.

One thing this latest report is quite clear on is that care must be taken when interpreting data from such systems. Earlier this year, the Technology College Trust published its own research. Though its report was much warmer towards ILS than any of the two previous reports, it pointed to some clear gains - results which were, in the main, derived from the systems themselves.

The elementary safeguard of calibrating results against some external measures was forgotten. No one seems to have realised that SuccessMaker, a leading ILS product in this country, inflates any gains quite legitimately in order to boost the student’s confidence. Many schools make the same mistake, regarding the gains reported by the system as “objective”.

The BECTA report describes some positive aspects to these products. A good system can analyse a student’s difficulties and reporting mechanisms have been developed to a high degree. The detail that can be given to parents is excellent. Pupil motivation also appears to improve.

Barry Taylor, who is responsible for ILS developments at RM, the education ICT supplier, feels the report confirms RM’s belief that the implementation model adopted by schools is vital to the successful use of this kind of software.

The response from SIR, the company behind the Global open integrated learning system, illustrates the complexity of the issue: “The main software systems used in the evaluations were almost incomparable in size, cost, role in the classroom and pattern of implementation. The result has been confusion over the term ILS and a report which relates, in the case of Global material, to only a very small part of an open system.”

Companies such as RM, Longman and SIR have a great deal invested in these types of systems. They need help and guidance to make them more effective. And schools need help to devise strategies in using them.

After spending Pounds 1.3 million of the taxpayers’ money BECTA now possesses the most extensive body of research into ILS in the world, leading the evaluators to conclude in their report: “Integrated learning systems have a long way to go before they can receive unqualified endorsement.”

Collaboration between BECTA, research groups and the computer industry could in future produce a more positive comment. At present, though, anyone thinking of using an integrated learning system should read this report very carefully indeed.

Any work on integrated learning systems should not hinder urgent investigations into how mainstream uses of ICT, such as word processing, authoring, desktop publishing, simple programming, spreadsheets, information retrieval, Logo and control, can assist both numeracy and literacy.

The UK Integrated Learning Systems Evaluations: Final Report costs Pounds 7.70from BECTA 01203 416669