The degree to which Integrated Learning Systems (ILS) have been evaluated in schools in this country is unprecedented. An ILS consists of a body of curriculum-related software tasks; these are presented to students and their performance is recorded. The order of task presentation, and the analysis of scores is mediated by a software management system. The concept is so seductive: it promises to solve the problems of underachievement in literacy and numeracy in as little as 30 minutes a day. Computers are used to teach basic skills more effectively than the teacher alone.

A major evaluation, funded by the Department for Education and Employment (DFEE), and managed by the National Council for Educational Technology (NCET), has already produced substantial findings and is now in its fifth phase. There have also been countless studies in other countries, mainly the US and Israel as well as Australia. The latest study to report in the UK is the Technology Colleges Trust’s Integrated Learning Systems - First Year Report. But no matter how many evaluations are funded, not one has come up with the incontrovertible evidence that would justify major government investment in ILS for all.

Interestingly, all major studies have fought shy of doing direct comparisons of different ILS products, and the TC Trust is no exception. This is a great pity as it is probably the key piece of information that any school considering the substantial cost an ILS represents would really like to know. They did look at four products: SuccessMaker; Global Maths and English; Learning Expedition (all featured in the NCET report); and the Plato Curriculum.

Unlike the NCET-managed study, the TC Trust did not use independent measures of results, but relied on those provided by the ILS themselves. As has been pointed out in many reports, this makes interpretation very difficult. Systems tend to place students rather low when they do their initial assessment, and unfamiliarity with the assessment tools and methods of presentation used in the system can aggravate this. The result is impressive gains at first as the students become more familiar with the system. Additionally, it is impossible to know if the apparent skill gains are transferable - will the students’ spelling, for example, be better when they are writing a story on paper?

The NCET study, designed and carried out by Leicester University, used rigorous methodology and independent measures of learning gains. The results they confirmed, relating to numeracy gains in students using SuccessMaker, were undeniably real and spectacular. They did not, however, confirm gains reported by other systems, or by SuccessMaker in literacy. It is therefore difficult to see what the TC Trust reported gains really mean. The internal reporting systems in an ILS always show pupil improvement.

The TC Trust report does look at the role of the teacher closely. It finds that the systems cannot replace the teacher, and that the better prepared the teacher and the more integrated the use of the ILS with the rest of the curriculum, the better the result. This mirrors the findings of the NCET report, and of the considerable work relating to the role of the teacher published by Roy Clariana.

The conclusion to the TC Trust report points out that its findings reinforce those of other studies. Implementing some forms of ILS does have some advantages under some circumstances. Not all products are the same. They must be implemented fully, with appropriate staff training. This is expensive, but cutting corners will lead to failures. There is nothing new to report from the TC Trust study, but the weight of evidence is important. It is unfortunate however that this does not include independent assessments of results.

There are still many questions remaining to be answered about the use of ILS. I will leave the final word to Roy Clariana, writing in the Journal of Computers in Mathematics and Science in 1995: “No one instructional method is appropriate for every learner under every circumstance. An ILS, like any good instructional method, provides an alternative and a supplement to classroom presentations. An ILS is only a part of the total learning environment but may change any or every aspect of that environment. ILS research should now focus on how an ILS can change the total learning environment, with particular attention to the teacher.”

Integrated Learning Systems - A TC Trust project - First Year Report Pounds 7.50, TC Trust, 9 Whitehall, London SW1A 2DD