Margaret Bell outlines the issues emerging from the NCET’s controversial study of Integrated Learning Systems.

New projects are often misunderstood, and are sometimes intentionally misrepresented. This fate may have befallen the National Council for Educational Technology’s evaluation of Integrated Learning Systems (ILS). In such circumstances, it is easy to become engaged in wrangling and to lose sight of objective and quantifiable results which clearly show that there are benefits to be gained from ILS for learners and teachers.

Some of the students’ comments during the pilot study give an idea of the classroom comments ILS has provoked: “It doesn’t go off and help someone else”; “I like the computers because they’re not too easy and they’re not too hard - they’re just right”; “I like the new system because you instantly know whether you are right or wrong”.

Among teachers, typical responses to the system were: “I have a lot less marking to do, so I can now spend much more time on lesson preparation and individual support”; “The system gives me so much more time to go round and work with individual pupils” (secondary teacher); “I can now concentrate on the more creative things” (a primary teacher).

Both teachers and students using ILS in the NCET’s pilot evaluation made positive comments, and that alone was encouraging, particularly as there was considerable misunderstanding of the role and function of ILS at the time. However, the aim was to collect data on learning gains.

In phase one we looked at two different systems: SuccessMaker, sold by Research Machines, and Global Maths software, from Systems Integrated Research. Despite their marketing as ILS, they differ greatly. SuccessMaker has a management system which automatically moves the learner through a series of tasks, while Global Maths depends on teacher or learner choosing the set of tasks.

We only have results on the outcomes for SuccessMaker, and these showed significant gains in maths, equivalent to 20 months of learning in a six-month period. Schools using Global Maths reported improved exam results after its use for revision purposes, although these results were not measured independently. Could this be an early indication of the need for different systems for different types of learning? (The “automatic” model like SuccessMaker, which takes students through the learning, and the “choice” model which is useful for revision and practice after the initial learning.) Global Maths will continue to be evaluated, but we have to concentrate on the evidence from SuccessMaker.

We found that SuccessMaker is individualised; helps students work to their potential; stretches them (learning is challenging while building on success); motivates most students and improves concentration; saves teachers’ time and allows them to concentrate on creative and social aspects of teaching; provides individualised reports and profiles.

The learning gains found in the first stage of the pilot suggest there is value in a model which automatically and continually challenges. This seems especially true in those parts of mathematics and language needing a structured approach and repetition and practice.

The NCET is not advocating this approach for all learning. We understand the need for more open-ended learning experiences. In particular, we understand the power of combining the highly structured approach of SuccessMaker with the creative work managed and directed by the teachers.

There were signs that the students themselves became more independent and demanding in their learning in class lessons, sometimes surprising the teacher by their awareness of their progress. They would point out topics that they had already learned on the computer or areas that they knew they had difficulty with.

The evaluation has now moved on to phase two and continues to monitor progress of the children in phase-one schools for another year, while extending the project to more schools. If this second stage continues to demonstrate positive results then we will be able to state with some confidence that SuccessMaker has improved learning outcomes. We then face the possibility of employing a computer-based learning system in our schools which could have a real impact on the mathematical (and possibly literacy) skills of our children. The NCET hopes that such systems would be welcomed throughout education.

There is no doubt that proving the efficiency of ILS will raise many issues. I believe that we should attempt to anticipate what these might be and to use the evidence collected so far to start the debate. Some of the issues may prove serious, some insignificant; some may prove easy to resolve, others not. Whatever the difficulties, we must concentrate on solutions which will deliver the gains that ILS appears to offer, rather than seek to identify difficulties and thus prevent children from benefiting from such systems.

Among the issues are: SuccessMaker is American; it is expensive,with inflexible pricing; as yet no other systems produce comparable results; the system is closed and does not allow integration of the learning management system with third-party software; the learning management systems are based on pedagogical models of mathematics and literacy which are not openly known; the use of SuccessMaker may be confused with delivery of IT competence; expenditure on an ILS system may deflect resources from the purchase of IT equipment for other purposes; what level of support for ILS is required in schools?

The first two issues are relatively straightforward: a more international version of the software is about to be released and more flexible packages and pricing are being negotiated. Of course, there remains the question of how schools afford the systems. Interestingly, all the schools in the pilot study have made an investment in these systems, so we should be able to use them as case studies.

The issues of competitiveness and open systems are difficult. Can other companies make the investment such a system requires? If so, will development come from the evidence of this project plus market forces, or will it require central co-ordination or intervention?

Moving on, it is important to draw a clear distinction between ILS and developing IT capability. There is little overlap and no justification for re-directing the scarce resources that schools currently put into IT to integrated learning systems. An ILS is a method of improving learning in, for example, mathematics; it does not develop IT capability.

It is a measure of the maturity of thinking about the roles of IT in schools that we see, in addition to its essential role in developing new skills in children, that IT can also be applied to the process of learning. There is a closer link between ILS and computerised admin than between ILS and creative IT.

Finally, the pilot showed that schools need considerable support for ILS work, and this is often underestimated by both schools and support providers. At a basic level it means having access to technical support which is immediate, reliable and effective.

Whatever we think about the literacy and numeracy skills students should leave school with, we can at least agree that current standards are not good enough by international comparisons. If ILS provides part of the means of improving these skills, then IT will have made a truly significant contribution to learning. The jury is still out the evaluation is not over.

Meanwhile, we must anticipate the issues a successful trial would bring. I have made a start and I look forward to hearing your views.

* Margaret Bell is chief executiveof the National Council for Educational Technology * The NCET is collating views and seeking guidance on ILS. An ILS newsletter will be available on request or via the Internet: at http:ncet.csv.warwick.ac.ukindex.html. A bulletin board will be created on the Internet. Contact Catherine Gow, NCET, Milburn Hill Road, Science Park, Coventry CV4 7JJ. Telephone: 01203 416994. Catherine Gow@ncet.org.uk.