Unless you have spent the past 10 years on the planet Zork, you will have noticed that the Department for Education has had a pre-occupation with “basic skills”. The official view is that children in UK schools are not achieving high enough standards of numeracy or literacy. Against that background, it was no surprise when the DFE, with the support of education departments in Northern Ireland, Wales and Scotland, commissioned the National Council for Educational Technology (NCET) to manage a limited trial of integrated learning systems (ILS) in the UK.

An integrated learning system is a large resource bank of learning material stored on computer, with management software which can monitor the interaction of pupils and record their performance. The most common implementation in schools involves the use of a dedicated network and server carrying the content, which may be supplied on as many as 15 CD-Roms.

Children log on to the system for a short period each day, don headphones and complete a set of tasks presented to them by the management software. Decisions about the content of a session are based on recorded “scores” from previous occasions. The result should be a course which suits the pace and ability of each student, thereby allowing individuals to achieve their full potential.

These systems, which have had millions of dollars pumped into them in the US, have proved effective in boosting basic skills in some American schools, particularly for remediation in mathematics.

On Monday the NCET will publish the findings of a six-month UK evaluation. Twelve schools have been trying out one of two systems, the maths and language elements of SuccessMaker from a US company called CCC, which is marketed in the UK by Research Machines, and Global Maths a UK product from US-owned Systems Integrated Research. The two systems are so different in style, scope and content, that the report points out they cannot be compared. Rather it was the concept of ILS that was under scrutiny.

Children, mainly in key stages 2 and 3, used the systems from January to June 1994. A team led by Jean Underwood at Leicester University carried out an independent evaluation, within strict terms of reference defined by the NCET. They looked at learning gains in numeracy and reading skills, together with changes in motivation and attitude. Factors such as gender, age, ability and learning style were taken into account. They were also asked to identify features of an ILS which support learning, comment on effective implementation strategies, and look at transferability of any skills acquired to the rest of the curriculum.

A pilot and a control population of pupils for each system were identified. Both followed the normal curriculum, but in addition the pilot used the ILS for approximately 30 minutes a day. These populations were not necessarily well matched the researchers had to use children who were available in the trial, rather than set up special groupings. The children in both groups were tested in basic numeracy and reading before and after the trial, using standardised non-verbal reasoning scores which enabled some recognition of varying student ability when interpreting the outcomes.

Initially pupil motivation was found to be high, but there was evidence of a decline over the six months. The recorded learning gains are interesting, not least because they are the opposite of the pupils’ and teachers’ own views of how they did.

Users of the American CCC maths program made a 20-month advance during the trial, compared to the six-month gain of the control group. A “month gain” is not defined in the NCET report, nor does it mention how many children made this gain, what the spread of ability was, or the range of gains made. And no attempt has yet been made to explain why pupils using the CCC maths program made this gain.

For pilot users of the UK Global Maths and the US CCC language materials there were no conclusive gains. Extenuating circumstances for this are argued convincingly.

The report finds that pupils generally display a high level of time on task and concentration, and that ILS classes are generally quiet and well behaved. Children also show evidence of an increase in responsibility in managing their own learning. However, we do not know if they were badly behaved in non-ILS lessons or if they had ever been given the opportunity to take any responsibility for their own learning before the trial.

There are inevitably serious problems with evaluating the impact of a major education resource, especially one with the cultural impact of an integrated learning system. It might seem unrealistic even to attempt to do this in six months. However, the Leicester team and the NCET, together with the school staffs, have managed to gather some valuable information in this short time. For example, some students have benefited particularly from the instant feedback and results, the content matched to their ability and the opportunity to work quietly and privately, which ILS provides.

The NCET report contains a section on implementation which schools considering ILS should address. This covers issues such as student reports which should be easy to read and informative, enrolment, adjustment of content to student performance, breadth of curriculum coverage, mapping to the curriculum, and cost. No integrated learning system currently meets all of its recommendations.

There are indications in the report of problems with both the US and the UK systems tested. The CCC product was designed for the US and children in the UK trials complained about its language and content. Some found the American accents difficult to understand.

The volume of material in the American system is vast. It covers children from kindergarten to US grade 12, and comes on some 15 CD-Roms. Last year alone CCC is reported to have had a budget of $26 million to spend on content development. Schools here would have to spend a lot of money before extensive UK versioning of the material would be commercially viable. (CCCsays it will include “UKelements” in 1995.) The American system selects material for each pupil session, which provides individualised experience but can remove control from the teacher. In the British system the teacher reviews the pupil scores and decides which work plan is appropriate for each pupil. In the trials this usually meant that pupils did not get an individualised work plan, but a class would be allocated two or three. It is difficult to see how you could make 30 valid work plans, given the volume of content in the GLS program the maths curriculum for key stages 1-4 fits on a single CD-Rom.

An announcement of Pounds 100,000 from the DFE to develop courseware for the “Open Integrated Learning System”(OILS) which is being promoted by Systems Integrated Research and other UK developers, may sound like a lot of money, but it is unlikely to produce a vast amount of extra content.

So, where to spend your money? It would be inadvisable to do anything before reading the NCET report and the fuller report from the Leicester research team. And it would be safer to wait until the next phase of the evaluation, just announced by the DFE, reports next autumn. That should establish whether any of the gains seen so far are real, sustainable and transferable.

SuccessMaker starter network package from RM Learning Systems Pounds 850 per work station, for five courses, including material for maths and English. Contact Barry Taylor, 01235 826700 Global Maths from Systems Integrated Research, Pounds 250 per user for the Global Maths CD-Rom, management software for 10-user licence, Pounds 495. Contact Tom Powell, 0773 820011 Evaluation reports from NCET Sales, 01203 416994. NCET report Pounds 5.50, Leicester report Pounds 10.

KEY FINDINGS * Children working on the American CCC ILS maths program performed significantly better than children working in control groups, making gains of 20 months over the six-month period * Behaviour was found to be good when using the ILS, with longer time spent on task and a calm working atmosphere maintained * Primary and secondary students were found to maintain a higher level of attention during the ILS sessions, though primary students showed some deterioration of behaviour over the trial period for post-ILS lessons * Pupils with behaviour problems seemed able to concentrate more during ILS sessions than in normal classes * Students were observed to work constructively on the ILS for about 15 minutes at a time. Lower-ability students tended to find even this beyond their attention span * Many students have shown an increase in responsibility in managing their learning.