Unless you have been sojourning on the planet Zog you must have at least heard of Integrated Learning Systems (ILS) by now. These are essentially large banks of tutorial and test exercises held on computer. The order in which students tackle the material is controlled by a management system which also keeps a record of their achievements. The first such products concentrate on exercises in maths and language skills.

Great things are claimed for ILS. However, it is very expensive so, rather than wade in with the cheque book, a number of bodies have commissioned independent evaluations. The biggest of these is the project managed by the National Council for Educational Technology (NCET) for the Department for Education and Employment (DFEE), which has already published two substantial reports. These show variable results. Sometimes, and this is by no means universal, the gains are spectacular; mostly they are not.

We can learn one thing from these evaluations: it is pointless to talk about the use of ILS with children as though it were some uniform experience, with uniform outcomes. The most crucial variable is which product is being used.

The NCET has not carried out direct comparisons of the different products, but the City Technology College (CTC) Trust has just commissioned such a study. It will look at the four products on sale in the UK: SuccessMaker, Jostens (now looking for a new distributor after its deal with Heinemann fell through), Plato and GlobalOILS. The background paper announcing this welcome comparative study is surprisingly positive about the concept of ILS, given the genuine variability and overall disappointing results reported in the research papers it cites. The starting point seems to be that any ILS is good - but which one is best?

One curious thing about this system is that it seems to raise blood pressure in a number of quarters. Commentators are violently for or against its approach, so trying to reach an objective view is very difficult. For example, a salesman whose product did not perform at all well in the first NCET trials was happy to report the finding that ILS was responsible for massive learning gains in maths - true, but it was from a different company. It’s a bit like asking someone to bet on your horse because another horse won the Derby.

So far, the studies in the UK have not produced a list of features which distinguish the effective ILS system. The key is likely to be in the management system. This is the software that determines the exercises the pupil gets when she logs on to the system. It keeps track of her score which it reports back to the teacher.

In the most advanced systems, the management software determines which exercises the child gets, choosing from a vast range according to the mastery shown by the pupil. No two children get the same set of problems, and each session offers material from a range of topics, not just endless essentially identical sums. Teachers can intervene if necessary, for example to exclude a strand or topic. It is this type of system which has proven effective.

The alternative is that the teacher alone determines what the student does, and usually from a smaller range of materials and with less scope for differentiation. Other than making them do the units in a unique order, it would be impossible to determine individual work plans for each class member. This is the model used by the systems which have yet to proveeffective.

This difference is, however, often presented as a virtue because it supposedly offers the teacher better control over the system. Which is then equated with the research findings that ILS works best when the teacher integrates its use into the rest of the curriculum. These two things are very different.

When the rest of the experience of maths education that a child has is complemented and reinforced by the materials that such systems offer, it is hardly surprising that outcomes are improved. Compared to the alternative extreme - where the teacher sends children to another room to use the ILS and makes no reference to it in any other activity - the integrated approach has to win. However, these scenarios are not about the teacher having to manipulate the system so that he chooses which exercises each child does. “Teacher control” is offered as compensation for the absence of a powerful management system, driven by a knowledge of pupils’ performance based on years of observation and data collection, and backed by a vast array of curriculum materials developed with vast budgets.

In an attempt to compete with the mature products from the US, British software manufacturers have developed something called an Open Integrated Learning System (OILS) protocol. This is a system which will launch a variety of curriculum software, collect data on pupil use, and present that data in a form which can be read by other systems, such as the school administration software. All of which sounds very promising.

The latest version allows for the passing of data on time spent and number of items attempted, as well as scores. However, this assumes a wide variety of computer-based learning materials, the scores from which accurately reflect pupils’ skills and knowledge, and which can offer a number of routes through materials, varying context and approach, until the one that suits the pupil is found. Whether or not OILS-compatible products can actually do this is a good question to put to the people who would like to sell you their software.

* BETT CONNECTIONS

Global (SIR) stand 660

OILS look for the logo on members’ stands

Plato Tro Learning (UK) Ltd stand 302

SuccessMaker (RM) stand 136