May was an unusually busy and enjoyable month for me. It was particularly enjoyable, because I got out of the office and visited a number of different schools to watch Year 9 pupils take the new key stage 3 ICT test.

I watched Mark shuffle in, sit down, log on. He looked relaxed and in control, and appeared confident. From where I was sitting I could see he had completed some excellent responses to the on-screen tasks. At the end, I asked if he would mind talking to me for five minutes.

It was clear he knew he had done well: “I usually work at about level seven,” he said. He was especially proud to tell me about the technology he used every day at home. So I asked him what he thought I could do to improve the test.

“You should be able to open more windows at the same time,” he said. “I have about nine open at once. And it’s too slow. I wanted to be able to go on at my own speed without waiting for the computer to give me the next task.”

Mark had a lot more suggestions he wanted to tell me about, but he also thought other statutory tests should be on-screen, too.

So, was this the best on-screen test we could produce? I’ll come back to that question. The reason I visited so many schools in May was to hear at first hand what we had got right and what needed to be improved.

I spoke to a network manager who told me that his school had no trouble in downloading the test. One ICT teacher said that the pilot had thrown up a lot of network issues - a good learning experience for her, but she wasn’t feeling really pressured as the test was not yet statutory.

Many teachers told me how much their pupils enjoyed being the first to experience a high-quality on-screen test. They said they knew on-screen, computer-delivered tests were the future of assessment.

So, did the test work well?

Overall, I am extremely pleased. More than 45,000 pupils entered and received their results. Beyond any doubt it is now clearly feasible to run a high-stakes national test on screen. And we have a major innovative test ready to roll out to every secondary school in England.

Pupils certainly enjoyed it. After one session I asked one of the them what she would tell next year’s Year 9. “I would tell them not to worry about it, because it was a lot easier than we thought - we were all worried about it. I would just tell them to stay calm and get on with it.” She said it was hard, but interesting - “a great test, and I would recommend anyone to take it.”

Another pupil added that it was “more fun than your average test”.

It seems to me that these pupils responded well, as did schools in managing the administration. I had expected only 12,000 pupils to complete the test, but teachers administered almost four times that number.

There were lots of clever ways in which teachers organised the classroom to take the two 50-minute sessions. Some ran them in scheduled ICT classes with the two sessions a week or two weeks apart. Many ran them back-to-back in a single day or on consecutive days. Both models worked well.

The design of the test also worked well. One teacher commented: “I thought they were very impressive. They tested all the applications students had been learning and they put them into the same sort of context we use in lessons using the KS3 strategy. So it was quite familiar to them, even though there were different applications than they were used to using. It contained a good range of material we have covered over the three years.”

So, if there was general praise and good feedback, what did we learn? The short answer is, an enormous amount. Here are my top four lessons - the first two are for schools and the second two for the QCA (see box). And these are not the end of the matter. We are also making other important improvements that schools have asked for.

This year, pupils did not like being constrained by a timed task window.

Next year, they will be free to move through the test in their own time. We have also improved the functionality of the toolkit for pupils, and teachers will find we have made the administrative system easier to use and more efficient.

What next?

The first step is to register, if you haven’t already done so; you can do this online (see below). Our support service will provide help at all stages. Government technology agency Becta will also give advice on technical issues.

The KS3 ICT test is redesigning testing in schools. I believe every pupil has the right to experience a test of their ICT abilities, which is the best available anywhere in the world. A full report of the pilot is on our website.

* www.qca.org.ukeassessment You can register at www.ks3ictpilot.com

Martin Ripley is head of e-strategy at the Qualifications and Curriculum Authority

GETTING PREPARED

1 Get involved early

It’s time for schools to prepare for - and get involved in - the 2006 pilot. All schools should be using it, and the 2007 pilot, as crucial familiarisation in running the on-screen KS3 ICT tests, because they represent significant change. Make sure:

* the school network is ready; l at least one technical person in the school is available during test sessions in case of computer problems;

* your school rehearses the procedures, as scheduling a 50-minute test, with time for logging on and so on during a lesson takes practice.

The school that leaves it until 2007 runs the risk of encountering difficulties in 2008, when the tests are likely to become statutory, through insufficient experience.

2 Prepare pupils well

New tests always have significant curriculum effects. The pilot showed that pupils need time to access the practice and familiarisation materials.

However, they also demonstrated an incomplete knowledge of the programme of study, especially struggling with modelling and data-handling. So preparation - and more preparation - will help.

As one teacher commented: “It was a very good test to undertake and I look forward to doing it with next year’s group. I hope we can use the practice materials a bit earlier than this year. I would recommend schools to take part from now.”

3 Practise early

This year, the QCA did not get practice and familiarisation materials to schools until late February. This was too late for some schools to timetable familiarisation sessions with their pupils. As the test is completely new, the practice materials are vital so that pupils can understand what is expected of them. The pilot has shown that they have very little difficulty with the test environment, provided they have time to become familiar with it.

* For schools registering for the 2006 pilot, the practice and familiarisation materials will be released in the autumn term.

4 Rethinking the reports

A major benefit of on-screen testing is giving the learner better, more diagnostic feedback about individual strengths and weaknesses. Teachers should be able to run a KS3 ICT test session early in Year 9 to obtain a detailed report on each pupil. The summative test should also give teachers and learners helpful and detailed reports.

The reports we produced this year did not work well enough. They were too difficult to access and the information they contained was not as useful as it needs to be. Getting the reports right for 2006 is a major priority for us. We will be asking teachers to evaluate our improvements and will listen carefully to school reactions next summer.