Computers are increasingly part of the furniture in classrooms but when it comes to formal assessment, says Martyn Ware of the Scottish Qualifications Authority, “we are still sitting kids in a big hall behind a shoogly desk with a pencil and paper.”

It is a model that is increasingly untenable, believes the SQA’s business manager for computer assisted assessment. The benefits of bridging the gap between the classroom and the exam hall include fairer and less stressful assessment practices, which are integrated more closely with learning and teaching.

The change to online assessment will not happen just yet. Anton Colella, the director of qualifications, pointed out recently: “A great deal of research still has to be done in order to make such a system deliverable and flexible.”

Mr Ware, remembering the traumatic events of 2000 when changes to the exam system were made with insufficient preparation, believes the SQA has the balance right this time: it is cautious but engaged.

“Experience with online assessment is demonstrating a number of positive findings: The systems don’t fall over every time you use them. They are secure. Candidates often quite like to do assessments online. Teachers and centre staff can understand and work with the software,” he says.

Aspects of computer assisted assessment are being explored in projects that include a study of e-moderation and a pilot of 5-14 formative assessment in Dumfries and Galloway. The two flagship projects are Understanding Standards, an online guide on exam marking, and Pass-It, a pilot of online assessment for the internally assessed components of national qualifications, especially Highers.

Pass-It projects in maths, computing, chemistry, music and French at 17 schools and further education colleges around the country will end in October with the delivery of a final report. It is expected to conclude that online assessment brings considerable benefits and should be more widely available.

“I couldn’t hang my hat on a date by which it will be available in all subjects for all exams,” says Mr Ware, “but we now have evidence from a well-resourced, well-run research project that online assessment works.”

Balerno High in Edinburgh is one of two schools at which candidates have been testing online assessment in earnest. Two Higher maths classes sat their end-of-unit assessments online in January.

The students were apprehensive when they first heard that a computer would be assessing them for real.

“I was a bit anxious,” says fifth-year pupil Jamie Wishart. “I didn’t know what to expect.”

“I really didn’t want to do it online at all,” says Robert Dewar, who according to colleagues is a strong maths student. “I lay out my working with weird bits all over the place. It make sense to me but it wouldn’t to a computer.”

Their anxieties were partly allayed by learning that the assessment is not yet fully automated and retains some level of teacher involvement.

“In fact it wasn’t too bad,” says Robert. “We weren’t just typing in answers on the computer. We were also given paper for our working, which our teacher could use to give us some credit even if we got the final answer wrong.”

Maths is a subject which lends itself better than most to online assessment. However, the difficulty with computer marking is getting a machine to judge a person’s expertise.

“In subjects such as chemistry, where you have to give explanation type answers,” says Robert, “it almost has to match word for word or you get marked wrong. Different words with the same meaning won’t do.”

“The USA is often cited as a country where online assessment is the norm,” says Mr Ware, “but it tends largely to be multiple-choice testing. If we wanted to go down that route, we could do it quickly because it’s established technology.”

Instead, technology that can cope with more open-ended answers is being explored and developed, by experts such as the Scottish Centre for Research into On-Line Learning and Assessment at Heriot-Watt University.

Marking of explanation answers is also improving as moderation by experienced teachers builds up a bank of acceptable answers. “The more we use it the more reliable it becomes,” says Campbell White, the SQA’s centre liaison officer.

A key test of online assessment is whether the system would award significantly different marks to an expert marker. So far the answer has been no. Statistical analysis of phase 1 results shows the computer performing essentially as well as a person.

Pilot projects are set up to identify practical problems and, inevitably, there have been some. Maths teacher Christine Bond says Balerno High’s computer system went down just before the holidays. “So I still have one or two Higher students who haven’t completed the assessments,” she says.

“We’ve also had access problems. One group were getting on fine when the bell went and they had to stop because another class needed the computers.”

However, the teachers and pupils taking part in Pass-It found a lot to like about online assessment. Reduced teacher workload was a big bonus. “I’d like to see all end-of-unit assessments online as soon as possible,” says Ms Bond. “Pupils would soon get used to it.”

“The instant feedback is a major advantage,” says S5 pupil Jamie. “Knowing that you have passed right away, instead of waiting a week, is great.”

“The notation took a bit of getting used to because it’s different from the usual maths notation,” says Erin McCanny, “but I like the way you can practise as much as you want. It makes the actual assessment much easier.”

Mr White believes that a large number of teachers are not yet ready to embrace online assessment. “They will accept it as an alternative to paper but they don’t want to be told to do all their assessments online,” he says.

This could change as the large proportion of Scottish teachers aged over 45 retire and as teachers become more proficient and confident with information technology.

Already Mr Campbell has seen resistance to computer assisted assessment crumbling. “There was one session where we were talking about putting French comprehension assessment online. At first the teachers didn’t like the idea but they gradually realised that many things were done in a certain way - listening twice to a passage, for example - because it suited the teacher and the technology available at the time. As the day went on you could see the enlightenment dawning on the teachers’ faces.

“New technology means we can do things better.”

FUNDAMENTAL CHANGE FOR ASSESSMENT

The Scottish Qualifications Authority has defined computer assisted assessment as “assessment activities supported by the use of information and communication technologies”.

This broad definition includes but is not limited to assessments that are delivered to schools by information technology, to be downloaded, printed and used as normal; those that are taken by candidates at a computer, with the scripts printed off for marking; and those that are delivered by ICT, taken at a computer and marked by a computer.

One option the SQA is not currently pursuing is scanning exam scripts and sending them electronically to markers. “We have not seen strong enough evidence that this works well enough,” says Martyn Ware, the SQA’s business manager.

There are good reasons for involving computers in marking, he says.

Currently, there is a gap between assessment on one hand and learning and teaching on the other, which benefits neither.

There is also the unfairness of asking candidates to work through a vast amount of material over a year and then having their future depend on their performance on one particular day. “I think that’s an untenable model in the longer term,” he says.

Most of all, there is the potential for better, more authentic assessment.

“In future kids might work through lifelike scenarios on a computer and be assessed as they go along.”

None of this envisages dispensing with the teacher, says Mr Ware. “We want to integrate ICT into teachers interacting with kids.

“In Scotland we have a rich variety of assessment for national qualifications, with teachers having a big input to the final marks, through a project, a piece of work, a performance. We don’t want to lose that; it’s a huge strength. But ICT does offers the opportunity to fundamentally change what we assess and how we assess it.”

ONLINE GUIDE TO EXAM STANDARDS

Exam marking has traditionally been a black art, but this is set to change, says the Scottish Qualifications Authority’s business manager, Martyn Ware.

The Understanding Standards website lets markers practise their skills on exam scripts, then instantly compare their efforts with official SQA marks. Teachers marking Highers this year will be able to hone their skills before they start.

Better marking is not the only benefit: the website is also being made accessible in a controlled way to classroom teachers. Seven hundred have access to the password-protected site, which offers materials for eight subjects.

“We want to expand the range of subjects and levels and make it more widely available,” says Mr Ware, “but we need to be sure we can satisfy the demand.”

Feedback from teachers has been positive, he says. It ranges from “Best thing in ages” to “Refreshing to come to a useful in-service day which directly contributes to teaching” to “I felt as if I was at the Berlin Wall when it came down.”

Lyn Smith, the principal teacher of English at Banchory Academy in Aberdeenshire, says: “The SQA introduced us to the website, then we had a hands-on session on Higher close reading and critical essay marking. We marked the candidates’ responses, then got a commentary on how it was marked by the SQA. I found it really useful and came away all enthused.

It’s great to get the SQA’s voice in the school; it will definitely help the confidence of staff.

“We also had a very useful in-service day on marking, but the advantage of the website is you can go on at any time and you don’t need to find a whole day.”

Graeme Nixon, the principal teacher of religious, moral and philosophical studies at Aboyne Academy, Aberdeenshire, says: “Potentially the website is very useful for teachers. In practice there aren’t enough materials in our subject yet. For example, philosophy Higher isn’t available yet.

“One thing I would like to see is some kind of forum. On one paper the SQA awarded an A, but most of us on the course didn’t think it was that good.

I’d like teachers to be able to discuss that sort of thing with markers.

“I am very positive about the initiative and the new openness it demonstrates is very welcome.”

Gordon Doig, acting depute head at Ellon Academy in Aberdeenshire and an SQA marker who helped to develop the online physics materials, says: “Kids do better in exams, I believe, if their school has an SQA marker in that subject. Making the website available to all teachers means that all kids can get that benefit. It gives teachers instant feedback when they’ve marked a question.

“The response from them has been very positive. It’s now available in all Aberdeenshire schools.

“There was talk of using the system to do online assessment for new markers, but that hasn’t happened yet.”