In what would be one of the boldest changes ever made to the exam system, qualifications officials are considering scrapping detailed mark schemes for GCSEs and A-levels and instead relying on examiners’ judgements of the overall quality of a script.
Under the revolutionary system, instead of allocating candidates marks for each answer, examiners are given two computer-scanned scripts to compare “holistically”.
The only judgement they make is which one is better overall. This is then fed into the computer.
Each script is compared with at least nine other pupils’ papers, by different examiners, allowing the pupils to be ranked. As the process progresses, scripts with similar rankings are compared again, with each other, allowing increasingly fine judgements.
The computer can work out if standards have improved on the previous year, because a number of papers from the previous year are included in the comparisons.
It also sets grade boundaries, based on the idea that a paper from the current year should get the same grade as one of similar standard from the previous one.
The system is being investigated for the Qualifications and Curriculum Authority by academics at the University of Cambridge Local Examinations Syndicate (UCLES) and the Welsh Joint Education Committee.
Given public concern about the reliability and consistency of examiners’
judgements, boards have produced detailed mark schemes to ensure any piece of work will get the same score if assessed by different markers.
But Alastair Pollitt, leading the research for UCLES, said that this produced exams which favoured questions susceptible to mark schemes, even if this was inappropriate to the subject.
A holistic assessment would remove this problem and also could produce more accurate grades because the results pupils get through a mark scheme may not always match examiners’ assessment of the overall quality of their work. Mr Pollitt said although the sytem relied heavily on professionals’
judgements, it would be easy to check by computer whether any markers were out of line with their peers.
Later this year, the two boards will do trials of the system using papers from this year’s A-levels. But the boards are cautious about relying on unproven technology and the QCA has no current plans to introduce the system.
A source at one of the boards said that although the new system might eventually be used - for checking coursework marks, for example - its adoption for major GCSE and A-level exams remains a long way off.
