, they warn that the regulator has relied on research that “overemphasised” the unreliability of examiners and “tended to ignore more positive evidence”.The Cambridge Assessment paper also raises concerns about the National Reference Test (NRT), which Ofqual wants to be used alongside statistics in GCSE grading decisions.
The test - which will cover only English and maths - will not provide a “strong source of evidence for genuine changes in performance in other subjects”, the paper says. This contradicts the views of Ofqual chief regulator Glenys Stacey, who has said that the NRT would need to look only at the two core subjects because they are “good predictors of general achievement”.
The Cambridge Assessment paper takes issue with a summary of research by Professor Jo-Anne Baird, director of the Oxford University Centre for Educational Assessment, which Ofqual has cited to support its case for moving away from examiner judgement.
The exam board says that the studies used by Professor Baird also show that examiners have “a good sense” of how grade boundaries should change and that using their judgement is “entirely appropriate” as long as it is informed by statistics.
In 2017, the first year that the new GCSE exams will be taken, Ofqual will use the results of primary school tests sat five years previously as the key reference point for setting grades.
The watchdog is undecided as to whether examiner judgements of pupil work should be used again in future. But the Cambridge researchers insist they “should never be discarded”.
“Although there are certainly reasons to avoid using expert judgement as the sole means of maintaining standards, much of the existing research used to decry the use of expert judgement is more positive about its potential than is often acknowledged,” they write.
Last month, the Association of School and College Leaders (ASCL) expressed concerns that Ofqual’s plans to base grading on historic statistics could lead to similar exam performances being awarded different grades in different years. They warned that this would confuse parents and employers, make it hard for employers and universities to understand what results meant, and make a mockery of Ofsted’s method of judging schools.
Professor Baird said: “Most standard setting methods include statistics and expert examiner judgements. That is also the case in England today.
“A convincing, evidence-based rationale needs to be made to move away from the statistics. Cambridge Assessment’s reinterpretation of the existing research does not convince me.”
An Ofqual spokesperson said: “Examiner judgement currently plays a role in the setting of grade boundaries. However, in the new GCSEs, boundary setting will be statistically driven for the first year. This is to ensure that no advantage or disadvantage is given to students simply through the system changing.”
He added that Ofqual had invited the Cambridge Assessment researchers to give a presentation on the alternative ways of using examiner judgement that they favoured.
`The use of evidence in setting and maintaining standards in GCSEs and A-levels’ by Tom Benton and Tom Bramley will be published at cambridgeassessment.org.uk
