Fears about digital exams don't 'stack up', says AQA boss

The case for on-screen assessment and digital exams has sparked plenty of debate recently - but the required sense of urgency is just not there yet, according to one exam board CEO, who says slow progress on this front is leaving many “baffled”.

“My position has long been that AQA should be in a position to execute digital exams tomorrow morning, if we are asked to,” says Colin Hughes in an exclusive interview with Tes.

The obvious question about this stance - and it applies to other exam boards, too - is: given that delivery of more traditional pen-and-paper exams is not exactly a stellar success story at the moment, what confidence can there be in a rapid push towards a fundamentally different approach?

For its part, AQA has, in recent weeks, apologised for mistakes in GCSE science and GCSE maths papers (while insisting that, respectively, “we have processes in place to make sure that no learners are disadvantaged” and that it will explain before results day “what we’ve done to make sure no learners are disadvantaged”).

Are problems like this a sign that exam boards are finding it increasingly difficult to design and deliver exams smoothly, even before they try a new approach?

Hughes’ response? “No, I would argue the reverse is the case.”

‘No significant increase’ in exam paper errors

He adds: “The truth is that the quality of what we do is going steadily up by the year.”

Any mistakes, he suggests, are exceptions to the rule and “you’re not seeing any significant increase in errors or any of those kinds of issues over time”.

Last month, the NAHT school leaders’ union’s head of policy described exam errors as “disruptive” for schools, saying they were likely to drive up staff workload significantly. And this was before Tes revealed that key stage 2 Sats results - run by the Pearson exam board - would be delayed by more than a week.

Regarding problems with exam papers in the secondary sector, however, Hughes argues that there may even be a positive side.

“What is, I think, correct and proper is that there is a much higher sensitivity and a much greater openness about it and [exams watchdog Ofqual] is very clear that we are obliged to tell the regulator,” he said.

“We are immediately very public about anything that’s happened, because we want to be clear with people that they’re not going to get disadvantaged by it.”

Hughes adds that “accumulated experience over decades” means that “if for some reason a paper has been difficult to decipher because there’s been a mistake...we know how to deal with those situations”.

Ofqual has upheld a number of complaints against exam boards in recent years, issuing some large fines, including to Pearson last December, when the board received the biggest-ever collective fine issued by the watchdog.

Hughes says it is unclear whether the overall number and size of fines are going up or down.

At AQA, he says, “quite a number of errors are relatively minor, [they] might be just literally typos or something like that, [and] the bulk of errors we spot before the exams”.

He adds: “I am very, very confident that we are in a state of continual and progressive improvement on that front [but] if we do make that kind of mistake, Ofqual will step in and we’ll get punished.”

‘Almost no downside’ to digital exams

Despite all of the reported mistakes in pen-and-paper exams lately, Hughes remains bullish about the prospect of digital exams. While he accepts the practical reality that these would have to be phased in over several years, he would like to see more signs that on-screen assessment is a priority for leaders in schools and beyond.

One concern about digital exams is that not all school buildings will be suitable for them without extra money being found for equipment and infrastructure - prompting fears about potential inequalities if, for example, there is a need for dedicated exam centres and no clear strategy on who would pay for and provide these.

An AQA spokesperson tells Tes it has not considered the potential for shared premises, “largely because it’s an approach that would lie beyond our remit”.

Hughes sees “almost no downside on the overwhelming majority of fronts” when it comes to digital exams.

He adds that AQA has carried out experiments with “typed-in, short-answer text boxes” in which “students tend to give better answers” - and says this is “beneficial for less-high-performing students”.

Students ‘baffled’

In education, says Hughes, “we currently assess using pretty much every media available to humankind”, including writing, sound recordings and visual imagery.

Given this, it “just doesn’t stack up” to avoid on-screen assessment. Meanwhile, “students are slightly baffled” by the lack of digital exams, he adds.

Hughes says that “being able to use keyboards is a really important skill that we should be supporting students with from a much earlier age”. If young people struggle with such skills, “then we have failed because we haven’t set the system up to ensure that they [can] use a keyboard, use screens properly”.

Echoing comments from another exam board boss, WJEC Eduqas CEO Ian Morgan, Hughes dismisses the idea of the UK having a “digitally native generation” as complacent: while young people are adept at using social media sites such as TikTok, the “same generation is actually really seriously not competent, for example, in very extensively used pieces of digital technology, such as spreadsheeting and presentation technologies”.

He would have liked the Francis review of curriculum and assessment to have “gone further” in “weaving a great degree of digital fluency into the curriculum”, which he sees as another big equalities issue: “There is a strong argument for saying that we have a high responsibility to teach people to bring everybody up to the same level of digital fluency.”

Hughes also rails against what he sees as a “daft conflation” of “all the problems that arise from social media use and excessive screen use” and a subsequent reluctance to support the use of digital exams.

He describes assessment under timed, controlled conditions as “fairer”, raising concerns that misuse of AI could undermine the integrity of coursework.

Hughes also points to the “extra layer of burden” placed on teachers as a result of AI misuse in coursework, although he believes coursework is still “very much the best way” to assess some activity in courses such as history and English literature.

When Tes went back to AQA to ask for the evidence that exam-style assessment is fairer, the board referred to its recent response to the national consultation on on-screen assessment, which said that “digital exams can help support students with special educational needs and/or disabilities”.

The use of AI in marking

Hughes also sees potential upsides to AI in its use for marking exams: AQA is running a pilot programme to assess the capability of AI as a “marking supervisor” this year, he tells Tes.

The board aims to use the technology to “mark the markers” rather than use it for marking directly; AQA’s “assumption” is that AI will be “very, very good at supervising humans”.

AQA’s approach in the pilot is one where “the AI supervises on our behalf what different examiners are doing, looking for examiners going out of line or making anomalous judgements”.

Hughes adds: “Obviously, machines are really, really good at that, and they can look at everything, whereas humans are slightly more variable in that.”

He emphasises that AQA has already carried out “extensive” testing over the past two years and that Ofqual is aware of the project.

Early indications from the pilot show “significant” reductions in variability, according to Hughes. Tes asked AQA for sight of the evidence base behind this, but the exam board was unable to provide it.

Hughes says: “It’s going to take several summer series to do this, but once we get there, there’s every reason for thinking that the measure of success will be that we can reduce the initial variability - that means we can speed up marking and be much more confident, much earlier on, that we’ve got consistency in marking.”