Fear of a social media backlash is preventing exam boards from setting harder questions and leading to papers that are too easy and predictable, a prominent assessment expert has claimed.

Robert Coe, from Durham University, believes that there is currently too much emphasis on “accessibility” in exams and not enough on “high expectations” and “challenge”.

“Too much low-level thinking, what I am calling ‘predictable regurgitation’, is rewarded in exams, and that’s such a dysfunctional and distorting thing to be contending with,” the academic, who advises exams regulator Ofqual on standards, said.

Professor Coe argues that a key reason for today’s “overly predictable exams” is “the sense of public backlash that we’ll have if we ever write an interesting exam paper…you ain’t half going to get slated on social media”.

“Exam boards will be very sensitive to this and Ofqual are very sensitive,” he added. “The students will be complaining [online] straight away - probably not before the exam’s finished but not many minutes after.”

‘Unexpected questions’

Professor Coe gave the example of a maths Higher paper set in Scotland last year (see box below, “Case study 1: crocodile tears”). “They did put some innovative and unexpected questions in, and everyone thought they had made a terrible mistake,” he said.

“It was much, much too hard - people were complaining. There was a whole campaign, with petitions online. It ended up with the chief executive of SQA [Scottish Qualifications Authority] having to apologise to the education committee for having written an exam that was too hard. Included in that apology was a statement about how they’d failed to make it accessible for the majority.”

Some exams are more formulaic in their style and approach

Professor Coe said that today’s exams often rewarded “recall with limited thinking required”, “question spotting”, and “playing safe”. Speaking to the Headmasters’ and Headmistresses’ Conference this month, he said: “We have had a period where accessibility has dominated.

“I think we ought to be devoting a bit of effort to thinking about how we can, yes, keep hold of accessibility but also make sure we are not losing high expectations and we are providing challenge as well.”

The academic - who is director of Durham’s Centre for Evaluation and Monitoring - also argued that it was “an awful lot easier” for exam boards to write “predictable” papers.

“Much, much, harder is to come up with innovative questions and, quite often, when you do come up with innovative questions they don’t work very well,” he said.

“Students get confused. They don’t interpret it the way you thought it was meant to be and then you’re in trouble. So examiners play safe.”

But the Joint Council for Qualifications (JCQ), which represents the UK’s seven largest exam boards, has insisted that there is no fear of innovation.

Michael Turner, JCQ director general, said: “Teachers can be confident that a huge amount of research and expertise go into creating examination papers so they can properly assess a student’s skill and knowledge - at both ends of the grade spectrum.

“As media stories and Twitter show each year, exam boards are not afraid of being innovative in setting questions. And with the new set of reforms coming, we will see further stretch and challenge in the system.”

Malcolm Trobe, interim general secretary of the Association of School and College Leaders, recognised that “some exams are much more formulaic in their style and approach”. But he warned about the dangers of creating an exam that is “completely alien” to the student.

“There is an element of predictability on the format and style of questions in our assessments but there does need to be some of that,” he said. “You don’t want to completely throw a student because something is unfamiliar.”

An Ofqual spokesperson said: “Our regulatory requirements place a limit on the marks available for straightforward recall of information and place greater emphasis on students demonstrating their understanding and application of knowledge.

“This, in turn, introduces a greater range of opportunities for interesting and challenging questions to be set on both familiar and unfamiliar contexts and to reward those higher order skills Professor Coe is looking for.”

@Eleanor_Busby

Case study 1: crocodile tears

The Scottish Qualifications Authority (SQA) provoked a social media outcry in May 2015 with a Higher maths paper question about a crocodile stalking a zebra, which some candidates said had reduced them to tears.

Almost 12,000 students, teachers and parents signed a petition calling for the SQA to explain why it had set such an “impossible” Higher maths exam.

The question

A crocodile is stalking prey located 20 metres further upstream on the opposite bank of a river. Crocodiles travel at different speeds on land and water. The time taken for the crocodile to reach its prey can be minimised if it swims to a particular point, P, x metres upstream on the other side of the river as shown

The time taken, T, measured in tenths of a second, is given by:

T(x)=5 √36 + x2 + 4(20-x)

(i) Calculate the time if the crocodile does not travel on land.

(ii) Calculate the time taken if the crocodile swims the shortest distance possible.

The social media backlash…

“That crocodile question will actual haunt my dreams”

“That maths exam was absolutely disgusting. What even was that crocodile question and why did I get an answer of negative time?”

“In all seriousness though, surely a zebra could outrun a crocodile. I still don’t even know what the question was asking”

“My mind’s still broken from lookin at that question…”

Case study 2: bitter sweets

Edexcel sparked an internet storm last June when its now-infamous GCSE maths question about “Hannah’s sweets” left teenagers perplexed and outraged.

Rod Bristow, UK president of Pearson, Edexcel’s parent company, responded, saying: “The power of social media to give young people a voice is heartening but the one concerning part for me is that it gave many the idea that there was something wrong with the question.

“In fact, it was a good question - a question aimed at students aspiring to an A/A* grade - and tested not just arithmetic computation but mathematical literacy.”

The question

There are n sweets in a bag: 6 are orange, the rest are yellow. Hannah takes at random a sweet from the bag. She eats the sweet. Hannah then takes at random another sweet from the bag. She eats the sweet. The probability that Hannah eats two orange sweets is ⅓.

Show that n² - n - 90 = 0

The social media backlash…

“If Hannah’s sweets make me lose out on a B, I’m gonna find her and stuff them down her throat...”

“The entire country is crying about that Hannah’s sweets question”

“Hannah’s sweets had to be the worst question on any exam I have been given in my whole life, thanks Edexcel”