The creation of an exam paper: from start to finish
In a warehouse on an industrial estate in Rotherham, South Yorkshire, conveyor belts are rolling. This is a processing plant for Pearson, the exam board that runs the Edexcel exams. If your students completed an exam produced by Edexcel a few months back, this is where it will have ended up.
Although more than 5 million GCSE exam papers are taken each year, the process of how the papers come into being, what happens to them when they are sent back to the board and why pupils gain the results that they do remain a mystery to many in the teaching profession.
So here is where the life cycle of a GCSE exam paper begins.
Conception: the 'parents' of an exam paper
An exam paper begins its journey of life approximately two years before it is due to be taken. The gestation period can begin, perhaps surprisingly, with just one parent mulling over the course content, rather than the orgy of debate between multiple experts one might expect.
This single parent is called the “principal examiner”, usually a teacher or recently retired teacher who is an expert in their subject and who has extensive marking and examining experience.
In some cases, the principal examiner may write the whole first draft of the question paper and corresponding mark scheme alone (though multiple experts each submitting questions into a pooled first draft is also common).
But the first draft is not wholly down to the parents; there is a basic genetic framework for the exam that has to be adhered to. The board can’t change the duration of the exam, for example, which is fixed when boards first receive accreditation from the exams watchdog, Ofqual, to run the qualification.
But it can make small changes – adding in more two-mark questions and fewer three-mark questions, for example. It can also decide the topics to be covered by the exam.
Unsurprisingly, the information that exam boards are prepared to give out about how topics are chosen is patchy.
All that Sharon Hague, senior vice-president at Pearson Qualification Services, will say is that core topics will come up on a regular basis and that principal examiners “will look at what has happened in previous years”.
With the topics decided, the lone writer or group of writers then has to pen some questions. This, like naming a baby, is something of an art form.
Questions have to be set out in a logical order and in simple language. A guidance document from the WJEC exam board recommends rephrasing a question from, “What kind of cleaning agent will remove the hard-water stains left by a dripping tap on a washbasin?” to, “A dripping tap leaves hard-water stains on a washbasin. What kind of cleaning agent will remove them?” The latter “follows a logical narrative flow”, according to the board.
Labour and 'the birth'
And narrative is important. These exams are not meant to test knowledge in isolation, say the boards. Alex Scharaschkin, director of research and compliance at AQA, says that exams require students to show they can “apply knowledge to situations over and above being able to remember a formula and write it down”.
The way the boards test this ability is by writing the questions as scenarios. This can cause problems if done badly – for instance, by referring to things with which students are unfamiliar.
A good example was an exam paper that included a case study about the newsagent WHSmith, which was difficult for pupils in Northern Ireland where the chain did not have any branches.
Taking all this into account, a first draft is produced – and the seed has been sown. Over 18 months that seed develops into the final paper through multiple stages of approval: a single paper can go through as many as 10 rounds of redrafting and discussion.
The Braxton Hicks contractions (the practice labour contractions) in this scenario involve the “scrutineer”. This subject expert is trained to sit an exam and respond as a student would, checking the questions are understandable and the paper can be completed in the time allowed. This removes the need for the exam paper to be tested on real students.
If a paper has made it this far, we can view it as being head down and ready to go. It is signed off by the principal examiner and a “chief examiner”, whose job is to oversee a whole qualification rather than just one paper. This should happen by the autumn before the exams are taken. The Department for Education and Ofqual will not see the exam paper until the day that pupils take the test.
This is an edited article from the 19 August edition of TES. To read about the next stages in the life of an exam paper, see this week’s TES magazine. Subscribers can read the full article here. To subscribe, click here. To download the digital edition, Android users can click here and iOS users can click here. You can also download the TES Reader app for Android and iOs. TES magazine is available at all good newsagents.