Daisy Christodoulou, head of assessment at Ark Schools, explains the process:
“At Ark Schools, we’ve done a number of trials of comparative judgement, and have worked with Dr Chris Wheadon, of No More Marking – a company that provides schools with a free online CJ engine.
“In practice, the process works as follows: you take the set of scripts that have been written by your pupils and scan them into a digital format. Most photocopiers can scan a set of paper scripts and turn them into a series of PDF files.
“You then upload these PDFs to the CJ engine – in our case, the No More Marking website. You then put in the email addresses of your judges. They will then be emailed a link; when they click on it, they will see a pair of essays on their screen.
“They have to decide which essay is the better piece of writing by clicking on it. Once they’ve made that decision, it takes them to the next pair of essays, where they make another judgement. And so on. The pairs of essays are selected at random. Each judge has to do a number of judgements, and the exact number depends on how many essays you want to judge, and how many judges you have.
“Once all the judges have finished, the algorithm that sits behind the system crunches all the decisions, and gives each essay a score.
“What kind of score can be awarded? To a certain extent, that depends on how you set up the task. If you want to award pupils a GCSE grade, or a primary grade like EXS or GDS, then you can do so in a couple of ways.
“One way is to use exemplar scripts that have been pre-judged as being at a certain standard. If you scan these scripts in as well as your pupils’ scripts, then you can use their results to help award grades to your pupils.
“Another way of awarding grades is to participate in a larger task. You still scan in and upload your scripts, but at the same time, lots of teachers from other schools do the same. You still make the same number of judgements as before, but this time, you are not just judging your own pupils: you are being given scripts from pupils in other schools.
“This way, you are getting results that are based on a national sample of pupils, not just your own.”