Essays to be marked by ‘robots’

As computers are used in English tests, experts predict GCSEs and A- levels will be next
25th September 2009, 1:00am

Share

Essays to be marked by ‘robots’

https://www.tes.com/magazine/archive/essays-be-marked-robots
Thumbnail

The owner of one of England’s big three exam boards is to introduce artificial intelligence-based, automated marking of exam essays in the UK next month.

The decision by Pearson, parent company of Edexcel, to use computers to “read” and assess essays for English tests is fuelling speculation among assessment experts that GCSEs and A-levels will be next.

The head of research at another of the big three boards told The TES it was a question of “when not if” automated marking was introduced extensively.

But last night academics attacked the idea of machines judging the quality of extended writing as “ridiculous nonsense”.

They fear that pupils will start writing to impress computers, removing human creativity from their work.

The TES revealed in 2006 that Edexcel planned to start trials of the automated marking of essays in dummy GCSE-style questions.

The launch of the new Pearson Test of English Academic (PTE Academic) on October 26 will mean the controversial technology is being used for essays in live exams.

Pearson boasts that its “proven automated scoring” will provide a test that accurately measures candidates’ English writing abilities.

Bethan Marshall, senior lecturer in English and education at King’s College London, said: “A computer will never be unreliable. They will always assess in exactly the same way. But you don’t get a person reading it and it is people that we write for. If a computer is marking it then we will end up writing for the computer.

“People won’t be aiming for the kind of quirky, idiosyncratic work that produces the best writing. So what is the point?”

Tim Oates, research director at Cambridge Assessment, which owns the OCR exam board, said: “It’s extremely unlikely that automated systems will not be deployed extensively in educational assessment. The uncertainty is `when’ not `if’.”

The technology being used by Pearson is designed to allow computers to assess pupils’ use of grammar and vocabulary. But some experts say newer, more effective systems are available.

The Pearson approach is based on correlations between human judges and artificial intelligence systems. Machines are “trained” to learn from the scores given to specific texts by humans so that they will be able to achieve the same results on their own.

Mr Oates said: “In simply getting an automarking system to agree with human markers you are ignoring the vital question of exactly what parts of performance are being ranked.

“Other developers are working on more valid approaches, of greater merit and promise. Crucially, these aim to be sensitive to the concepts and language structures actually being used by candidates.”

A Pearson spokesman said that its system produced the accuracy of human markers while eliminating human elements such as tiredness and subjectivity.

“There are many technologies and we feel this is the best for its designed purpose,” he said.

The PTE Academic, being introduced in at least 20 countries, including the UK, is designed to help English-speaking universities assess the English language proficiency of potential students.

An Edexcel spokesperson said that the board was not planning to use automated marking in mainstream exams for anything other than multiple choice questions.

She said the trials that saw the technology used for GCSE-style essays were very small scale and not being pursued “at this time”. She could not say whether the results were positive or negative.

Want to keep reading for free?

Register with Tes and you can read two free articles every month plus you'll have access to our range of award-winning newsletters.

Keep reading for just £1 per month

You've reached your limit of free articles this month. Subscribe for £1 per month for three months and get:

  • Unlimited access to all Tes magazine content
  • Exclusive subscriber-only stories
  • Award-winning email newsletters
Recent
Most read
Most shared