American software that grades essays is ‘damn reliable,’ but could do better with nuances, reports Kevin Dowling. Struggling carbon-based life-forms - students and their teachers - are about to be confronted with yet another giant step for the silicon chip; the Intelligent Essay Assessor, a computer for marking essays.
Fed the entire texts of required books, the American device can be calibrated with a range of sample compositions already graded from “A” to “unspeakable” by human beings.
The software identifies concepts contained in the books, compares (in around 20 seconds) how a student has expressed them against its bank of marked work, and awards grades accordingly.
The program, with its promise of relief from a much-hated chore, has won a cautious welcome in the United States, reports The Washington Post.
Schools, colleges and universities across the country are reportedly queueing up to welcome the virtual marker into their classrooms early next year.
The Educational Testing Service, which administers most of the US’s standardised admission tests for colleges and graduate schools, will use a similar program, the E-Rater, to grade essays written for major exams.
Studies have found the computer and the teacher agreed on an essay’s mark as often as teachers agreed with each other. However, while machines can dramatically cut the cost of academic labour required to mark exams, “they don’t do nuances”.
Peter Foltz of New Mexico State University, who helped develop the software, said his goal was not to replace teachers. “My goal is to have students do more writing.”
He said the software had uses beyond marking work: it could provide students with feedback to help improve essays over several drafts. When New Mexico undergraduate Hugo Rousselin submitted his first essay in psycholinguistics, the computer gave him a “B”. “You need to define the term ‘superiority effect’ and what it does,” the program told him. So Rousselin did - and instantly raised his computed grade to an “A.”
“I think it’s creepy,” Rousselin said. “On the other hand, in my other psychology class, I wrote a five-page paper and I didn’t get it back until two weeks later.”
Florida State University plans to use the software to score 200 papers on one of its introductory courses. “I believe human beings are more accurate, ” said Professor Myke Gluck, “but the damn machine is as reliable as hell.”
Ron Lamb, a sixth-grade teacher at Platt Middle School in Boulder, Colorado, added:“The computer can’t look at an essay and say, ‘Does it make sense?’ It just compares it to the original text.”
Comment, page 16
