Sats in hands of below-par markers

26th June 2009 at 01:00
Leaked emails reveal exam board's strategy to `train out' errors as it tries to deliver on time

Substandard national test markers are being allowed to continue working, despite making errors on dummy papers that would usually lead to them being barred.

The revelation comes in a series of internal emails from Edexcel, the national tests contractor, leaked to The TES. The emails offer a graphic illustration of the inexperience of some of the markers being used this summer and raise further questions over whether test papers will be returned on time.

At the same time, the authorities are working hard to avoid a repeat of last year's Sats fiasco when 1.2 million pupils had results delayed.

The emails will cause particular disquiet as Ofqual raised inaccuracy in national test marking as a "significant cause for concern" in March.

A study by the exams regulator suggested that only 55 per cent of pupils taking the key stage 3 English writing test were correctly graded in 2007.

Earlier this month it emerged the Qualifications and Curriculum Authority had disrupted the procedure for checking marker accuracy by providing dummy papers with flawed questions that were artificially penalising "good" markers.

As a result it was decided to let some markers continue even though their margins of error were beyond normally acceptable limits.

But the leaked emails relating to English reading papers reveal that the relaxation of the rules goes beyond errors caused by the "problem questions" and extends to markers' own mistakes.

Janet Bickford, Edexcel subject adviser manager, writes: "The team leader should use their professional judgment to decide whether these markers are making repeat errors on any questions that could be trained out.

"The repeat errors could be on the already identified problem questions from training or on other questions specific to that marker."

Another email reveals the inexperience of the former KS3 markers Edexcel has been forced to recruit to mark KS2. It offers ex-KS3 markers "pointers" on how to mark KS2 but was sent out two weeks after marking had begun.

On writing, it says: "Eleven-year-olds tend to express themselves less clearly than 14-year-olds. Even the able level 5 pupils, because they are being ambitious, may occasionally tie themselves in knots when attempting complex sentence structures.

"Remember to look for what is there rather than what is missing."

In February Edexcel said it would only use KS3 markers if it had problems getting those with KS2 experience. But the board is desperate to avoid a repeat of last year's delays under the previous contractor, ETS.

An email from Ms Bickford responds to markers' concerns about "tight deadlines" and reveals Edexcel considers they will be achievable only in the "majority" of cases.

Another leaked message reveals that markers have been making errors on a dummy script because they have been "undermarking" pupils who express themselves "succinctly" and with "brevity".

A spokesman for the QCA said: "We are currently on schedule to meet our targets for July 7, 2009.

"As in previous years, quality assurance measures and support from supervisors help to identify any problems markers may have in applying the mark scheme correctly.

"Markers can then receive targeted support to address these issues and further improve the quality of their marking."

Marked down

How markers went wrong on a "longer task" dummy English writing script, according to Edexcel:

"The pupil is consistently using complex structures with varied connectives; a range of verb forms and expansion . The pupil expresses ideas quite succinctly and makes the occasional error; this could be why markers are undermarking."

"The pupil clearly supports the organisation of the report with well- linked paragraphs . Again, the development is done quite succinctly and this could be leading to undermarking."

"The brevity and the fact that there are few descriptions of personal experiences means that this script is not as lively as some others that receive high marks; this could be leading markers to award low marks."

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number


The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now