Teachers and test examiners are to be given a much greater say in the marking of Sats from 2009, the independent inquiry into this summer’s “shambolic” arrangements has concluded.
The inquiry’s recommendation comes after it found that customer service provided this year by the American contractor ETS was “wholly unacceptable and lacked professionalism”.
The six-month inquiry by the former chief schools inspector Lord Sutherland completely exonerated the markers involved this summer. They “did a professional job and public service to persevere with marking in the face of numerous challenges”. Markers and schools had been let down, the report said.
The inquiry demanded that whoever succeeds ETS - likely to be Edexcel, the previous marking firm - should set up a panel of markers and school leaders that it must consult on marking changes.
The company failed to tailor its systems to schools’ and markers’ needs, Sutherland found. Its unfamiliarity with UK education was highlighted by one senior marker, who said that when a colleague had told ETS she would struggle to attend a meeting because of an Ofsted visit, the company needed to be told what Ofsted was. A much criticised helpline provided at premium rates was also described as “poor” and “a very serious failure”.
The number of schools challenging marking this year has soared, the Government has revealed. Around 218,000 appeals against marks were received, a more than fourfold increase on the 50,000 last year. Some 18,000 are still being processed. But this was not the fault of the markers.
The news of the appeals come as Sutherland said ETS should bear the principal responsibility for this summer’s events, which delayed results for 1.2 million pupils and left markers and schools frustrated by disorganised training, multiple computer glitches, scripts going missing and a failing helpline.
The Qualifications and Curriculum Authority, and its testing division, the National Assessment Agency, were also blamed for a culture of “It’ll be all right on the night” complacency which meant it failed to realise until too late that the results would be delayed.
The QCA was also censured for failing to check ETS credentials properly before awarding it the contract. The American firm was the cheapest bidder.
This week, the QCA responded by disbanding the National Assessment Agency, and suspending Ken Boston, its chief executive, and David Gee, the agency’s managing director. Dr Boston had already tendered his resignation.
ETS had its Pounds 156m five-year test marking contract terminated in August. Edexcel was announced as preferred bidder to replace it for the 2009 key stage 2 tests.
Ministers are not saying what will happen after that. Ed Balls, the Schools Secretary, told MPs that alalthough national key stage 2 tests were essential, the system was not “set in stone”. Key stage 3 Sats were scrapped in October.
Teachers’ leaders welcomed the report but said a more fundamental review of testing was required. Mary Bousted, general secretary of the Association of Teachers and Lecturers, said the Government should give serious consideration to the rationale for these tests and their impact on the curriculum.
Mr Balls apologised for the “shambles”, and promised to accept the report’s recommendations in full. But Opposition politicians said he should take more responsibility.
Dr Boston told the inquiry that Government officials were involved at all stages, from the appointment of ETS to delivering the tests and were not operating at arm’s length from the QCA. The report offered no criticism of ministers.
ETS said it had been prevented by the QCA from submitting evidence to the inquiry until after its deadline, and that the QCA bore “significant blame” for problems.
THE STEPS THAT LED TO THE MARKING CATASTROPHE
The Sutherland report describes catastrophic disorganisation. Among its findings are:
- Less than a third of markers were satisfied with online training developed by ETS in a trial in autumn 2007. It was eventually abandoned.
- When face-to-face training took place, sessions included “little or no” information on how to use new online systems for inputting data and to check markers’ capabilities.
- Some 10 per cent of schools failed to complete a new online registration system, designed to record the number of pupils who took the tests. This meant that the system was not recording pupils as present, even when markers had complete scripts for them. Markers found they could not enter these scripts’ results. ETS then corrected this by changing the system to record all pupils present.
- Because the ETS system thought all pupils were present, it expected markers to have scripts for them, even if the pupils were absent.
- Markers reported finding scripts from different subjects in same package.
- Markers complained of the ETS system for entering marks online. One used 20 mouse clicks to enter a pupil as scoring 20 out of 20.
- On receipt of scripts from markers, ETS sent them back to schools without checking them.
- A system designed to add up pupils’ marks and convert them to levels had faults, leaving some schools to do the work themselves.
- In July, with results due, ETS overrode its computer systems to record all pupils for whom it had yet to receive a mark - sometimes because the papers were missing - as simply “absent”. Schools were furious.
- The regulator Ofqual, on a visit to ETS warehouse in West Yorkshire, “discovered” 453 parcels containing 16,000 unmarked scripts. Another 4,000 were unmarked at ETS offices in Watford.
- ETS was to provide information on marking to the NAA. But in July, an official questioned its figures. ETS data said 770,000 to 990,000 scripts were sent to markers, despite there being only 600,000 pupils; that 27,000 markers had had a collection even though there were only 10,000.
