WHERE IT ALL WENT WRONG. The Educational Institute of Scotland

The disaster of this year’s exams is in no way the responsibility of teachers,” the Educational Institute of Scotland says in evidence to Parliament.

In reality, most teachers “with whatever misgivings, and with whatever absence of support” did implement Higher Still. But an early difficulty was in registering candidates early last autumn. “In particular, all candidate information entered before October 1 appears to have been lost from the system from that changeover date.

“Teachers saw further problems emerging in January and February of 2000. As teachers sought to update the list of candidates to be presented at a particular level, the SQA seemed to be unable to amend their own presentation list to reflect the changes submitted by schools.

“The names of pupils who had left school were published repeatedly on SQA presentation lists despite their repeated deletion by schools. Even as late as September 2000 teachers are being asked for material for pupils who left school in September 1999. More alarmingly, throughout the session, SQA was failing to add in pupils whose names had been added by schools.”

The union says matters grew worse. “Teachers were asked for and supplied results of specific units on up to four different occasions, sometimes by fax, e-mail, post and telephone. The SQA seemed repeatedly incapable of processing this basic data ...

“When results were published in August, it became obvious to teachers that in the final assessment of a significant number of Higher candidates in a significant number of schools, despite up to four submissions from the school, certain units were never added for some candidates. This means that in every school, some pupils, indeed sometimes whole classes, were reported as not having obtained the course award. Other candidates who had elements of their external examination missing were reported as having failed.”

In May, the problems were exacerbated when some schools did not receive exam papers or the wrong quantities. “The EIS is aware of at least one school where the SQA had no record of anyone sitting Standard grade English and therefore sent no papers at all, with the result that the chief invigilator had to use the school photocopier to make enough copies for the whole of that school’s fourth year.”

Teachers were clear why some exam results were issued late. “Because the presentation lists were clearly inaccurate, the recruitment of a sufficient number of markers was delayed.

“The effect of this was that when the total number of papers were delivered to SQA, it became necessary to recruit at very short notice more markers or to ask already contracted markers to carry out more marking. This must have contributed to the delay.”

Assessment panels warned SQA management they would need more markers. “Yet the invitation to mark did not arrive until March instead of the usual February. Many of the markers on whom the previous Scottish Examination Board relied I were by that time making alternative arrangements for the summer, making the assumption they would not be marking this year.

“When questions were asked, teachers and markers received assurances that everything would be fine. However, the shortage of markers for several papers quickly came to crisis point. Markers who had already marked were given the option of marking other papers. Many agreed to do so; others refused.

“The timescale for marking was substantially reduced when deadlines were difficult enough. Recognising the amount of work to be done within the new deadline, some markers sent the extra papers back. The administration of this part of the process was, for those involved, an unprecedented disaster.

“It has been reported to the EIS that some examiners arriving for the process of finalisation (the setting of pass marks, etc) were faced with a number of unmarked papers which they had first to mark. The technical aspects of their work was made much more difficult because computer-generated information had omitted many of the marks that were to contribute to the final grade.

“This lack of complete assessment information will have had serious implications for the calculation of concordance statistics - a key quality assurance mechanism for SQA procedures whereby a school’s predictions are statistically matched with candidates’ performance in external examinations.”

The EIS says that teachers believed there was an over-reliance on untested computer systems in processing exam data. Changes in the remits of many SQA staff had a profound impact on day-to-day work in the organisation.

“From a teacher’s perspective, one consequence was an apparent absence of overview within the SQA of the new computer systems by staff who had a deep understanding of the exam systems. The fact that some data processing was contracted out by SQA to a commercial firm raises a number of quality assurance issues.”

It concludes: “Almost every part of the process happened too late to meet what, in retrospect, was an impossible deadline for issuing the new exam results.”

HIGHER STILL WAS IMPLEMENTED AT THE HEART OF THE CHAOS. The Scottish Secondary Teachers’ Association

At the initial stages, this association was told that certain key principles were immutable. These were the unitisation of all courses at all levels, the use of a hugely increased internal assessment element as a dominant feature of all courses at all levels, and the insistence on limiting external examinations to three hours, including intervals between papers. All of these contributed greatly to the chaos which ensued.”

It continues: “We regularly raised problems of late or very late delivery of promised materials, inconsistent quality in materials, late changes in NAB (National Assessment Bank) materials, inconsistent (and thus misleading) advice at training briefing meetings, inconsistencies on advice on assessment and reassessment, changing of advice without national notification.

“On every occasion, whether at the Higher Still liaison group, or meetings with ministers and officials, we were accused of over-reacting and misreporting. We were, in point of fact, relaying the reports of secondary teachers all over Scotland.”

The SSTA believes there was “a political determination” that the programme should go ahead regardless of its readiness.

“Most regrettably of all, however, is that the determination failed to take account of the now obvious inabilities of the SQA to resource the change at the due date.”

The SSTA complains its concerns were repeatedly ignored by ministers, even in the Higher Still liaison group. “This association was still met with bland assurances of timetables being met and materials delivered. Experientially, this was not happening but again the voice of teachers - in this case all of the unions in one united choir - was largely ignored. We were repeatedly accused of pandering to the unwillingness of teachers to embrace change. The reality was that we were reporting the inadequacies and not getting a listening ear.”

TIME PROBLEMS. The National Association of Schoolmasters Union of Women Teachers

The NASUWT submission was very brief, but the key points were that the SQA wildly underestimated the number of markers required and the time allocated for marking was too short, with later exams having to meet the same publication date for results. It asks why computer checks were not run.

For problems, the union highlights the “unnecessary complexities of data entry and recording at SQA, eg 7 at SCE = B 7 at Higher Still = fail 7 at Standard grade = course complete”.

It also says data was sent to the SQA but not acted upon, “eg estimates from schools and presentation levels”.

Other problems it mentions are “failure to pilot the software” and the implementation of Higher Still: the “timetable was politically driven (by both governments) and educational arguments were discounted as obstruction”.

THE REPETITIVE ENTRIES EXERCISE. A school SQA co-ordinator and SSTA member

In my own school, with 270 students in S56, the process of entering students for units courses and recording results is unacceptably time-consuming and repetitive. Well over 100 hours have been involved in this exercise, and athough it is not a high level skill, it still requires the involvement of someone who understands what it is all about.

“I hope to delegate the more mechanistic elements of this to a member of the office staff in future, but this is only possible when there is confidence in the reliability of data management systems.

“Even then, an experienced eye will still be required to monitor the procedure. 270 students equates to approximately 250 individual course entries, 3,750 unit entries and a further 3,750 results. Add deferred results and reinstatements (around 1,000) to give nearly 10,000 actions each year. Unfortunately, this year many of these actions had to be repeated once or twice, or even three times!”