Without a pilot, exams will crash and burn

8th August 2014 at 01:00
Trials of new qualifications are essential, as the disaster in 2000 proved. So why are we making the same mistake again?

That the school exam reforms planned to take effect from September 2015 will suffer major problems, and that an urgent public debate is needed about them, seems obvious to us. In failing to adopt a consensual approach to changes such as the radically remodelled GCSE, the government has undertaken no real discussion with universities, schools and exam boards.

The case for the revolutionary alterations demanded by ministers is weak. The claim is made, with little clear evidence, that rapid and fundamental change is needed to remedy public concern over exams and international failure as measured by the Programme for International Student Assessment (Pisa).

But surely the immediate issue is that, without trials of the new qualifications, England risks a system that will malfunction far more than at present.

First, it is not clear that the objective of "raising the bar" can be achieved. The government believes that making tests harder leads to increased performance. Yet the previous attempt to do this in Curriculum 2000, notably in maths, was a disaster.

The situation back then is perhaps best summarised by Margaret Brown, professor of maths education at King's College London. She writes: "A group of mathematics assessment experts met to decide the common core for mathematics.the core content they proposed included a good deal of content which would previously have been covered in Year 13.The outcome was that the first [2001] sets of AS results in mathematics were catastrophic.

"In other subjects, moderation.was conducted by comparing proposed grades with students' mean GCSE results. Mathematics was the only subject where this process was impossible as the marks were so low."

Statistics on exam entry show the consequences for maths of Curriculum 2000. The number of A-level entries dropped markedly. Remedial measures had to be brought in to revise the AS-level syllabuses, and it was half a decade before entry levels recovered to the relatively healthy situation we now find. There is a clear danger of this pattern repeating itself from 2015, and this time across both GCSE and A-level - a potential catastrophe resulting from the unprecedented decision to revise both sets of exams simultaneously.

The problems in 2001 led to an explicit warning from what was then known as the Department for Education and Skills. "We recognise absolutely that there are major lessons to be learned for the future about the way in which we implement major reforms," it said.

"Detailed planning and extensive trialling is essential so that we can be confident that all systems are in place and teachers and examiners are fully trained in new requirements before they are introduced."

Despite this warning based on bitter experience, the new GCSEs and A-levels to be taught from 2015 will apparently go ahead without trials or piloting. Exams body Ofqual has made no official statement explaining why, but privately it argues that piloting cannot be accurate: the trials do not reflect real exam conditions. This is not a valid argument. In aviation, no test pilot flies with a real cargo or fuel load. But no plane would be put into service without test flights.

The evidence is clear that "raising the bar" failed with AS- and A-level maths during Curriculum 2000. However, the conclusion drawn at the time that reforms must be piloted has been rejected by Ofqual. The regulator will allow exam boards to pre-test exam questions, but synoptic testing of all exams is not on the agenda. As the danger of failure with these reforms affects far more students than in 2001, why is Ofqual so reluctant to pilot the changes to obtain at least some evidence?

Flying blind

The issue of piloting comes into sharpest focus when the planned national reference test is considered. Ofqual issued a consultation document for GCSE in April announcing that these "pre-tests" will be used to set grade boundaries for the whole cohort. This document argues that "the test will be piloted in 2016 and run annually from 2017".

If the government accepts that pilots are possible for the reference test, trials of the new exam systems would be just as invaluable for letting the public know in advance of any major problems. Indeed, the planned reference test might be more difficult to run than pilots of the new qualifications, since schools are likely to try to avoid doing it. Piloting the GCSEs and A-levels would carry no risks for the school and would be a vital indicator of potential problems - or successes.

Many other issues beyond classroom level also remain unresolved, notably the break-up of the pan-UK examination system. Wales and Northern Ireland will not go down the route England is taking, which will have knock-on effects for the international market for exams. The grading changes and the confusion implicit in the new systems are going to be difficult to handle when users find different systems operating across the UK.

Disillusionment with the ongoing mess is reflected in the increasing abandonment of GCSEs and A-levels, especially by independent schools. Increasingly, they are choosing to take the IGCSE and the International Baccalaureate instead. It would seem that we are developing a two-track qualification system, reflecting a two-track educational profile in exams. How can the system be credible in this situation?

What we have now is working but could be improved. Unless raising the bar can be shown to work and the new exams are piloted, improvement may not be on the agenda. It is vital to stop and take stock of where the English exam system is heading.

Richard Pring is a former professor of educational studies at the University of Oxford. He led the Nuffield Foundation's six-year review of 14-19 education and training in England and Wales. Trevor Fisher taught to A-level for 37 years and writes about educational and historical matters

Log-in as an existing print or digital subscriber

Forgotten your subscriber ID?


To access this content and the full TES archive, subscribe now.

View subscriber offers


Get TES online and delivered to your door – for less than the price of a coffee

Save 33% off the cover price with this great subscription offer. Every copy delivered to your door by first-class post, plus full access to TES online and the TES app for just £1.90 per week.
Subscribers also enjoy a range of fantastic offers and benefits worth over £270:

  • Discounts off TES Institute courses
  • Access over 200,000 articles in the TES online archive
  • Free Tastecard membership worth £79.99
  • Discounts with Zipcar, Buyagift.com, Virgin Wines and other partners
Order your low-cost subscription today