Scrutiny is welcome;Opinion

16th July 1999, 1:00am

Share

Scrutiny is welcome;Opinion

https://www.tes.com/magazine/archive/scrutiny-welcomeopinion
Equipercentile equating may not be your forte, says Nicholas tate, but curriculum testing remains open to all.

Pushing a trolley around the supermarket on a Saturday morning at the end of May, my eye was caught by the newspaper headline “English test pass marks ‘cut secretly’”. The quality of my consumer choices went into free fall and within hours I had been sucked into a media circus that continued throughout the bank holiday weekend. To halt damaging rumours, Education Secretary David Blunkett within days had announced an independent scrutiny of the allegations.

This was not the first time national tests had been the victim of misreporting and headline-seeking. Nor was it the first time I had despaired of explaining to people who wished to believe otherwise that tests vary slightly in difficulty from year to year and that in order to maintain a consistent standard, pass marks will also need to vary.

The 13,000 teacher-markers who mark the tests know this; but the rest of the world is easily seduced by the naive cynicism of those who take it for granted that politicians and public servants are always up to no good.

The independent scrutiny team has now reported on the English and maths tests for 11-year-olds. Its report vindicates both Government and the Qualifications and Curriculum Authority. There was no political interference. There was no “fiddling” with the difficulty of the tests to improve the results.

Marks to achieve each national curriculum level - “pass marks” - are set with great care, using data from pre-tests, comparisons with the previous year’s tests, teachers’ judgments, and an analysis of a large sample of marking from the most experienced markers.

There is also no secrecy. Schools are told exactly what procedures the QCA uses to set the levels and the final level-setting meeting is attended by observers from professional associations.

On the specific point of changes to the “pass mark” in this year’s English tests, the scrutiny concluded that changes of this kind are a common occurrence and made to maintain a consistent standard, not to lower it.

The report concludes that “our system of national tests is well in advance of that of many, if not all, of our international counterparts”.

None of this means that those running the national tests should be complacent. Teachers and schools are always being urged to do better. The same must apply to public bodies. The scrutiny report therefore makes a number of recommendations.

The first is for a thorough review of the purposes and procedures of the tests once the 2002 tests, and their associated targets, are behind us. Given the levels of investment in the national tests, such a review will be essential for the sake of accountability. It will be an opportunity to ask fundamental questions about the testing system and its role in raising standards of attainment, and to compare our experience of national testing with that elsewhere.

A second major recommendation is for a periodic scrutiny of test development and assessment arrangements. The QCA is already setting up an independent division to audit its work as the national test agency, using the same principles it applies to its audit of awarding bodies. There will also continue to be the need for outside evaluations of the kind we have always commissioned and whose reports we have always made public even when critical.

A third recommendation is to strengthen arrangements for involving teachers in reviewing test questions. Although we have one of the most advanced systems in the world for doing this already, with hundreds of teachers giving advice on questions, there is room for improvement.

We will be looking carefully at ways of strengthening teacher representation on our test review groups and in the vetting arrangements used by the test development agencies, while keeping a close eye on security.

Finally, the report urges us to do even more to explain and publicise our arrangements. This we will do, while being aware that equipercentile equating (a statistical technique allowing scores in equivalent tests in two consecutive years to be linked) is not everyone’s cup of tea.

One message I am keen we get across is that assessment is not the exact science it is sometimes supposed to be. The national tests deliberately assess the breadth of the curriculum in the subjects tested and avoid over-simplified assessment methods which, especially in English, would have a negative backwash on the curriculum. As a result, in tests that have to be different each year, there is always a margin of error which has to be taken into account.

Too much should not be read into minor changes in performance year on year. More significant are patterns over a period of years. It is here that evidence from the tests of teachers’ success in bringing about substantial improvements at age 11 is overwhelming - evidence corroborated by the Office for Standards in Education in its recent positive primary report.

Will this improvement be sustained in this year’s results? Schools received their own results, on time, a few weeks ago. What we don’t yet have is the full national picture. For this we must wait until autumn.

Dr Nicholas Tate is the chief executive of the Qualifications and Curriculum Authority

Want to keep reading for free?

Register with Tes and you can read two free articles every month plus you'll have access to our range of award-winning newsletters.

Keep reading for just £1 per month

You've reached your limit of free articles this month. Subscribe for £1 per month for three months and get:

  • Unlimited access to all Tes magazine content
  • Exclusive subscriber-only stories
  • Award-winning email newsletters
Recent
Most read
Most shared