In September, I wrote a blog entitled “The progress loophole of despair”. It dealt with how progress is calculated for pre-key stage pupils (ie, those working below the level of KS2 tests) and how some pupils are excluded from progress measures.

It considered scenarios in which schools’ headline measures would be improved if certain pupils failed KS2 tests and whether this might result in “tactical” assessment. If schools actually benefit from assigning inaccurate assessments and from pupils scoring less, then this flies in the face of accountability guidance, which states that “it is important that schools are held to account and given recognition for the progress made by all of their pupils”.

If schools actually benefit from assigning inaccurate assessments and from pupils scoring less, this flies in the face of accountability guidance

First let’s consider those students working below the level of KS2 tests. The vast majority of these pupils were not entered for KS2 tests and instead were assigned a “nominal score” ranging from 70 to 79, according to the particular assessment; in nearly all cases, these resulted in negative progress scores and, in many cases, deficits were huge and had a considerable effect on overall scores.

For example, a pupil with an average point score of 10 at KS1, assessed as “early development of the expected standard” (PKE) in writing at KS2, ended up with a progress score of -16.6. I have seen scores as low as -21.75. Such scores can wipe out the progress made by other pupils.

Pupils not meeting expected standards

Let’s consider pupils assessed as having not met expected standards. The “has not met” (HNM) judgement, which applies to reading and maths, sits between pre-KS and expected standards, and receives no nominal score. These pupils took the tests and most achieved a scaled score, which counted in the progress measure. However, many assessed as HNM did not score enough on tests to achieve a scaled score; because there is no nominal score linked to the HNM code, they were excluded from progress measures, thus avoiding the deficits associated with pre-KS assessments.

There are, of course, pupils with HNM codes that scored just enough to achieve the minimum score of 80, and this caused a substantial dent in schools’ overall progress scores. In these cases, the school would have benefited from those pupils scoring fewer marks, which would have seen them excluded from measures.

Some schools have seen a sudden drop in progress scores

Something else has now come to light: pre-KS pupils who sat the test but didn’t score enough marks to achieve a scaled score. It transpires that these pupils were not included in unvalidated data, but have now been assigned nominal scores and included in performance tables. So some schools have seen a sudden drop in progress scores, possibly taking them into negative territory or pushing them below the significance threshold.

The progress loophole goes deeper than I previously thought and one has to worry about the consequences.

James Pembroke founded Sig+, an independent school data consultancy, after 10 years working with the Learning and Skills Council and local authorities (www.sigplus.co.uk)