In TES on 2 August, Andreas Schleicher suggests that I may be unfamiliar with advances in item response theory in the past 50 years (“Attacks on Pisa are entirely unjustified”, Comment). I can assure him that this is not the case, but what concerns me is that he must know that nothing in the literature of the past 50 years addresses the central point made in my paper.

My case against the Programme for International Student Assessment (Pisa) is that all these advances simply produce more completely “static” models of the wrong type, while the measurement model I suggest is “dynamic” through and through. This confusion is another by-product of the profound conceptual error at the heart of Pisa; Pisa misconstrues the nature of ability itself.

All measurement is made against a “background” that gives meaning to the properties being measured. For example, physics textbooks often portray Einstein’s general relativity as a re-construal of the background that Newton (pictured, right) took for granted (for example, he assumed absolute simultaneity, Euclidean geometry and so on).

Suppose a Pisa item were to require the examinee to find the square of 11. There is scant, if any, evidence in the literature that when one pronounces 121 the correct answer and all other answers incorrect, one is drawing on a background of mathematical practice. If one reads the psychological and psychometric literatures one has the clear impression that the question in itself is the measuring instrument. This is incorrect: without the evolving mathematical practice within which this question is embedded, the question has an infinity of answers. Without the background mathematical practice, notions like incorrect and correct lose their meaning.

This leads to a dynamic picture of psychological and educational measurement completely beyond the capabilities of Pisa’s software, no matter how advanced. The examinee’s response of 121 represents a contribution to the mathematical consensus that underpins the measuring instrument. An incorrect answer, for example, weakens that consensus. (Needless to say, I am not suggesting that the correct answer to any given problem is established by taking a vote among mathematicians.)

Once again, we arrive at the conclusion that no one at Pisa seems ready to face: there is no clear dividing line between the examinee being measured and the instrument carrying out the measurement. Ability is a joint property of the person and the practice of mathematics, and not simply an intrinsic property of the person. Rather, a deflationary account of ability would be to consider it to be a property of the interaction between examinee and the practice. Ability isn’t something the examinee has; it’s something he or she has for us.

Dr Hugh Morrison, Queen’s University Belfast.