Not everyone leans the same way over Pisa

2nd August 2013, 1:00am

Share

Not everyone leans the same way over Pisa

https://www.tes.com/magazine/archive/not-everyone-leans-same-way-over-pisa-1

Ironically, your feature “Is Pisa fundamentally flawed?” (26 July) was itself so fundamentally flawed that it simultaneously misrepresented expert critics and misled non-specialist readers.

First, the Rasch model does not require questions to “be equally difficult in all participating countries”. If it did, it wouldn’t be a model and all countries would get the same score. Differential item functioning compares the difficulty of a question across groups only after they have been matched for ability. This is a problem not only for the Programme for International Student Assessment (Pisa) but for all comparative studies.

Second, the Rasch model describes a probabilistic relationship so it does not imply that students of the same ability will “all give a correct answer or would all give an incorrect one” to the same question. As Harvey Goldstein says, there is no conceptual error here - even if there are serious technical shortcomings.

Third, Pisa’s plausible values are dismissed as if they were randomly plucked from thin air, even though the Organisation for Economic Cooperation and Development (OECD) makes it clear that they are numbers randomly drawn from a known probability distribution. This approach is entirely rational - even if in this case its validity is questionable.

I am one of Pisa’s many critics but this article did little to advance our case or clarify our concerns.

George Bethell, Director, Anglia Assessment, Stowmarket, Suffolk.

Want to keep reading for free?

Register with Tes and you can read two free articles every month plus you'll have access to our range of award-winning newsletters.

Keep reading for just £1 per month

You've reached your limit of free articles this month. Subscribe for £1 per month for three months and get:

  • Unlimited access to all Tes magazine content
  • Exclusive subscriber-only stories
  • Award-winning email newsletters
Recent
Most read
Most shared