Not everyone leans the same way over Pisa

2nd August 2013 at 01:00

Ironically, your feature "Is Pisa fundamentally flawed?" (26 July) was itself so fundamentally flawed that it simultaneously misrepresented expert critics and misled non-specialist readers.

First, the Rasch model does not require questions to "be equally difficult in all participating countries". If it did, it wouldn't be a model and all countries would get the same score. Differential item functioning compares the difficulty of a question across groups only after they have been matched for ability. This is a problem not only for the Programme for International Student Assessment (Pisa) but for all comparative studies.

Second, the Rasch model describes a probabilistic relationship so it does not imply that students of the same ability will "all give a correct answer or would all give an incorrect one" to the same question. As Harvey Goldstein says, there is no conceptual error here - even if there are serious technical shortcomings.

Third, Pisa's plausible values are dismissed as if they were randomly plucked from thin air, even though the Organisation for Economic Cooperation and Development (OECD) makes it clear that they are numbers randomly drawn from a known probability distribution. This approach is entirely rational - even if in this case its validity is questionable.

I am one of Pisa's many critics but this article did little to advance our case or clarify our concerns.

George Bethell, Director, Anglia Assessment, Stowmarket, Suffolk.

Subscribe to get access to the content on this page.

If you are already a Tes/ Tes Scotland subscriber please log in with your username or email address to get full access to our back issues, CPD library and membership plus page.

Not a subscriber? Find out more about our subscription offers.
Subscribe now
Existing subscriber?
Enter subscription number


The guide by your side – ensuring you are always up to date with the latest in education.

Get Tes magazine online and delivered to your door. Stay up to date with the latest research, teacher innovation and insight, plus classroom tips and techniques with a Tes magazine subscription.
With a Tes magazine subscription you get exclusive access to our CPD library. Including our New Teachers’ special for NQTS, Ed Tech, How to Get a Job, Trip Planner, Ed Biz Special and all Tes back issues.

Subscribe now