Suddenly, we’re all in love with data. We run schools that are, according to taste. “data rich”, “data driven”, or “data empowered”. Well here are some data to think about.
A school - it’s a real one and there are lots like it - turns in A*-C grade GCSE results over four successive years of 60, 58, 67 and 56 per cent.
What’s the story there? On the face of it they started well, took a drop in the second year, pulled up their socks a bit in the next year, but then failed to consolidate their achievement in the fourth year. And that is exactly how many people, including some who ought to know better, will read it.
But you and I know that what we have here is the “decent restaurant effect”. You ask the manager of your favourite lunchtime restaurant how many covers she does on a Friday and she may say “about a hundred”, But she actually means it will be 94 one week, 110 the next and 99 the next. And she won’t worry - because she knows the difference is down to a combination of endless factors that are beyond her control. All she can do is keep everyone focused on the core business of cooking and serving.
It’s the same in schools such as the one I mentioned. Typically, there will be a couple of hundred children in a GCSE cohort - the sort of number within which a few individuals or events make a significant percentage difference. Heads and colleagues know this and, if asked, can easily (because they know their job) talk you through the figures, showing clearly that they can hear what the statistician Donald J Wheeler calls “the voice of the process”.
I’d be interested to know how that school’s local authority responded to the fourth year drop in results. You trust that it would be understanding, ready to listen. In some places, though, they might easily have panicked, conscious of their targets and their accountability to Ofsted, and sent in an inspection team to sort things out.
And just how helpful would that be? Donald J Wheeler deals with just such questions and misunderstandings in the context of industrial production, though this actually makes many of his points clearer in his book, Understanding Variation: the Key to Managing Chaos (SPC Press, 2000) His relevant sentence here - it’s elegant and neat, and could well be put on your school’s letterhead (or at least on the letters going to County Hall) - is: “While it is simple and easy to compare one number with another number, such comparisons are limited and weak”.
