The relative ratings for 199697 Standard grade physics puzzled principal teacher James More. Two years of progress in which his department at Glenwood High, Glenrothes, performed above the school average were reversed by a rating of -0.14.
“That was a real challenge for me to understand. It’s not a lot of fluctuation, and we were not doing anything desperately wrong, but we managed to go from being 0.2 above, to a negative. We had to look at something specifically there,” Dr More recalls.
The answer lay in the decision to introduce a dedicated class for boys after previous fears that the department was short-changing girls in what, traditionally, has been a boys’ subject. Staffing had been stable and there had been no great changes in teaching methods.
“When we looked closely at the performance of boys together, they were particularly poor. They pulled the statistics down and certain boys did not perform. There was something about the group dynamics. This year, we’ve gone back to mixed courses,” Dr More says.
In Higher physics, the opposite happened. Over a four-year period since Dr More took over the department, the ratings have risen from -0.13 to 0 and up to 0.07 in 199596. This year, ratings improved to 0.48, almost half a grade better than might be expected.
Something happened to produce the latest result which outstripped previous progress by a significant factor. Dr More pins it down to a core of committed fifth-year pupils who shared a class with students repeating sixth year repeats and those on modular courses. The S5 group appeared to pull up the S6 cohort, who often find motivation a problem. From now on, S5 and S6 classes will be mixed.
Dr More attributes the steady progress at Higher grade to more appropriate targeting of courses, improved team approaches to teaching, revised handouts to pupils on experiments and better equipment. Fifth-year pupils are also asked for their opinions about strengths and weaknesses in the courses. High expectations, good relations with pupils and extra lunchtime and after-school classes are other reasons.
“The main change was the placing of pupils, based on their Standard grade results, either into Higher or modules, level 1 or 2. Pupils were previously going into Higher as a two-year course through the process of failure. They were retrying in sixth year,” he says.
On the value added comparison, measuring progress between Standard grade and Higher, the department has come out well against the national standard with a plus rating of 0.27. Or as Dr More puts it: “We’ve turned pupils into B candidates where the national average was struggling to make a C.” The department is achieving a quarter of a grade more than physics departments nationally.
For Dr More, the key indicators of performance are the relative ratings, value added statistics and the trends over three years. National comparisons come down the pecking order. “I like to look at performance relative to similar departments such as maths. That’s a pretty good start,” he confides.
He checks whether they have taken pupils forward from the prelim and any differences in performance between classes. Information feeds into a report presented to rector David MacKenzie, followed by a de-briefing. “They’re looking for gradual improvement and are prepared to accept you can have the odd blip,” he says.
Dr More believes the relative ratings assist him to monitor the quality of the department, but, with a training in statistics, he is confident enough “not to get tied in knots about them”. Like other principal teachers at Glenwood, an average Scottish comprehensive with a roll of 1,200, he is sceptical about some figures and their influence on determining the overall picture.
Classes with under 10 pupils are not recorded, yet may constitute a significant achievement for some schools. Even the 35 currently taking Higher physics is a small statistical number, he points out.
Mr MacKenzie shares some concerns about the returns and what they reveal, or do not reveal, about departments. But if results are going to be published, he accepts he is bound to be involved in serious discussion with his principal teachers. His school is not any different to others, he argues.
“The results have brought home a sense of accountability to teachers and they’ve realised they cannot blame the catchment area. That’s why we emphasise relative ratings. We’re talking about the same children. How does John Smith get half a grade more in English than history? It’s more meaningful if we can talk about our own children,” he says.
Where trends highlight continuing problems and after talks to resolve issues, Fife’s advisers may be called in to offer curriculum support to individual teachers. “I think we know how staff are performing,” he says.
Mr MacKenzie believes parents do not place too much emphasis on results and will not choose one secondary over the other two in the town because of them. Approaches to bullying, discipline, uniform and family connections may be as important.
Mr MacKenzie is presently mulling over the emerging set of data produced by Fife’s own value added measures on individual pupils. Standard grade results are measured against pupils’ score on the Edinburgh Reading Test in the final year of primary. But findings have yet to be released to staff because of Mr MacKenzie’s continuing concerns about the test’s validity.
However, the value added concept is accepted by Mr MacKenzie and Dr More, provided Scottish Office researchers can establish a fair system of comparisons. In the meantime, principal teachers such as Dr More will continue to monitor the minute percentage swings as evidence of their classroom performance and that of their colleagues.
