How to quarry pupil feedback

THE Office for Standards in Education plans to give pupils more say during school inspections and the thought has struck fear in many a teacher’s breast. Bad enough, they say, to be inspected by the inspectors, let alone the little horrors of 2C as well.

“I’ve got one child in my class this year who drives me to absolute distraction,” says Ginny Heale, a Surrey primary teacher. “She never sits down. She never does what she’s asked. She’s always in trouble. Suppose I shout at her on Friday - and most weeks you’d have to be a saint not to shout at her by Friday - and then the inspectors come in on Monday and ask her how she rates me as a teacher? It doesn’t bear thinking about.”

But the idea - part of a series of proposed tweakings to the inspection system - is one whose time has come. Whitehall has woken up to listening to young people, with health and social services now paying increased attention to what they have to say, and Ofsted on the same path. Its inspectors have always talked informally to pupils during school visits. Now it has taken the first step in formalising this procedure by introducing a questionnaire for sixth-form pupils this September.

However, researchers who are familiar with gathering pupil feedback warn that it is a tricky business, needing the kind of sensitive hand Ofsted is not normally noted for. How you frame your questions is vital, they say, as is the way you collect the responses and what you want the information for. Ensuring confidentiality is essential, while young people also need to believe that someone is going to act on issues they raise.

“Ofsted tends to be seen as a threat to a school,” points out Carol FitzGibbon, professor of education at Durham University, who has been working with pupil data since 1984, “and in that ethos of naming, blaming and praising, pupils can feel very undermined and threatened.”

She believes that collecting the viewpoints of pupils should only be done as part of a collaborative self-evaluation process, and under strict confidentiality. Other researchers point out that pupils, far from being out to get teachers, as some teachers fear, are often touchingly loyal to their schools, and may mask their responses if they feel they are being asked to “shop” them to outsiders.

In reply, Ofsted says there is “no question whatsoever” of pupils being asked to rate individual teachers, only respond to general questions. No primary pupils are to be asked their views, only pupils aged 11 to 16 in secondary and special schools, and no pupil will have to fill in a questionnaire against their wishes. Ofsted says it is sensitive to issues of confidentiality, and to how pupils’ views are reported, and knows that even something as small as the time in the school year that they are being asked for their views could affect the responses. It is looking at major work already done on pupil feedback to find the best way forward. However, it is convinced pupils have valuable things to say about standards, discipline, and overall school culture, and says that the positive responses it has already had from schools indicate that their views are likely to be part of the regular inspection process by 2003, after they have first been piloted.

All of which makes grim reading for the teacher unions, which have warned that allowing pupils to appraise their teachers in the context of an inspection could “wreak havoc” and make teachers vulnerable to the “malicious minority”.

But this minority is a myth, says Mike Johnson, a research fellow at Keele University, which has been surveying pupil attitudes for 12 years. “We must have dealt with 100,000 pupils and we can’t have had more than a hundred of what we’d call ‘spoiled papers’.”

Pupils give “very fair and balanced responses”, he says. They may be preoccupied by issues, such as whether girls are allowed to wear trousers, but they can also raise highly pertinent questions about things such as the relevance of some homework tasks.

Pupil attitudes also contribute a lot to building up a broad picture of what is going on in schools. For instance, the Keele data shows that a growing number of pupils now say their lessons are frequently interrupted by disruptive pupils. It also shows that, although school attainment levels are climbing, pupils’ attitude to their schools are becoming increasingly negative.

However, this is probably not the case in schools which go out of their way to involve pupils in evaluation and development, judging by the enthusiasm shown by those who have tried it.

“It changed the whole ethos of our school. It was a total culture change,” says Bernard Trafford, head of the independent Wolverhampton grammar school, which set out to involve its pupils in the running of the school through a number of avenues including a vigorous school council. “Our students feel empowered. They get drawn into everything from little daily things, to major things like building work. It has had a fantastic effect on the school.”

Kate Griffin, head of Greenford high school, Middlesex, agrees. She involves her pupils in everything from filling in questionnaires to help subject departments review their teaching, to helping interview candidates for teaching jobs. She says: “They are absolutely on the ball. They know their views are valued and they respond very positively.” As for teachers’ fears of what pupils might say of them: “If they are doing their job properly, they’ve nothing to worry about, have they?” But herein lies the crunch. Because without a doubt the most powerful information that pupils hold is that they - and only they - know what it is like to be in individual teachers’ classrooms. But this is exactly the area that teachers are most worried about, and which Ofsted is determinedly steering clear of. And understandably, since, according to Mike Johnson, when Keele’s confidential surveys ask questions like: ‘do you find this teacher’s lessons interesting?’, “the results can be eye-popping”.

So how can schools quarry this priceless information without upsetting everyone in the process?

By using it in a voluntary professional development programme, says Russell Hobby, manager of Transforming Learning, a project launched last year to allow teachers to use pupil feedback to improve their skills.

Transforming Learning involves a cross-section of pupils completing an online questionnaire to analyse a classroom teacher’s performance. The results are confidential, and fed back to the teacher via a chart which can highlight gaps between how the teacher perceives the classroom and how the children do, as well as between how the children see it, and how they would like it to be. However, they are not asked to rate their teacher, only to comment on the classroom environment.

“Children hate the idea of being asked to appraise their teacher. So we tried lots of different ways to get their viewpoint. But we’re convinced you have to, because we know that, done correctly, it leads to a better climate in the classroom.”

The project was launched last year by the Hay Group, the management consultancy that produced the leadership programme for serving heads, and costs a school between pound;250 and pound;2,000 depending on its size. It has already been used by 3,500 teachers and turned up interesting information showing that in the area of fairness, for primary schools, and safety, for secondary schools, teachers are not doing as well as they thought they were. However, in other areas, such as making lessons clear and interesting, and being supportive, pupils rate their teachers more highly than they rate themselves.

Richard Morris, deputy head of Queensbury upper, Dunstable, a 13-to-18 comprehensive, has piloted the scheme among staff volunteers and is now introducing it for the whole school. “This is the kind of feedback you can’t get in any other way. It’s unique. If someone comes in to observe you teaching, the whole climate changes so they’re not seeing what they’re supposed to be seeing.”

When he was put under the spotlight himself, his pupils’ perception of his teaching tallied closely with his own. “But it still made me think harder about the areas I want to work on. One was having higher expectations of students. In the past, if someone hadn’t done their homework as well as I thought they could, I might have not wanted to discourage them and let it go, but now I’ll probably come down on it harder. And they knew it was like that. They rated me like that, students always know.”

So can Ofsted hope to do anything half as useful with its own stab at pupil power? It seems unlikely. Even so, giving pupils any sort of bigger say in schools is probably a move in the right direction.

And if this first step goes smoothly, primary pupil surveys may well be added to the Ofsted process later.

SIXTH-FORM QUESTIONNAIRE

Ofsted put 15 statements to sixth-formers and asked them to rate their responses from “strongly agree” to “don’t know” to “strongly disagree”. Students had room to make additional comments. This is a selection of the statements:

* I was given helpful and constructive advice on what I should do in the sixth form * I am taught well and challenged to do my best in all or almost all of my subjects or courses * My work is thoroughly assessed so I that can see how to improve it lTeachers are accessible to help me if I have difficulties with my work * Outside my main subjects, the school provides a good range of worthwhile activities and enrichment courses * I feel I am treated as a responsible young adult in the school

WHAT THE CHILDREN SAY

importance of pupil feedback:

“The teacher affects whether you like a subject; you might have liked it before but how that teacher is can mean you don’t like it now.” Nick, 14 “This is an opportunity to tell someone how you feel. Normally you’d just tell your mates, but now you can tell the teacher how it feels.” Harsit, 14 What makes a good class:

“The teacher knows how to talk to the children.” Melanie, 10 “You work together and listen to each other’s ideas.” Rachel, eight “When you get on with things without any fuss and you ask the teacher if you don’t understand.” Tony, 11 What makes a good teacher:

“If you’re naughty they tell you off, but if you’re not they don’t” Tommy, eight “Someone who knows what they’re doing.” Billie, eight “Listens to the students and knows what they mean.” Lauren, 11 “Helps the children with difficulties, like if they can’t hear properly they don’t just shout at you.” Bradley, 11 Pupils interviewed for the Transforming Learning project