The latest Ofsted annual report states that behaviour is good or outstanding in the vast majority of primaries and secondaries, with less than 1 per cent considered inadequate.
So how we can reconcile that with statistics showing that 17,000 pupils were expelled for physical attacks on adults last year? Did they all take place in that small proportion of schools where behaviour is officially inadequate?
I have conducted several studies in the area of "classroom climate" and the results suggest that behaviour is a problem in far more than 1 per cent of schools. And the disruption teachers talk about is not limited to low-level incidents such as pupils not putting their hands up or leaving their seats without permission. Many schools have to deal with pupils who are very difficult to manage, and teachers often have to make tough decisions if they want keep everyone in the classroom while preserving a climate that is ideal for learning.
Some of the research involved using a 10-point scale (see panel), with level 10 representing a climate which is perfect for learning, and level 1 representing an atmosphere where behaviour makes learning virtually impossible.
Nearly all the teachers I interviewed said there were times when they struggled. Although there were many schools where the bottom three or four levels on the scale did not occur, most respondents recognised the intermediate levels.
In those lessons, pupil behaviour would limit not just learning and outcomes - it would also affect preparation, as some planning would be directed towards keeping control rather than learning.
We surveyed more than 200 student teachers, asking them to reflect on what levels on the scale they had encountered as pupils. More than 90 per cent reported that they had sometimes been in classrooms where poor behaviour had limited learning, and a similar proportion felt they had encountered level 6 or below.
Even very experienced and accomplished teachers, who were widely acknowledged by their peers as being as good as it gets in terms of managing pupil behaviour, talked of working below level 8. One of these was an Advanced Skills Teacher working in a challenging school, who said it had been a "minor triumph" reaching level 5 with a particularly difficult group.
None of the teachers I interviewed suggested that it was becoming any easier to create an ideal working atmosphere in the classroom. But many experienced teachers also stressed that a golden age, when all pupils did as they were told and all parents gave their children a clip round the ear if they had been reprimanded at school, is a myth.
How has behaviour changed?
I am often asked whether behaviour has got better or worse. When I first taught, in the Seventies, corporal punishment was still used: pupils had to hold out their hand to be "strapped" with a leather belt, or (less officially) to bend over to be slippered across the backside.
Corporal punishment did not guarantee good behaviour: I recall looking through the door of a classroom to see a teacher threatening her class, "If you throw one more thing .", then seeing her hit by a volley of books.
Some researchers have suggested that western culture, with its emphasis on liberal individualism and the questioning of authority, has meant a decline in deference and compliance in schools in many countries. Professor Joe Elliott has argued that even in countries with strong traditions of obedience in schools, such as those in eastern Europe and the Pacific Rim, respect for authority, and for teachers in particular, has declined.
Successive reports from the Organisation for Economic Co-operation and Development and other major comparative surveys suggest that behaviour in UK schools compares unfavourably with many other countries. This is attributed, at least in part, to differences in culture and attitudes: parents and pupils value education and teachers more highly outside the UK.
Computer gaming has been cited as a harmful influence on pupils' attention spans, and mobile phones have made new forms of misbehaviour available to pupils. Others have blamed an increase in poor parenting, with many pupils arriving at primary school lacking in social skills and the ability to conform with classroom culture. One recent study suggested that there were more than a million severely neglected or abused children in the UK.
It is difficult to prove conclusively whether any particular aspect of behaviour has got better or worse, given the number of variables involved. But the question should not be how behaviour compares to the past, but how it affects schools now.
Pupil feedback also suggests that the problem is not limited to a handful of inner-city schools. In a survey of more than 700 pupils in which they were asked to comment on what factors most put them off being in a classroom, 104 responses related to poor classroom climate, with comments such as "Other pupils disrupting", "When teachers can't control the lesson", "Getting stuff thrown at me", "People throwing things round the room", and "People disturbing others on purpose".
A higher proportion of pupils are aware of their rights today. Physical presence, a booming voice and the capacity for physical intimidation are no longer particularly relevant assets, as pupils are aware of what teachers are and are not allowed to do. Teachers today need to rely on their intelligence, skills of interaction with pupils, good judgment, adroit use of school systems, and collaboration with colleagues.
What should we do now?
The danger in suggesting that the situation is unsatisfactory is that any deficit is simply blamed on "inadequate" teachers or "bad" schools, particularly those working in challenging contexts (it is the "floggings will continue until morale improves" response).
Level 10 is not a natural state of affairs. It takes considerable skill to get all pupils to commit wholeheartedly to learning in some classes. I am arguing for stronger and more concerted support for schools from parents, governors, local education authorities and government. There is a need to change culture as well as policy and practice.
The overwhelming majority of pupils, parents, teachers and policymakers want classrooms to be calm and ordered places where all pupils can learn. But until there is a full acknowledgement of the scale, nature and complexity of this problem, these deficits are likely to persist.
The working atmosphere in the classroom has an important influence on pupil attainment. Countries with good classroom climate consistently perform well in international tests of educational performance. It is also an important factor in teacher retention and the quality of teachers' working lives - there are very few things in professional life less edifying than being, in effect, locked in a room with 30 children not fully under your control.
It is difficult to enjoy teaching if the working atmosphere in the majority of your classes is below, say, level 6. Conversely, teaching is very fulfilling if you are working at levels 9 or 10.
As one newly qualified teacher put it: "In terms of how much you enjoy your teaching, there's a massive difference between operating at levels 7 and 8, which are OK, no big hassle . and level 10, when it's just a fantastic job, pure pleasure, and you can get a real buzz out of the interaction with pupils. It's like the adverts for teaching on the TV, but in real life."
The fact that many pupils are not in classrooms that are under the relaxed and assured control of their teachers is a major source of inequality of educational opportunity in the UK.
Of course, good schools and good teachers make a big difference to behaviour. I believe strongly that the complex and sophisticated skills that enable teachers to cultivate a perfect classroom climate with groups of all ages and abilities are one of the most important factors in influencing pupil behaviour. There are not just differences between schools in terms of classroom climate; there are differences within individual schools.
The behaviour policies of the new Government focus, seemingly, on empowering heads to exclude. Several teachers I interviewed felt that their school was under pressure not to exclude, even when there was quite strong evidence that the pupils were regularly interfering with others' learning. But none of the heads I interviewed felt that "zero tolerance" was either morally defensible or practical.
As one pointed out: "Zero tolerance in someone else's school usually means that the pupil ends up coming to this school, and we already have more than our share of difficult pupils."
It is sometimes said that there are not enough good schools, but part of the problem is that there are not enough good pupils - young people who are perfectly socialised, keen and quick to learn, who value education and have fully supportive parents.
I think many parents and policymakers would be surprised at how much teachers have to put up with, and how difficult it is to establish and sustain a classroom climate that is perfect for learning.
Under the current system of league tables, Ofsted reports and parental choice, discipline and behaviour are sensitive issues for heads and governors, and it is sometimes difficult for them to be completely open about the problems.
But if we are to gain for teachers the support they need to secure the right to learn for all pupils, we must first acknowledge that this is a real and difficult problem.
Terry Haydn is a reader in education at the University of East Anglia and the author of "Managing Pupil Behaviour" (Routledge).
- Related article: Behaviour now - The whole school - Better discipline is part of all-round improvement
HOW DO YOU FARE ON THE SCALE?
10. You feel completely relaxed, able to undertake any form of lesson activity without concern. You and the pupils work together, enjoying the experience.
9. You feel completely in control and can undertake any sort of activity, but you need to exercise authority at times, in a friendly way, to maintain a calm, purposeful working atmosphere.
8. You can maintain a relaxed and co-operative working atmosphere, but this requires thought and effort at times. Some forms of lesson activity may be under less control than others.
7. You can maintain a co-operative working atmosphere and undertake any form of classroom activity, but this requires more considerable thought and effort.
6. It is often a major effort to establish and maintain a relaxed, calm atmosphere. Several pupils will not remain on task without persistent surveillance, exhortation or threats. It is sometimes difficult to get pupils to be quiet while you are talking, but there is no major disruption.
5. Your control is limited, and there are times when you would be embarrassed if the head walked in. The atmosphere is rather chaotic at times, with several pupils manifestly not listening to you. But pupils who want to work can get on with it, albeit in a rather noisy atmosphere.
4. Your control is limited: it takes time and effort to get the class to listen. You try to get onto the worksheet or written part of the lesson fairly quickly in order to get their heads down. Pupils talk while you are talking, and minor transgressions go unpunished because too many occur. You try to keep a lid on things and concentrate on those pupils who are trying to work.
3. There is major disruption and many pupils pay little attention to your presence. Swearwords may go unchecked and pupils walk round the room at will. When you write on the board, objects are thrown around the room.
2. The pupils largely determine what goes on. You take materials into the lesson, but once distributed they are ignored, drawn on or made into paper aeroplanes. When you write on the board, objects are thrown at you rather than around the room.
1. Your entry into the classroom is greeted by jeers and abuse. There are so many transgressions of the rules it is difficult to know where to start. You wish you had not gone into teaching.