Raucous and unruly boys are distracting teachers’ attention from troubled girls, David Bell, the chief inspector, said.

Pupils self-harm in about a third of primary and special schools and about half of secondaries, an Office for Standards in Education report on behaviour found.

Overall, behaviour in primaries has improved since 1997. It is good or better in 90 per cent of schools, though children as young as five are being educated in special units, it said.

The proportion of secondaries where it is good or better has fallen from three-quarters to two-thirds. Behaviour in one in 10 secondaries remains unsatisfactory.

Mr Bell said some schools’ failure to tackle bad behaviour was “worrying”.

School discipline has become a political football in recent weeks with the parties setting out pre-election plans to ease public concern.

The report said drug abuse is a “daily challenge” in some pupil-referral units and colleges, although it is less of a problem in schools.

One in five of the 15 secondaries visited by inspectors for the report said gang culture was widespread but could offer little firm evidence to support their claims.

“Acts of extreme violence remain extremely rare” and most indiscipline amounts to no more than low-level disruption, the report said.

Mr Bell said: “Although the large majority of schools are orderly places where children behave well, it is worrying that unsatisfactory behaviour has not reduced over time.

“Unsatisfactory behaviour by a minority of pupils causes nuisance and distress and disrupts the learning of others and I hope this report helps early-years centres, schools, colleges and LEAs tackle it effectively.”

Derek Twigg, junior education minister, said: “Of course, tackling perceptions, even where there is little evidence, is important.

“We fully back heads in taking robust action against any violent behaviour, including permanent exclusion.

“We are developing new powers for heads to search pupils suspected of carrying knives.”

The report is based on inspection and research evidence and visits by HMIs to 78 schools, colleges, early-years settings and pupil-referral units.

The inspectors found that despite overall improvements in primaries, schools face particular problems with children aged four to six who are “ill-prepared socially and emotionally” for school.

After the age of nine, incidents of bad behaviour increase sharply, with particular problems emerging among pupils aged between 11 and 14.

Many disruptive pupils have special needs, poor language skills or come from disadvantaged or dysfunctional families.

The report stressed the need for effective teaching and a curriculum which engages potentially disaffected young people, but warned that a quarter of the institutions visited failed to reach the required standard.

It also makes clear that schools need the support of parents and other services to win the battle against bad behaviour.

Less than half of schools and local authorities have established good partnerships with health and social services.

Tim Collins, shadow education secretary, said: “Parents and teachers might care to reflect that it was this Labour government that gave appeals panels the power to override headteachers on expulsions, forced every school to take disruptive pupils and failed to protect teachers against malicious abuse claims from pupils when they were just doing their job.”

* jon.slater@tes.co.uk

Managing challenging behaviour is available from www.ofsted.gov.uk