It is 40 years since I failed my first behaviour test, after the pupils of Niddrie and Craigmillar chewed me up and spat me out on the lino floor of my English classroom. Into the dustbin, by the end of my first teaching day, went a year’s worth of Moray House theories about exerting effortless control through my skilled teaching, my inspirational material - and, of course, my sparkling personality.

Stripped of any such delusions, I set off on a long search for the thing that would make youngsters behave. I haven’t lacked for advice during my quest. Everybody on my bus knows exactly what should be done with bad children - and these social commentators aren’t shy to share their opinion.

“Shoot them, flog them, lock them up” - and even, occasionally, “love and understand them”.

It’s easy to laugh at the layman’s crude philosophy, but, if we are honest, the professional view of how to manage behaviour has not been much more coherent or balanced. Our behaviour policies and strategies have been full of contradictions, inconsistencies and somersaults. Amid the confusion, however, it is possible to identify one continuous theme - namely the battle between the “dictators” and the “persuaders”. The dictators want to tell children what to do: the persuaders want to let children decide what to do. The battle has gone through phases. These have been: control by brute force, gentle persuasion, segregation, inclusion and “everything and nothing”.

Control by force, led by the dictators who were in charge 40 years ago, usually disregards justice and ignores feelings. Its perceived strength is its simplicity and its effectiveness in deterring and eliminating resistance. There is no doubt that, overall, pupil behaviour was less challenging in the 1960s. But right-wing nostalgia conveniently forgets that, by the time corporal punishment was abandoned, it was already failing.

Gentle persuasion was based on the assumption that, if children could be helped to identify the reasons for their bad behaviour, they would be motivated and able to change. While variable in its effectiveness, this method has worked for the majority of youngsters, whose secure upbringing has made them capable of participating in the process. Significantly, however, it has failed a minority of young people whose background of instability, insecurity and abuse has rendered them incapable of engaging in a process of discussion, analysis and decision-making.

In the 1980s, the segregation of problem pupils, from mainstream into “units”, was seen to benefit everybody - mainstream teachers and pupils would get peace, and disaffected pupils would get education appropriate to their needs. While the radically different education provided by units was, and still is, right for some children, there are drawbacks. The price of segregation to a youngster is to be marked as different, a stigma that may last, in the minds of the pupil and other people, far beyond school. It may also encourage retreat into the comforting company of fellow failures.

The balanced view of segregation units is that we need them, but must put into them only those for whom the advantages of a different approach outweigh the disadvantages. Units must not alienate, but they must not be undemanding either. As one of those who ran a unit when we were initially, and justifiably, accused of “rewarding bad behaviour”, I think we were able to find the balance - using the closeness of our relationships with the pupils to challenge them to change their behaviour. The positive response to this, from very difficult youngsters, surprised me - and has guided my method and my thinking ever since.

The next phase, inclusion, is a bit like integration - difficult to dispute since to deny it is to seem to favour the opposites, exclusion and segregation. There is also validity in the argument that some schools and some teachers are too quick to segregate any individuals who are reluctant to conform to institutional norms. The mistake has been the usual political one of applying a single, blanket solution to a myriad of very different problems - then enforcing that solution with eyes shut and fingers in ears.

Proponents of inclusion failed to appreciate that some children are so disturbed and unstable that they cannot function in a mainstream classroom environment. Compounding this first error, they then gave these youngsters untrained support staff, who were expected to control the behaviour of those who had defied the best efforts of teachers, social workers, youth workers, psychologists, panel members and parents.

It is pertinent to ask where inclusion fits into the “dictate versus persuade” debate. At first glance, it seems like a child-centred policy, securing a child’s right to be the same as everyone else - and offering support to ensure that it happens. However, for many of the most disruptive pupils, it has become a failed control system. Too often the control fails, the support is rejected by the pupil - and disruption increases.

Finally, there is “everything and nothing”. The failure to acknowledge or resolve the conflict of ideas means that there is, currently, no model of practice. Everything is right and nothing is wrong. Inclusion proceeds doggedly - but exclusion numbers rise and segregation units are full.

Behaviour projects and initiatives proliferate, but, lacking a guiding philosophy, some are good and some are truly awful.

This is a bleak assessment that will anger many conscientious people who work tirelessly for young people. But that makes my point. There is no lack of effort - but there is a lack of proper evaluation of evidence that would help us to reach the right conclusions. Working with challenging youngsters is so damned difficult that nobody wants to criticise those who are willing to do it. As a result, well-meaning but ineffective methods are tolerated and encouraged.

The lesson to be learnt is that children will not become peaceful until adults stop fighting among themselves. Perhaps each side can learn from the other. Behaviour change is achieved by people with the right relationship and the right method. The persuaders are already skilled at building the close relationship. They then need the courage to use the methods that will turn that relationship into a force for change. This must be change that is measurable - and change that will satisfy even the most grumpy dictator.

Sandy Peterson heads the Practical Training and Support consultancy.