There is a line, drawn between pupils who behave acceptably and those who do not. My first headmaster would say, “Aye, laddie, ye’ve stepped ower the line” - as he unholstered his belt. His modern equivalent would use standard English and would only hit the pupil with an exclusion.

Old-timers will complain that the line has moved towards anarchy. Politically correct new managers will say it has moved to become socially inclusive. The latter may even claim that the line has disappeared - but not in my experience.

Despite all the changes of approach and method, the behaviour line remains - with a stubborn and disaffected group of pupils on the far side of it.

To reach them we either have to persuade them to cross towards us - or we have to step over onto their side.

In the past the line was thicker and seldom crossed. Badly behaved and under-motivated youngsters were kept apart. While the conforming sheep were herded into mainstream pasture, the errant goats were chased out of sight - into junior secondary schools and then, with the arrival of comprehensives, into segregated units and residential schools.

Those of us who worked in these segregated units spent a lot of our working lives across the line. We joined our pupils in a twilight zone of bad language, bad habits, bad clothes - an anti-authority conspiracy which we justified by the power it gave us to challenge the deeper problems of damaging and destructive behaviour.

We worked from the inside, taking on new identities to infiltrate the pupils’ territory.

Did we, as often alleged, lose our professional identity in our enthusiasm? I believe not.

But it is significant that I feel embarrassed to admit my scruffy past in these days when to enter a school without wearing a suit and tie is to risk being mistaken for the joiner arriving to fix the broken windows (a mistake that we would have found flattering in those distant days).

Segregation was replaced by integration. Difficult pupils were to be brought into the sheepfold and helped to become behaviourally and socially acceptable. Cynical economics played a part in this change, but there was also a genuine belief that segregation equalled discrimination.

Those of us who had worked on the outside welcomed the chance to try and lead our pupils back into the warm, bright light of the mainstream.

I certainly do not regret that move. It has brought many benefits to disadvantaged youngsters and has forced schols to face the challenge of educating the reluctant as well as the ambitious. Many previously segregated pupils have been given the support to enable them to prosper in mainstream.

Unfortunately, the toughest group got left behind - excluded for behaviour or self-excluded by truancy. They got lost along with the things they needed - the ruthlessly relevant curriculum, the truly individualised programmes, the spontaneous social education exercises. These have all disappeared under the pressure of rigid and uniform exam courses - laudable in intention but regrettable in effect. Another loss, less tangible but more vital, has been teachers’ ability to create close and trusting relationships with very difficult pupils. The closeness of relationships, between adults and youngsters in segregated units, has seemed difficult to replicate in mainstream.

Units unite staff and pupils against the outside world; mainstream divides them into their separate camps. Unit staff can give undivided loyalty to their pupils; mainstream behaviour staff are pressured to divide their loyalty between the often warring factions of pupils and teacher colleagues.

Guidance staff and behaviour support teachers struggle with this dilemma - to side with the pupils and risk the trust of teachers or side with the teachers and alienate the difficult pupils.

The result is often an uneasy compromise which merely loses both.

Consciously or unconsciously, mainstream staff avoid the issue of relationships by changing everything else. Schools deserve praise for their willingness to adapt timetables, establish support bases and initiate positive behaviour programmes.

These measures have retrieved pupils who were just across the behaviour line.

But they have failed to entice those pupils whose fundamental distrust and dislike of adult authority can be changed only by those who get close enough to earn the right to challenge these beliefs.

Two things need to happen. Staff working with difficult pupils have to learn to make effective relationships - and their colleagues and managers have to give them permission to do so. Permission to step over the line, to take the kind of risks taken by staff in units, to bring into mainstream the kind of results - good attendance, willingness to work, acceptance of sensible rules, reasonable behaviour - shown by many units.

These results are achieved only after the right relationship is established. They cannot be achieved without it.

Sandy Peterson was head of the Wester Hailes special unit in Edinburgh and is now a freelance consultant on behaviour policies